Frank looked on in disbelief moments later when Konsa flew into James after being beaten to Yoann Barbet’s loose pass.
The defender had already been booked, but Stuart Attwell showed him a straight red card.
Celina further punished their indiscipline after 67 minutes when he tricked his way past Segi Canos and Julian Jeanvier before beating Daniels at his near post.
Swansea thought they had a fourth a minute later when Celina’s shot was saved into the path of Connor Roberts, but his follow-up was wrongly ruled out for offside.
The fourth finally came in the last minute of normal time when Byers side-footed in from the edge of the box to finish the job with his second goal in two weeks.
“We made two big mistakes which we could have avoided, and we gave the ball to James, who is the only player in the country who shouldn’t be given the ball 30 yards into his own half,” said Frank.