by the second party not signing. you cant have them back. because it requires dual signing.
by this i agree the second party cannot simply "take it from you" without your signature, but you cant take it without theirs
IIRC there is a timeout period after one party initiates the closing of a payment channel. Please take this with a grain of salt, as I'm unsure and this would be a *stupid flaw* to enable the possibility of indefinite state of irresoluteness. I'll do some research on this once I find more time.
I have read somewhere in the forum that the lock out time if in case there is an uncooperative channel is 2 weeks. It would be stupid of the developers not to put this feature, making it possible for your coins to be locked forever. I would suggest that the person opening the channel should be able to choose how long the minimum time out is as long as it is not less than 24 hours. I believe that would be better.