You should have used an escrow, I don't see why the other party needs to refund you for your loss. I certainly wouldn't.