The (potential) trouble is that you cannot get blood from a turnip.  The exchanges are one screw-up away from being insolvent, and your Yubikey is only partial protection against one form of possible problem.  Should any one of a number of possible problems happen it will not matter what the disposition of the operators might be.

For my part, if I have excess funds in the form of BTC, I take them home ASAP.  And put them in encrypted storage wallets with backups and all that jazz.  If/when I want to sell, I'll crack some of the savings wallets open and transfer them right back.  I limit what cash I store with the exchange (Tradhill primarily in my case) to a realistic minimum.

If an exchange wants to hold my BTC, they can pay me interest.  In fact I would be happy to let them hold a portion of my stash to perform futures operations and what-not...but I want a cut of the action.  And at this point none of them are even close to meeting my criteria for confidence in this part of the game.

But, of course, do as you see best.  These are just the ways I do things.