I mean what if the NSA makes an all-out attempt to bring monero down. Political hysteria about dark markets, ...could lead to that, in the worst case.
It would be a good problem to have. Being a speculative vehicle on one u.s. crypto exchange isn't going to suffice. I look forward to the day that XMR is strong enough to warrant a crypto arms race. The trading ops will be stupendous.
If you really want to worry, worry about core devs getting coopted (bribery, extortion), or quantum crypto destroying all the ECC infrastructure. That is what troubles me.