My main fear with Monero is that the government just bans exchanges from listing Monero. Sure you can use it, but there will be almost no place where you can exchange it for fiat (except meeting in a shady alley with a hooded guy bringing cash for your Moneros 
).
).Sooner or latter we will use only decentralized exchanges. Atomic swaps will happen cross blockchains so from one coin to another so that will basically be an decentralized exchange.
I am not saying that sudden BAN from all 180 countries that happen tomorrow would not hit Monero adoption. But this will for sure not happen. And even if it happen would have no big impact in years to come. Most likely a positive advertisement same as it had on Bitcoin in 2011 and latter.