1. People who don't understand crypto will think, that you are just trying to lose your money, or you did a bad thing
2. IRS can force you to sell your crypto - really there is an article https://news.bitcoin.com/irs-tells-couple-with-7-million-in-bitcoin-to-liquidate-crypto-assets-and-pay-off-tax-debt/
3. You can be liked with some coins which are from some crypto "robbery", but you bought them legally