Especially if you are in the US.  The IRS can audit anybody, at any time, for any reason or no reason at all.  And being audited is not cheap.  The lawyers' fees for proper representation can easily be far more than you pay in taxes.

Make no mistake, being associated with bitcoin in any sort of government or banking database is just begging for a third-degree rectal audit.

The next time somebody gives you that "nothing to hide nothing to fear" assclownery, tell them you just don't want to be audited.  Not wanting to be audited is not only perfectly legal but sensible too.