You can only be forced to do so if there is a record of the transaction taking place. You may be legally obligated to pay taxes after selling some gold to your buddy, but that doesn't necessarily mean the law is enforceable in that situation.
So you basically say people are not forced to use fiat, because they can violate their "legal obligations" ?