if you dont hold the private key, the coins are not yours
a century ago if you dont have actual gold. the bank note promissory note is telling you that you dont own gold.. you dont even own the paper. you just 'bear'(hold/keep) the paper that can be exchanged for gold.
That still applies to gold today.... if a government introduces new regulations where private ownership of the metal is prohibited, people will be blaming themselves for ever having trusted a bank to store their gold with.
In most cases the gold inside bank vaults is insured as well, and for your gold to be insured you must declare the quantity and all, which means that not only the bank, but also the government knows what you hold if they ask for it.