Larimer thinks you can have anonymity in such a system already:
For once I agree with you, if most transactions take place off chain/ledger/whatever then the majority of transactions are "anonymous" as they are not publicly recorded.
Smooth and I discussed this I believe in 2014 and the conclusion is that everything sent to the internet can be recorded, so there is no such thing as off chain anonymity (CoinShuffle excepted, and also CoinJoin if jamming and DoS is not your worry) if you are referring to protection against national security agencies and government capital controls.
And if for a business or high net worth individual, then you may also want to be very safe against espionage and mobsters, so again your upstream ISP, masternode/delegated node, or what ever might be compromised.
Also one of the most important points is that only on chain anonymity obeys the End-to-end principle of networks. The means your anonymity is orthogonal to any agent in the network. This is critical for scalability, redundancy, and resilience.
So please enough with the off chain anonymity. It is highly inferior. It is a hack that got some play in terms of quick way to get anonymity rolling (e.g. Dash), but it is not the future. If the internet had been invented without the end-to-end principle, then TCP/IP wouldn't work and we'd not have the scalable, resilient internet we have today that enables to even be here.
Except for CoinShuffle, Off chain mixing = trusting someone (node/server) you can't prove you can trust.
I put anonymous in quotes so as to highlight it as a hack, compromise whatever. Its not a real solution of course, but for most end users its sufficient so that regular people you transact with cant find out all of your transactions.
With unscalable networks its a mute point anyway, because you cant store all of the transactions on-ledger, which is why I agree with r0ach and smooths arguments that scalability is more important in the long term end goal.