If only the recipient (and sender) can read a transaction, how can you verify that the transaction is valid and based on previous unspent transactions (that you can't read)? You then have to be able to verify those previous unspent transactions are valid by checking the previously unspent transactions they depend on too.

I can't imagine how any sort of anonymous system could work besides one that abused BIP 32 or used Zerocoin's innovations.