One of the suggestions for reducing the UTXO set is to put a cost on holding lots of UTXOs. If there was a limit on the total number of UTXOs (like with the blocksize), then transactions which reduce the UTXO could be free.
A miner might be allowed to add 1000 UTXOs to the output set per block. If he has already reached his limit, he could include a transaction which spends 10 UTXOs to a single address and then include 9 transactions which increase the UTXO set by 1 each. The 9 transactions would be fee paying, but unless he includes the first transaction, he can't include them, so he may include it for free (or lower fee).
How are DoS attacks prevented? Say the network is flooded with these output creating tx.
If I'm not mistaken you'r not a UTXO until after some time you remain unspent. We already have the more confirmed tx as input is priority and can even be sent with zero fee, so I don't see the advantage here.
One issue with UTXO is old transactions never being spent and thus not prune-able on partial-nodes. Though this form of check-pointing adds to the overall blockchain size and this doesn't help at all on a full-node.
What are the other issues output script combining with transaction to self meant to solve? They add more issues with wallets not knowing the txids of the spendable outputs.