But then why does it need to redownload everything to rescan an address ?
Full nodes that need to rescan the entire chain to check the balance of a new address don't keep the entire UTXO set. Remember that you should keep only your own UTXOs if you're using Bitcoin for personal use. You need all the UTXOs only when you want to analyze the chain or use commands such as
getrawtransaction.
I suspect that if you set
txindex=1, you won't have to rescan and wait for so long, whether you're pruning or not.
But then if you dont have all the utxo you cant validate the transactions in New blocks either ? ( unless they are related to address in your wallet or spend utxo from block ).