Start using a better wallet.
I know certain wallets may automatically generate new addresses for each transaction, there are better options that provide more user control. For instance, I switched to using the Electrum wallet which lets me view all my used and unused addresses, and even select which address to send transactions from. This extra control helps me to more easily monitor all my bitcoin funds across addresses.
My 'Ledger X' has an option to use Electrum as an interface to itself. But multiplying the amount of addresses is making it messy to conclude total balance and increases amount of fee unnecessarily.