I've been doing some testing with larger long-term staking wallets and it seems limiting the coin fragmentation is only part of the problem. Another huge element turns out to be the fact that the wallet loads your entire transaction history when you open the wallet, including all of your stakes and orphans, regardless of your filters. An experimental wallet I've been working on that only loads the last weeks worth of staking history (full history of send-receives loads normally) has had dramatic results for responsiveness on wallets that have staked tens of thousands of times.
There's still some cosmetic bugs I'm trying to squash, but implementation ideas are welcome. At the moment I've just hardcoded in a week and put an option to enable full histories in the Settings -> Options -> Display window.
Personally I don't see the need to pull up stakes going back further than a week unless your stake history is rather light in the first place (hence the option to disable it). But if people want more control over that I can implement a slider or textbox to enter the number of days to load stakes.
I'm also toying with the idea of implementing an option to disable staking notifications - so heavy staker's can still enjoy standard send-receive notifications without being annoyed every few minutes with a staking popup.
Thoughs?