It would be fantastic if we could take a note from DMD's book and use a bit from their book. POS takes up so little resources on my DMD wallet and the coins are automatically regrouped after staking. Makes for a much easier time loading the client.
Reason Diamond loads faster is it has a smaller blockchain to load into memory.
Same reason new coins load faster smaller blockchains.
Diamond has less than 2 million coins total, Block Speed 100 seconds ,
VS
HBN has over 21 million coins with a Block Speed of 30 seconds
(Which means over 3x faster speed and 3X the ability to carry more transactions than diamond, side effect to that is a larger blockchain which is why it takes up more memory resources) Algo might make some difference HBN scrypt vs Diamond groestl, but probably not enough to worry about.

The coin count itself does not really matter. Each 'coin', or unspent transaction out, has value represented in 64 bits. Whether it is 100 coins or 1 million coins, it will occupy the same 64 bits, the only difference is where the 0's and 1's are placed.
The amount of blocks and the algorithm are really the biggest determining factors.