It seems like the analogous situation in bitcoin is being fed shorter block chains that were dropped. When synchronizing we don't give shorter chains to people because it's irrelevant, we only give people the longest one a.k.a the contemporary state.
What would voting history give? What if they only replay certain votes? History is really unnecessary.
In bitcoin you can give any node any set of chains - the node simply chooses the longest one; this is trustless and always consistent.
If you had voting history in raiblocks, at least the synchronizing client could make some kind of judgement based on several forks which it is presented with. As it stands, it has to ask other nodes which is correct, which is not a trustless operation and on top of that, said nodes must expend their own resources to re-vote on the historical forks (the DDOS angle, I've previously mentioned).