I had to refresh from beginning with 820,000 blocks. Monero currently has low transaction volume, if we have would ever gain significant adoption and number of transaction shoot up 10 folds you'll see the problem with refreshing (together with syncing database and disk space usage).
Currently it does have low transaction volume. Early on it had huge transaction volume because the newly-developed pool software was very stupid in how it did payouts (every single miner no matter how small was paid out on every single block). Later there were two spam attacks. When refreshing from the beginning you will have to scan through that first several months with high volume before reaching the current, relatively low, volume.