Yes, you are correct.  They are there so some quantum computing farm (doesn't exist, but...) can't come out of nowhere and roll back blocks by mining a long chain from far in the past.

The whole point is to thoroughly check all the hashes from block 0 to today, that is how it's secure.  It only has to check all of them once though, and then trusts that nobody will tamper with the disk data to invalidate that check (safe assumption, I think.)