Why do you think that? SPV clients are hardcoded with the contents of the genesis block just like any other node. There is no difference between the two. If you are following a chain which doesn't begin with the Bitcoin genesis block you are not part of the Bitcoin network.
Ok, I've got the wrong definition of SPV. The actual definition means they check blocks back to the genesis block.
In reality, bitcoinj (the most common SPV client) only downloads headers since the last checkpoint (included in a file shipped with the client). I was under the impression that this meant that by definition SPV clients don't have to check all the blocks just enough to have confidence that they are on the real chain.