In case it was not clear, in a 51% attack the attacker changes the "tip" of the chain not some random blocks deep into the chain. keep in mind that what we have in Bitcoin is a block-chain and if anything, even a single bit is changed in any old block, that block and the whole chain after that block become invalid.
An attacker with more than 50% is able to replace older confirmed blocks because they will eventually catch up and surpass the honest miners with a new longest chain.
Maybe it works better if I explain it with a picture. My photoshop skills aren't the best but here is what I was talking about. The attacker would have to change the chain from the tip down and a random block cannot change: