If timestamps were not validated, I think it actually would be a problem because attacker could build a very long chain very quickly and broadcast that whole chain.
No, the attacker cannot build a longer chain. Re-read my
first post, I don't know how to make it any clearer.