in the event a honest guy tries to reference a legit tip and the attacker's tip, he'll detect the contradiction and won't do it. Therefore, the attacker's subtangle will be abandoned.
This requires not just an honest guy, but a diligent one as well.
The risk is that honest guys will be lazy and rely on others to go far back in history to check all tx for double spending.
Even in bitcoin we've seen some miners being too lazy to do full node checking and instead relying on SPV, and thus risk building on top of invalid blocks.