Theoretically, though, I am not sure if the attacker couldn't try to fake the timestamp (he would set the timestamp of the first "private" block around one hour to the future).
That means that median of the last 11 blocks in the block chain. If the miner mines on block 100,000, then he only has to set his timestamp after the median of blocks 99,989 - 99,999. He doesn't have to worry around the main chain at all.