I'm pretty sure that when TF is fully activated, blocks will be 1 minute apart, no matter what. C-f-b mentioned every block would have a "group" of forgers (I believe it was 3) eligible to forge the next block. So they would all have to drop out at the same time to simply delay a single block (unlikely). This also has the side effect of being unable to guess who will truly release the next block within the group, thus make it hard to game your own transactions.
Ah thanks, that makes more sense. And that means they can check on each other and make sure all transactions are processed (instead of selectively including them).