Yeah, it definitely doesn't take that long at today's block size, I'm just thinking ahead to when they are much larger. I'm trying to come up with a core design that is able to elegantly handle large blocks, if the network gets to a point where large blocks are being used.
Sorry if I wasn't clear It shouldn't take much time regardless of the size because most of the transactions in the block are already validated and waiting in the mempool. Before the software cached that validation it was taking a fair amount of time, but not anymore. Getting most of the validation off the latency sensitive critical path is a big improvement.