As long as you expect to get at least a few blocks every 2 weeks between diff changes it's OK.
Since the last diff change we've so far had ... 3 blocks.

Our highest Diff block is 928% last October.

Time isn't a factor in the calculation - time is a calculation on the Diff value depending on the pool hash rate.

Yes you get more blocks more often on a larger pool and less variance, but that also means higher fees and long term less reward.
We did just have WAY better luck for a month than any other pool so it's pretty much irrelevant.

Have a read of Help->Luck for anyone who wants more info.
We've so far had NO blocks (before this one) in the last 108 that were over 400%
CDF stats, as shown on that page, says we should average, over a large number of blocks, 1 in 54.6 blocks over 400% so we are doing well there also.