This last N increase has hit my CPU mining hash rate hard, around 42%. It would be interesting to see how it hit GPU mining to see if the algorithm is going in the right direction.
Fortunately, difficulty will drop accordingly over time. It's too bad it doesn't rise or fall with a larger increment (faster rate) per block though. It's just a bit of a waiting game at this point for difficulty to become more reasonable. It's dropping quickly though, I see it's down to about 4.4 now.
I'm a bit concerned that N might be getting too big even for CPUs though. My single core EC2 micro-instance was capable of mining (at a slow rate) before the N increase, but now it's pretty much unable. Wondering what the rest of N increases might cause, perhaps the pools need to start adding variable difficulty to cater for slower CPUs or the next N increase might kill a good amount of lower-end ones.
I think at a minimum, pool operators need to stay on top of Nfactor++ events and adjust work size / difficulty to keep things working smoothly and keep the average time to solve a share below the average time between blocks. Otherwise everyone solo mining (like me) actually beat out pools with poorly adjusted work size. I still leave a handful of dual Xeon servers mining for testing purposes and their block solving rate still seems to be tracking inversely proportional with hash rate and difficulty. At this point, N is still way lower than Litecoin always had.
An Amazon micro instance mining on a poorly adjusted pool would indeed be a losing battle at this point, I'd think. That's basically just a small shared fraction of a single Xeon core!