Incrementing N will lead to immediate hashing speed decrease. Difficulty will reflect these speed changes (i.e. difficulty will decrease after N++) with lags. As soon as block reward depends on difficulty the reward will increase also and more miners will be motivated to join after N++. It's not easy to predict now but we can be sure that some waves will happen on each N++ event.
I think it is important to not that last part "we can be sure that some waves will happen on each N++ event."
NFactor changes have had a negative effect in the beginning, mainly because miners became disenchanted with having to adjust settings, and increasing block rewards made it seem that early sacrifices didn't pay off.
The great thing is that the NFactor makes it like going from sha-256 to scrypt to X11 and so forth... the algorithm changes on a SET schedule, which generally leads to different hardware that is most optimal for 'mining'. Unlike the flawed thinking in other crypto communities (a disease really), where a few power-hungry people control and manipulate the algorithm for their best interest or to whatever is the flavor of the month, YACoin's changing algorithm is on a SET schedule, and it hasn't been changed since inception years ago. Some predictability is important.
What is interesting right now is that the reward per block has reached the max that is set in the code (100 YAC per block). If the difficulty drops 100 fold, the block rewards will not change, unlike what has occurred up to this point at each NFactor change.
An optimistic note is that new CPUs and GPUs that are developed hence forth by mainstream, computer-tech companies can only benefit the YAC network and the marketcap and any new interest. I think an 'altcoin' shouldn't strive to be 2nd, 3rd, 4th, etc to Bitcoin... it should strive to be BETTER than Bitcoin. There is a lot of room for improvement technically and fundamentally, but if a coin ever surpasses Bitcoin, it will be through mass appeal to a wider audience that Bitcoin is failing to do on many levels.