So, does this mean that as difficulty increases with time, we may very well see a point where the rate of blocks found drops to below 6 per hour and thus difficulty will then remain the same or fall?
Is that how it may work?
Yes. But it is extremely difficult (impossible?) to predict when that will happen.