it only keeps going up as more mining power is added. if people stop adding rediculous amounts of power to the pool then the difficulty will drop. there were gentlemen's agreements proposed about things like this a while back. They go as far back as the transition from cpu to gpu.

https://bitcointalk.org/index.php?topic=12.msg54#msg54