So for those people if the difficulty exceeds lets say 6 or 8 billion they sure will quit since their hardware wouldnt even get them a thusandth fraction of a bitcoin per day in that difficulty so if and when they quit the difficulty should drop down (since 70% or 90% of the miners dont mine anymore having as a result the average time for a block to be found to increase noticably)
If you want to know about where the difficulty is headed given the current and soon to be delivered technologies read the OP in this thread:
https://bitcointalk.org/index.php?topic=281279.msg3008390#msg3008390
The maximum economical difficulty given the next generation technolgy is a lot more than "6 or 8 billion" - think 25 billion.
UNLESS the price of BTC goes up a lot and/or faster (smaller feature size) ASICS are produced then it will be even more.
This corresponds to a long term future network hash rate of about 180 PH/sec
Personally I don't think it will get that high, but we shall see.