Even if price stays the same, if enough miners quit, difficulty will go down, so the remaining miners will be still ok - it will always even out, either price goes up or difficulty down (or combination of both).