If the value of Bitcoin goes down below the cost of digging, people may not be able to start mining anymore but will buy back Bitcoins, which results in loss of diggers, increased transaction fees.
I agree with your opinion. But it is hard to make that happen because the community uses Bitcoin very large.