No it won't lose bitcoin users, what it will lose are speculators and traders who intend to take advantage of Bitcoin high volatility. Why would a user abandon Bitcoin when Bitcoin got adopted by masses or government? As far as I know, bitcoin users are those who use Bitcoin regardless of profit. They are the one who use it as a currency and often buy things using Bitcoin.
Yes, I mean investors in bitcoin, not people who are using bitcoin for their daily payments. Most people who own bitcoin are investors, speculators, not too many people are willing to use bitcoin as a currency, a means of payment for life. Even for large companies holding bitcoins, they consider bitcoin an investment. El Salvador is a country that accepts bitcoin as legal tender, they have bought and stored bitcoin as an asset and not a currency, even though it is legal currency in their country.