If you take emotion out of this, the version that reaches consensus first, reaches consensus for everyone. This happens every 10 minutes already. Joining the consensus opionion is part of the protocol.
Scenario: 75% miners adopt hard fork and it happens, 25% disagree. How can they reach consensus for "everyone"? This does not make sense.
On paper, why sould anyone leave? This is how bitcoin is supposed to work.
A lot of people will leave. However, maybe more would join. Who can know exactly what is going to happen?