Are all the core developers going to quit because of this ? No, they have jobs and careers established and most won't be going anywhere so they will just start working on an implementation which in compatible with Segwit2x and which also brings in the changes they want to see introduced. This could be the segwit2x code base or it could be a branch from the core code.
Several Core developers have expressed that they will not work on an implementation compatible with Segwit2x. I haven't seen one Core developer who has agreed to work on a post-fork Segwit2x-compatible client. And I believe that James Hilliard's BIP91 effort was entirely related to safely activating Segwit (instead of using BIP148), and not to do with the NYA/Segwit2x effort.
The minority of very very vocal developers who are against this may decide to leave the community but this may be because they have taken very strong stances which they don't feel they can back down from. If this is the case then I don't see this as a big loss. If you can't be pragmatic in a community project and accept "your" grand idea might not have made the final cut then the community is probably better off without you.
Certain people within the NO2X crowd (inc. developers) have talked about a change to the POW algo. or switching to Proof of Stake. This is up to them but in my opinion this is an even bigger change than amending a blocksize cap and anything new implementation with those changes won't be Bitcoin. With one or two exceptions I think this is a bluff anyway.
There's a couple very politically vocal developers (like Luke Jr. and Eric Lombrozo), but mostly the devs stick to the mailing lists and technical discussions. Luke
has been trying to fork off for years. He's been talking about changing POW algo and difficulty algo for a long time. He's an exception to the rule and I think much of the community would be happy to see him move onto an altcoin and away from Bitcoin.