I'm sure that you are a solid developer since you run a mining pool. Are you 100% confident that changing 4743 lines of code deep in the Bitcoin codebase won't have any issues? And what about wallet and 3rd party software? When you add it all up, we're looking at tens of thousands of lines of code changes, which guarantees roughly 150-500 bugs (industry average) in all of this software, of which 2-5 are potentially catastrophic...
A hard fork to 2MB would involve one line of code change and some late-night sweating and swearing for a week

Bitcoin Core does not follow normal industry standards in regards to defect rates, this is due to a huge emphasis on code review and testing by domain experts in cryptography and secure programming(a large amount of code written for SegWit is testing code), and when it comes to consensus critical code like SegWit extra care is taken during testing to prevent defects beyond what is typically done in areas of the codebase that aren't consensus critical. SegWit in Bitcoin Core is some of the most thoroughly tested code in the Bitcoin industry and is unlikely to have any bugs that would cause serious issues.
When it comes to support for wallets most rely on well reviewed 3rd party libraries that are ready to support SegWit. For some wallets like hardware wallets SegWit helps a lot since it
fixes issues with the transaction format.
Interestingly enough a lot of the code that was written for SegWit would also have needed to be written in the event of a hard fork, for example the preferential peering code written for SegWit would also be needed for any hard fork. Due to the difficulty in activation coordination and other issues such as the sigop/sighash scaling issue there's no such thing as a simple 2MB hard fork.