the test-net for SegWit is pretty thorough as it is. It's not logical to object to it on the basis of never having been tried before; every change big and small to Bitcoin has arguably never been tried, including the supposedly simple blocksize increase approach.
With blocksize increases, wallet users and miners bear the costs of the upgrade equally. With node numbers already dwindling, and mining centralising further, the incentive for the new blocksize to saturate quickly is too high under this attempt to scale up. Increased resources to run Core will push more wallet users away from running a full-node. A negative feedback loop.
With Segwit, the resource pressure is largely taken off wallet users, with the miners bearing most of the costs of the increases (and relay nodes doing so as they choose). Wallet holders using Bitcoin Core can enjoy similar levels of resource usage to before, encouraging adoption in that segment. Adoption that will be useful to the miners, who will have up to x4 the space for new transactions from these new users. A positive feedback loop.
And remember, x4 the space is multiplied by whatever the base blocksize is. The 4MB is based on 1MB as a base blocksize, but 2MB base would yield 8MB. So, you could argue that big blocks is additive, whereas SegWit is multiplicative

Yes, the timid "never been tried before" objection is pure hand-waving FUD, especially in light of the ongoing testing.
But perhaps segwit should be used in production (not testnet) by altcoin before getting merged into Bitcoin.
Viacoin successfully served as the testbed for CLTV, so why not do a wet run over there?
Blockstream has plenty of money for paying Peter Todd to do it, and he's familiar with VIA....