I get 0.08% rejects with SSL, which is about the same as without. You are assuming SSL adds a large overhead, it doesn't.
..
Or get Google Fiber.
Reject rate and latency heavily depend on your internet link. You are absolutely correct when you have a "Google Fiber". But don't forget that not everyone can get fiber at all, and some people still have to use GPRS (not me).
Answer size is less than packet size
Did you measure it? I did.
In ETH - yes, in ZEC - answer size ranges from 1360 to 1490 bytes per share. It is almost Ethernet frame.
SSL overhead is extremely negligible, you are overly worried. If you don't like SSL, or still think it's slower, though have no actual valid reasons, don't use it. It is there as an option. No one is forcing you to use it.
I don't use it.
But there are reports even here about long share pings over SSL. It was not me. Probably, not all have Google Fiber :-)
Again, let's talk about facts only, SSL encryptions adds nothing to miners apart the ability to hide t-address they mine to. Still, it does not hide the fact that IP X connects to IP Y port Z, where a pool is located. So IP X does mining. If someone is concerned, they will find a way to get more details. Security by obscurity never was good. Zcash adds an unique option of z-addrs, and this quite solves the issue of privacy. It still does not hide the fact that you mine at pool Y, but there is no overhead. Finally, it is ok when you have a choice. But when it is your choice, and you are not "forced" to use SSL due to lower dev fee. That's it.
I don't have Google Fiber, I have Comcast (bleh). I mine to Europe server, which is ~80ms ping higher than the US server for same pool. My reject rate is 0.08%, so obviously ping isn't a huge factor like you say. Reject isn't significantly altered by 80ms, so 0ms for SSL clearly wouldn't matter that much. 0.5% reduced fee vs 0.08% reject rate (assuming in perfect non SSL you got 0.00% reject....). Either way, you still aren't forced to use it.
SSL doesn't necessarily add anything to miners, but there are no drawbacks.