You're right, Emergencies doesn't give us advanced notice before happening,it's always takes us by surprise and for that reason alone,our emergency funds shouldn't be tampered with for any reason except for real time emergencies. Anybody who often diversify his emergency funds for other investments purposes is simply a gambler and might be doing so for quick profits .
People can change their initial plans but I think it's more possible when a person does not have any idea of an emergency fund or ignore to build up his emergency fund. In contrast, if a person already thought deeply about it, and actually built up his emergency fund, it's harder to withdraw money from his emergency fund and use it for other things like investment, trading or gambling.
I don't say it never happens, but it barely happens and people most won't break their own good initial plan by arbitrarily use their emergency funds for something in short term only.