Or is it more of a philosophical thing based on the original intention that addresses aren't used over and over by the same user (like how change goes to a different address)?
Using a hosted (shared) wallet helps to increase transaction privacy. Bitcoin addresses that are used for your withdrawal will not likely ever be ones that you used when receiving a deposit. So someone attempting to use the information in the blockchain alone to determine your activity when you use a hosted (shared) EWallet is something that is pretty ineffective.
A competing Exchange+EWallet service, Coinbase, takes a different approach. With them a user's account has specific Bitcoin addresses assigned only to that user. So all withdrawals do come from the Bitcoin addresses to which that user received payment. And that's why Coinbase could be used with SatoshiDICE whereas Mt. Gox should not.
After lamenting that this was yet another instance of someone losing funds due to this "return to sender" approach that SatoshiDICE utilizes I did ask Mt. Gox. The response:
we could provide a way to generate return addrs
Which probably means some special case handling might be implemented so that SatoshiDICE wagering from a Mt. Gox EWallet will be possible, someday.