Gambling money IMHO is only dirty if it was used to launder money from other sources but otherwise, it's money earned like in any other business.
People would spend their money on what they want and I believe they should be allowed unless they are harming others. Removing the outlet from gambling would just push these people towards other vices.
And just a few years ago, our Education Secretary shunned billions of state-funds because they are coming from gambling. She didn't think it moral and proper to buy books and even build classrooms using the dirty gambling money. [2]
Now that is just ridiculous. So she'd rather not have books and classrooms then?