Money runs gambling. It gets transferred from a point to another. From the poor to the wealthy, from the wealthy to the poor.
Let's first talk about how the poor gives the money to the wealthy.... They allocate a huge part of their money (which isn't a lot in the first place) to gambling thinking that this would be the quick fix to their problems. This is how they give the money to the wealthy in this case the casino owners.
But the money can also go from the wealthy to the poor. If the tax from casinos gets collected by the government and is used effectively, it can help a lot of those in need. The money can be used to improve infrastructure, education, healthcare and etc.
Your point is correct, but the proceeds are taxed as a percentage, so they get back in taxes, but not all of it.
The casino owners will spend their profits, so they enter the market anyway and will still reach those you call poor.
There are many variations, but the poor are the ones who suffer the most in this economic cycle.