It's only a sin if you're harming yourself , your family or any body else in your environment due to your gambling activities . Religions have mixed up definitions of sins and prohibited deeds that are enlisted in their holly books . That does differ from a religion to another , but i'd prefer believing my own definition that a sin is something that we can apply the right or wrong rule on it and that it's level of danger could affect not only your present , but your future or somebody's else present and future .
Gambling being a sin depends on each person's perspective. However, like you have said, using your gambling activities to hurt others is a very big sin and as long as that can be curtailed and you know how to control yourself, then no shit! Yet I believe, for those who are extreme in gambling, I do not see how they do not end up hurting the people around them.
If we are one with the intention of gambling only for fun then for sure we will not consider gambling as a sin, only those who are addicted and have ruin their lives will think gambling is a sin but they are wrong with that and they are just coward enough not to admit their mistakes, instead they blame it to gambling. If it's a sin then we are all sinners here, I don'a agree with that because gambling is just a form of entertainment to me.