I don't know if there are any special tools they use, but to me, it's very easy to know who is using VPN in some cases. If you notice a particular user who, each time they login to their account, appears in a new location with a different IP address from their last login, is that not an obvious sign of VPN usage?
And they can also confirm their suspicions by asking the user to provide KYC verification documents, which clearly show their country of residence, which will be different from the IP address the VPN will give to the gambling site.
I think this your answer is very correct for the Op to take home. There is no specific tool or software or app for that jobs but if the person device IP address has been changed from one direction to another place then the person is using VPN. There are times, the person registered with VPN but along the line because of one thing to the other he would like to use VPN and once he does the location change.