I don't know if there are any special tools they use, but to me, it's very easy to know who is using VPN in some cases. If you notice a particular user who, each time they login to their account, appears in a new location with a different IP address from their last login, is that not an obvious sign of VPN usage? 
 
And they can also confirm their suspicions by asking the user to provide KYC verification documents, which clearly show their country of residence, which will be different from the IP address the VPN will give to the gambling site.