Presumably GPU VPS's are the same to use as a CPU VPS e.g. the same Ubuntu OS. It's just that the hardware specs "behind the scenes" are different. Yes?
For most of the services, the GPU is connected to the server itself. You would be using the server like as if its a normal computer with a GPU inside it. However, you must pass the flag to the programme. (ie. adding a -gpu in the argument).