Anyone who knows your login and password and is in rpcallow range, can control your client remotely. It is as simple as that. I would not advise to allow anyone besides 127.0.0.1 to control your client unless you really know what your doing. If you have to use such config, at least encrypt your wallet.