Hello! I know I am late with this answer, but maybe you are sill around.

Did you tried the One-Click Miner from Vertcoin? If the answer is Yes, do you get the same errors?
I am using the provided software and have no issues.  Here is the official Github download link: https://github.com/vertcoin-project/one-click-miner-vnext/releases

Please also read the Hardware Requirements https://github.com/CryptoGraphics/VerthashMiner#supported-hardware and see if your card fits the requirements.

Last but not least, download and install the latest drivers for your card. Actually this is the first thing to do before even starting the miner.