Restart to safe mode can help to unistall driver properly from device manager.
Again tried safe mode earlier, none of the usb ports still work even in safe mode. Which means no keyboard/mouse for even doing the device manger thing.
A bit of a chicken and egg problem.