I have similar behaviour on a titan xp. Far worse performance with --enable-nv-hotfix.
System : windows 10 (creator update)
cpu : xeon E5 2696V3 (18 cores @3.2ghz with microcode/bios hack).
With the hotfix enables, performance is aroung 5 instead of 22-23. Cpu usage is very low ( less thant 20% for a dedicated core). Without the hotfix, performance is good and cpu usage is near a full dedicated core.
Would you allow me to send you a private message? Unfortunately the forum still indicates me as being a newbie and your settings currently does not allow me to send a message.