error with starting custom compiled x86 scrypt-jane enabled was in file pthreadVC2.dll - this i have replaced from original x86 cudaminer binary and - it is running! :-)
x86 version has higher performance
Interesting is, that now on my titan this build is running at 1189Mhz instead of 836Mhz ;-)