The wallet reads your 66.conf located in %appdata%/66 (in Windows usually  C:\Users\username\\AppData\Roaming\66 ) at startup.
rpcport=16666 should be fine or 12345 if you like.

You probably have something wrong in your batfile when starting your miner (or cgminer.conf / sgminer.conf if you use that) located in the miner-dir.

save this to solo66.bat in your miner-dir and run it to test:
sgminer.exe -o 127.0.0.1:16666 -u username -p password -I 13 -g 1 -w 512 --thread-concurrency 16384 --verbose
pause

or for cgminer:
cgminer.exe --scrypt -o 127.0.0.1:16666 -u username -p password -I 13 -g 1 -w 512 --thread-concurrency 16384 --verbose
pause

"username" and "password" should match rpcuser and rpcpassword in 66.conf.
intensity and thread-concurrency should normally be higher but this is just to test.
with --verbose you´ll see that your miner is alive you can turn it off with d and v .
pause is there to possably see what went wrong and not letting the commandline window close it self.

if you´re using a bat-file remember to rename cgminer.conf or sgminer.conf otherwise the miner will read parameters from it.