Found out why this happens - when you disable Ethereum for Claymore Dual. It still shows up in the benchmarking window, but benchmarking any dual algorithms then fails because it disables the Etherum main pool. Strangely benchmarking single Etherum still works though (even though it shouldn't even display the miner when its disabled).