Why don`t try benchmark mode for DAG 1 or so..
test it with:
EthDcrMiner64.exe -benchmark 2
it will use just ~1GB of ram for DAG
The problem is on driver level and it isn't related to DAG size - as far as I know. I tried to run a benchmark but the result is nearly the same (instead of 3,5MH/s I got 3,6MH/s

)