Well, they can talk all they want. Maybe Windows needs that much, I don't know. This is BAMT with 4 cards and vertminer:

Mem:   2830160k total,  1320704k used,  1509456k free

PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
20   0  591m  162m  67m S    3  6.0  25:58.63 vertminer


As you can see it's even less than 4GB total, I probably forgot to disable Intel graphics.