Because off-chip video memory is incredibly slower as compared to registers and on-chip memory.