You always do, vram is normally the problematic part of the card, since its not usually cover by the cooling system on the card (in some cards it is, such as any Tri-x edition)

If your hashrate jumps down due to temps, its normally vram