Not necessarily: Check out a good review and look at the efficiency curve of a PSU. Normally they're the most efficient (least wasted energy) at about 50% of their rated load. If you get a 1200W PSU for instance, you may get less efficiency because you're not drawing enough.

When you're counting energy consumption over several months of mining as part of your calculation for profit, every percentage point counts! If your plan is to buy more cards later, that may be a different story. However, I have never done my "upgrade plan" exactly as I thought.