Be careful here. Other than the obvious, checking you're shorting the correct pins you need to look at the minimum load specifications for the power supply. A lot of PSUs specify a minimum load on the other supplies e.g. 5V. You need to check this out before going any further. You may find you need to add a few power resistors to generate the minimum loads to ensure proper regulation.
If I plug the 24 pin into an old motherboard that won't be used, but turn on the power supply by turning on the motherboard should that work?
Hard to say. Again, check the load specs for the PSU you have in mind. If there's no CPU, memory or fans connected to the mobo your load may not be within spec on 3.3V, 5V and 12V. Your power supply manufacturer spec sheet should have the answers, choose wisely and you may be fine just with the paperclip though rigging power resistors isn't too hard. Take a look at:
http://www.playtool.com/pages/psuunderload/mincurrent.html