Be careful here. Other than the obvious, checking you're shorting the correct pins you need to look at the minimum load specifications for the power supply. A lot of PSUs specify a minimum load on the other supplies e.g. 5V. You need to check this out before going any further. You may find you need to add a few power resistors to generate the minimum loads to ensure proper regulation.
If I plug the 24 pin into an old motherboard that won't be used, but turn on the power supply by turning on the motherboard should that work?