Is it problematic to plug it in with this power supply just to give it a test run? I'm anxious to give it a try, even if I have to clock it down or throttle it due to the poor PSU (until I can get a better one).
I would highly recommend against it. That PSU can't provide half the power a 5970 wants, and that's not mentioning the other cmponents that want 12V power, namely your CPU.
Symptoms would range from mild to catastrophic.. mild would be it doesn't turn on, or does but is very unstable. Catastrophic would be the PSU failing and taking out your PC and your new 5970 with it.
I would recommend going with something like this PSU at least:
http://www.newegg.com/Product/Product.aspx?Item=N82E16817139028