I am not computer savvy in regards to programming or setting up servers etc. If one were to buy all the equipment and software, hardware etc? How much would something like this cost to set up a Bitcoin exchange better than Mt Gox?
Servers are the least expensive things as usual. A rough estimate for a serious exchange could look like this:
Software development or purchase of out-of-the box product + customization: 100k-200k USD
Servers: around 20k USD should be enough + Xk USD that you want to spend on DDOS protection, X can be as large as you want and still won't give you 100% DDOS resistance.
Initial legal costs: 10K USD-100K USD (heavily depends on jurisdiction)
what bitcoin-related startups need least is fiat money.
to OP: if you cannot develop by yourself, then don't try a bitcoin-related startup, because by doing so you give others opportunities to cheat you. Don't testing people.