There is always possibility of failure or a scam if we are dealing of centralized service. We will never be free from that thought.
And bitcoin seems to be magnet for whole bunch of shady entrepreneurs too. Mt.Gox was the biggest exchange of its time, we don't have that big volume exchange anymore.
So if any of smaller exchanges fell it will be not big deal.