You have no idea what you are talking about. Seriously. You may have heard something on the internet but that does not qualify you for the opinion because you didn't understand anything from what you heard.

Computing resources allocated in the cloud can be much more powerful than anything you could ever build yourself. The cost is a variable between providers, but as a rule of thumb, the cost of running your own commercial datacenter eclipses by several orders of magnitude any cloud on-demand scaling application. You can of course continue running your servers from your basement as MtGox does for a fraction of a cost, but that has no properties required in a commercial application. Like, backup uplinks, reserve power, controlled access, staff available at all times, etc.

Banks and many large instituions use their own hardware setups for none of the reasons you mention, but because of the requirement to keep all customer data on-site.

And to non-linear scaling - yeah sure, that has been figured out 10 years ago as well. Of course it requires redesign for horizontal scalability, but it is always possible.