I am more concerned with the amount of inter-nodes traffic.
A client can decide not to store everything, but the information still needs to be transferred by design.
To put it simply - when the number of users is big the block size itself will be too big to be useful.
A block needs to hold 10 minutes of _all_ transactions or the system starts lagging and transaction times will soar to infinity.
Let's assume each user does at least 1 transaction per day. There are 144 blocks per day.
A million users * let's say 100 bytes per transaction / 144 = almost 1 MB per each block = 6 Mb / hour constant incoming traffic to every client.
That's why I don't think it's something that can be solved easily. People pretend that scalability is just a feature you can easily add "when needed".
It might not be so. Being a programmer I know how hard it is to solve O(n2) algorithms.
I worry that it's the whole design of BitCoin that makes it unscalable and that this cannot be solved without a major redesign.
Essentially, creating a completely new currency. That's why I don't see BitCoin as any proof-of-concept.
Seems you're so far ignoring cached/pooled & queried/polled data options available to programmers (you say you are one).
Let's not incorrectly assume that every client must see every transaction. Many clients haven't even been invented yet, such as for many mobile devices and for those on slow or expensive connections. So what you said about problems "by design" need not apply to them. ("By design" is a Microsoft term used to excuse the company from doing proper software design and actually resolving problems. "By design, this bug has existed in all versions of Windows since Windows 3.1 and we have no intention of fixing it.")
Any client which can't handle the entire block chain can simply establish a secure, low-speed connection to a trusted client which can. For example, at your request, your mountaintop phone-based computer with an expensive satellite Internet connection can simply ask (poll) your desktop back in the city for the current balance of your wallet, and for the current number of confirmations there are for the handful of payments it has personally sent or received. So a mobile or dial-up client only needs to query a well-connected one for a FEW pieces of data on an OCCASIONAL basis.
Let's also not pretend the Internet can't grow if needed to accommodate a critical financial infrastructure -- although a massive bandwidth increase shouldn't be needed, as just discussed.
As far as distributing all or part of the massive block chain history is concerned, a cached (mirrored) copy from the cloud (provided by, say, Akamai) would suffice, provided it remains possible to connect to individual clients at random to acquire copies of the latest blocks. The latest blocks validate the prior blocks (not just the reverse), so Akamai could not simply invent a false block chain to distribute for fun & profit.