What IS important is if the growth trend is slowing or not.
Agree. For example, it is reasonable to suggest that "pregnancy" is shorter for early adopters and longer for general population. Then it would be steep curve in the very beginning, then less and less steep later on. Something like this: 
EDIT: It's real data
This has probably already been mentioned at some point, but have you given any thought to 'the chasm' that is inevitably encountered during adoption of a disruptive technology?
This feature (unique to disruptive innovations, in contrast with sustaining innovations) occurs after the innovators and early adopters (who, in total, account for the first 16% of total adopters) and before the early majority jump in. It appears while the innovators and early adopters revel in the adoption of a change agent, the early majority are much more comfortable using existing tools/methods and hesitate before adopting something that requires a fundamental change to their current operations. This hesitation forms 'the chasm'.
Using 2.1 million as the value of current adoption (taken from My Wallet), and assuming we have reached the chasm, that puts the future total number of Bitcoin adopters at approximately 13 million. (Granted, this value does seem a little low, especially considering it is only 9% of PayPal's current users.)
Anyway, it would be interesting to hear your thoughts on this...