Here's another way to look at this: 320,000 BTC is a little more than 1.5% of all bitcoins that will ever exist.
And for now, and possibly for a little while longer, SatoshiDice is a lot more than 1.5% of all that's worth owning in bitcoins. There are some good companies out there. Maybe ten, maybe twenty, certainly not one hundred. This is one of them.
Surely if one is bullish on BTC and believes that there will eventually be more good companies, 1.5% of all BTC seems unlikely in the long term, let alone growing beyond that. It might make sense if valued in USD though ($3.2 mil) - that number could have growth potential.