Nobody's 'taking' anybody else's money.  At whatever level the 'last ones to convert' happens if they were buying purely in order to make money from increasing BTC value relative to fiat then they will have 'lost'.  They will have made a bad judgement call assuming the future to repeat the past.  However if they are buying Bitcoin for any other reason such as a store of value or in order to use it to buy stuff (which even if those buying today are the 'last ones' is reasonable) they will have not lost.

However the longer and wider the distribution goes on the more robust Bitcoin becomes, the less prone it is to being devalued overnight by one means or another.  The reward for 'early adopters' (whether you're talking 2010/11 folk or today's) will have come from having taken that risk.  People keep knocking hoarders.  With bitcoin being as divisible as it is it REALLY doesn't matter how many people just buy and hold nor does it matter how much they hoard.  The amount that's left, however tiny, is still plenty to be used for trading for day-to-day goods and saving.  It doesn't deprive anyone else of anything. It's all good