Total removal of fiat from the society will do more harm than good.
No matter the level of technological development, not everyone will be able to embrace cashless transactions. And also, most services need to be paid in cash.
I agree with your idea mate. We can't afford to buy things lower than the transaction fee. As we all know fees are getting higher as price skyrocketed and how much more in the future? Are we going to buy candies with Bitcoins high fee? For me Bitcoin is only best for investments. I think buying goods with Bitcoins has issues with fees more or less four to five dollars per transactions really sucks. People using fiat all the tine are more affected in a cashless society and I don't know how they would adopt most especially less fortunate people with no idea on this blockchain technology. It is hard for these people to embrace such ideas and that is the fact about cashless society. I am one of those less fortunate people but I am lucky enough because I have now the ideas regarding blockchain technologies and I am happy with that.
