Total removal of fiat from the society will do more harm than good.
No matter the level of technological development, not everyone will be able to embrace cashless transactions. And also, most services need to be paid in cash.
We don't really know what new technology may come out in the future though. It's possible that somehow in the near future, everyone will have instant access to a bank account or some form of savings account that they can use instantly with some piece of technology. For now, however, I agree with you that the removal of fiat would not fare well with society.