But hear me out: The most secure way to store data is not to store it. Or to store it for a very limited amount of time.
With all the shortcomings of self-hosting mentioned above, it is also much easier (and verifiable) to completely delete customer data when you host it yourself.
It would be great if it worked that way. But due to regulations and local laws, businesses are required to keep records of their customers for X period of time. Unfortunately, the X seems to be different from company to company and depending on the territory. Some businesses anonymize private data after a while. Even that's better than storing it in their computers for 10 years. I think Ledger stores them that long. Would be even better if that anonymized data was taken offline and stored on paper in a company office space somewhere and then simply destroyed once the law allows it. I guess I am dreaming now...