This sounds like how to not eat and drink for the rest of our life. Getting married and using a bank is just an option but paying taxes, investing and getting a job is a part of survival. Taxes are like funds for the security of peace and good governance. Investing and getting a job are quite the same. Investing time, effort, and money is required to get a job and to do the job.
Getting married and using bank services are not strictly optional. I would rather say they are close to necessity. Live-in relationships are not stable. So if you want stability in your family life, then you have to go for marriage. But the risk will always be there. In case of a divorce, be ready to part with a large portion of your wealth. Regarding taxes, there is no escape unless you can find some loophole which is 100% legal.
A relationship isn't a means to end any kind of service being used. When one gets into commited relationship automatically he himself looks for security. This is where banking services and few more insurance plans serve to be a life necessity. Even with divorce there are legal procedures, based on which there will be settlement. If one isn't willing for a divorce then it isn't gonna end. Loopholes are found everywhere, and depending on the lawyer being hired only the fee varies.