The wealthy already avoid many taxes. I don't think Bitcoin will make that much difference in tax avoidance. Taxes will still be collected on purchases at regular business and possibly when cashing out Bitcoin.
The wealthy avoid taxes legally through the loopholes in the system. Bitcoin is a whole different ball game and there soon will be a team paid by the government to track bitcoin usage to be sure no one can avoid taxes using bitcoin. I am not sure how they will do it, but I am sure they are working on it already.