You are indeed allowed to move to pay less taxes (tax evasion is staying but moving your money illegally), unless a US citizen that is.
Do US expats who live and work in another country have to pay double taxes? Once to the IRS and once to the tax agency of the country they reside in?
You have to file a return, there are treaties which might reduce taxes (e.g. under FEIE). You may be able to get a credit or deduction for foreign taxes paid - varies by country and treaties with the US. Then there are FBAR reports for foreign bank accounts. Big government at work.