True (was kinda simplifying).  Although not a requirement, most legal counsel would recommend it. The only renunciations which have any legal weight are those done on foreign soil.  Even if you are not a citizen of the US, your host country, which likely gave you an entry VISA (explicit or defacto) may not take kindly to the fact that the circumstances that allowed your entry no longer exists.   You can be deported back to the US as a "non-citizen" which is likely a scenario one wants to avoid.  Most countries which allow "easy citizenship" for non residents limit the countries they will accept as applicants (namely US and other "first world" countries).  

Not securing citizenship first could put yourself in somewhat of a bind if you end up deported back to the US as a "non-citizen".  You can never regain US citizenship and most foreign countries will not allow entry to "stateless" persons and the few countries which issue citizenship to non-residents don't issue them to stateless persons.

Granted each persons circumstances are different, if you are a multi-billionaire looking to live permanently at sea as part of some seasteading city you are building well it might make sense to go "stateless".