You should be able to use any distribution you want.

I've successfully installed Debian on a stick. And Ubuntu is based on Debian.

But a stick is not as resilient as a (SSD) disk with regards to rewriting, so the stick might not last
as long. OTOH, the price of a 4GB stick isn't much, so it not a big problem (for me at least).