Not practical, if you insist want portable device which is open-source (at least on software level), you could always bought linux smartphone (such as PinePhone or Librem 5) or use custom android ROM (such as GrapheneOS)
The only thing that you can do is to save the private keys in the notepad.
At least according to this news (
https://www.techrepublic.com/article/pgp-mobile-high-security-for-palm-os-devices/), there's PGP application for Palm OS where you can encrypt and decrypt stuff.
But i don't know if you still can install it and how reliable the algorithm from early 2000 these days.
or whether you will need to implement these yourself (ie. ECDSA, AES256, RIPEMD160 etc)
And that's how you lost your Bitcoin, either by wrong implementation which leads to invalid public key/address or weak entropy source for private key generation.