If that were true... how do paper wallets work then?

Just because the majority of wallets these day are "HD" (Hierarchical Deterministic) wallets and generate a new receiving address each time the current address is used, does not mean that you can't have a single address.

In fact, if you REALLY wanted to set this up, you can do it pretty easily using Electrum.

Simply create a new private key (in "WIF" format) using your preferred method (dice, offline copy of bitaddress.org etc)... then in Electrum use "File -> New\Restore -> Import Bitcoin Address or Private keys".

Enter the private key you generated and click next.

You now have a software wallet that has exactly ONE address/private key in it... and it will NOT generate any more. It won't even create any change addresses, so any transactions you send from this wallet will send the change back to the original address.

NOTE: there is no way to make this setup work with your Ledger.