It doesn’t take much to change a finite supply into an infinite one.
Only if you want to make a hard-fork, which will quickly put you into an altcoin land.
What if that final drop never happened?
Blocks with additional coins would be considered invalid by old nodes, and would turn all users of a new version into an altcoin.
Would that make make Bitcoin any less hard a currency?
Note that in the current consensus, it is possible to burn coins. Technically, a coin can be made in a way, where all coins will always be spendable, but Bitcoin is not such coin.
So what makes a currency hard, if not a capped supply?
Mining. Proof of Work is what makes it hard, because then, changing things require consuming real energy, and this is what makes it hard to produce coins out of thin air. A good example of that can be seen in test networks, where testnet3 and testnet4 have a lot of chainwork, and are traded for real BTCs (because producing test coins requires mining, and it is costly), while signet, with quite low chainwork, is worth literally zero (also because it is centrally controlled, and developers can halt it at any time, by refusing to sign next blocks).
How low should inflation be to be considered negligible?
As long as people can burn coins, it doesn't matter that much. On top of a chain with tail supply, users can agree to burn their coins collectively, and to keep the supply limited, no matter how many coins will be produced. I guess if some testnet would really have doublings, instead of halvings, and if eventually people would get 21 million coins per block forever, then some people will opt into coin burning, just to raise the value of their coins. Also, they will introduce "ordinals", "rare satoshis", "NFTs", and other kinds of things, which would be worth more, than other coins around. And then, "1 satoshi != 1 satoshi" for some people, because things are worthy, if you put some work into them (the most trivial example is mining, but it is not limited to just that).