OK, so it is true then that if you use Electrum (or Sparrow I heard?) and just stay away from any coin that doesn't have a 3rd party wallet that can interact with the Ledger in place of Ledger Live, Ledger can't see your IP address?
If you are a Ledger user, you can't stay away from Ledger Live entirely. You have to set up your wallet when you first get it. You have to install the crypto apps from the Account Manager and upgrade the firmware, which also requires installing a certain version of the Ledger Live. After that, you can avoid using the LL, unless it's time for another round of upgrades.
But yeah, connecting to Ledger servers is a privacy leak. They will surely know your IP address and your balance. But it's an equal privacy leak to connect to someone else's Electrum server except yours. The only difference is that there isn't a company like Ledger receiving this information (publicly known at least), it's the subject running the Electrum server. Take note, though, that anyone can be behind those servers including chain analysis or government agencies. Nothing stops them from owning nodes the same way you can.
I think another reason is that the BTC community (not just on this forum alone) has promoted Ledger (and Trezor too) for a very long time as recommended hardware wallets, i am talking of before the community knew they were lying about wallets secrets never leaving the secure element and a lot of other lies and security flaws from Ledger.
You are missing something far worse that doesn't concern only Ledger. All hardware wallet manufacturers have lied to their customers that sensitive data can't leave the secure element. Ledger was just stupid/smart enough to tell the public about it. That's the worrying part.