For example, Ledger have option connect to your own Bitcoin full node.
An even better option would be to use your Ledger hardware wallet with Electrum, with Electrum pointed at your own Electrum server connected to your own node, thereby bypassing Ledger's software (which could still phone home even when pointed at your own node) altogether.
For getting the transaction history of these 20 addresses, electrum sends them to the server and the owner of the server will know that these 20 addresses belong to the same person.
Additionally to this, if you then open another wallet file, and the same Electrum server sees the same Tor exit node query a second set of addresses just a few minutes after querying the first set, then it is reasonable to conclude that both sets have a common owner.
Owner of the server will know about 7 addresses, they will not know anything about the 13 addresses.
No correct. The server owner will know about all 20 addresses. The only reason your Electrum client knows that the 13 empty addresses are empty is because it sends those 13 empty addresses to the server so the server can check if they have transaction history.