Got it! I was thinking that the golden rule "If you don't own the private keys to your coins, you don't own them." applies to all wallets.
It still does... it's just that wallets that use "seed words" derive all their private keys from those seed words... so as long as you (and ONLY you) have the "seed words"... you still control the private keys

Yes, the Seed (your 24 words) can generate all the private keys you need. All the private keys from your ledger wallet are derived from that 24 words.
For educational purposes only, go to the website iancoleman.io/bip39
Insert any seed, this one will do it "word word word word word word word word word word word word" (never insert your seed, ofc, as you can lose your funds when you write them in a computer)
All this time I was thinking that the only way to use the BIP39 private key generator software of Coleman is by downloading and installing it. Never thought there is a website for this. Thanks! Will try it for curiosity's sake.

Scroll down and you will see many Public Address and their respective private keys. This is what your seed generated from ledger can do.
Is it therefore a possibility that one can steal from it? By randomly arranging the 24-word mnemonic, you will arrive at a particular wallet address with its corresponding private key, and if that address has a deposit, you can easily import that address because you already know the private key, and make the withdrawal. Or am I wrong somewhere?