There are so many websites these days that require you to connect. I've been connecting other wallets with low balances to test things out, but there doesn't seem to be much information around on what is good practice here.
A few things for sure is to carefully choose which dApps you are going to use and also bookmark them.
My big balances are secured by Ledger so I suppose I don't need to worry about the coins simply being stolen without my approval, but what if I connect the Ledger to approve some coins and it steals other coins or uses the approval for something else. Is any of this possible?
Yes, it's possible and it is already being used by scammers. It's can be called as blind signing, Ledger site has some decent explanation about it (
https://www.ledger.com/academy/cryptos-greatest-weakness-blind-signing-explained)
When should I not connect to a website?
Literally, you should have scepticism on everything you interact with any smart contract whether it is known or not. The thing that there is a phishing site and deceitful smart contract really made newcomers have a hard time grasping what it is all about inside their heads. The simple thing you could do is to play it safe by using and interacting with established tokens and dAaps that have been running around sometime. Don't just randomly accept help and follow unknown person guidance. And also be careful not to use the link that another person gave you(e.g.,
https://app.uniswap.org/#/swap), you better find the site you really want to interact with by yourself, verify it is authentic and bookmark it.
Also, there is a collection of threads about security in general, I believe you should take a look at it:
Beginners & Help Encyclopedia: Security