It is no less secure than plugging your device in with a cable to a compromised computer. The Bluetooth set up is designed on the assumption that it is always compromised.

To pair your phone with your Ledger device obvious requires use approval. Once paired, all communication is AES encrypted to prevent a man in the middle attack. As always, your private keys never leave the secure element, and so can't be extracted. As with plugging your device in to a compromised computer, if you were to pair with a compromised phone via bluetooth, the worst that could happen would be some clipboard malware tries to generate a false transaction. As long as you double check what shows up on your Ledger screen, you are perfectly safe.

You can read more here: https://www.ledger.com/ledger-nano-x-bluetooth-security-model-of-a-wireless-hardware-wallet/