It's possible mainly if you are looking for a decentralized exchange. The problem with centralized exchanges is that usually you can't use your own address and coins to make trades, since everything is happening in the back end and you don't want to give them your private-key. You need to deposit first and trade with a IOU. Meanwhile, DEX are making P2P transactions on your own to someone's else address, so you can load your private-key in the client and do the transactions.
I'm sure there are others, but this is one I tested and can confirm that works for Ledger and Trezor:
https://www.airswap.io/tradeIt's an ETH/tokens exchange. You connect your Ledger, put how much tokens you want to buy, confirm in the Ledger to send the ETH, and receive the tokens in the same address. Quick and secure.
Thanks. I was curious is it possible to somehow integrate Trezor/Ledger with centralised exchanged to make it lets call it semi-centralised. Where for example exchange keeps your FIAT however does not keep your bitcoins or ethers. Currently there are a lot of DEX but they are based on smart contracts (ETH/NEO) however it looks like noone found a solution to integrate exchange with different types of blockchains.