I was tinkering with BTCPayServer, even though I do like the idea but it doesn't satisfy my needs (Test building Bitcoin Exchange - for fun dw dw)
First of all, i have doubt BTCPayServer is suitable to manage cryptocurrency exchange. As stated on their docs[1], the common case for BTCPay is for mostly receiving Bitcoin.
What I am thinking is, manual check-outs and automatically sending coins to a hardware wallet, and send them back to the users upon their withdrawal requests?
I believe you'll need to use pull payment feature[2].
[1]
https://docs.btcpayserver.org/UseCase/[2]
https://docs.btcpayserver.org/PullPayments/