Bisq doesn't have centralized servers. The software that you run connects peer-to-peer directly to other users over Tor.
You're probably right because I'm not good at reading Java but the "seed nodes" and the way they are being used all over the code always seemed to me like servers that aren't just used to find peers but also to coordinate trades (so they receive all trade information).
I'll concede the point about the middle man and fees, though, although I'm not sure how you eliminate them when one side of the trade is using fiat. The only way I can see is if you place complete trust in the other party to release the bitcoin after they have received your fiat payment.
For trading with fiat, sure. But that should require and escrow fee not an exchange fee.
For trading with cryptocurrencies however this additional fees shouldn't exist.
This also proves the existence of the centralized servers that you connect to and are fully aware of all your trades (is not 100% P2P, there is a middle man who knows everything).