Any suggestion as to what happened is welcome, the amount wasn't huge and I've already made that back twice so fuck the scammers but let's save someone else from falling for the same thing.
The scammer abused the RBF feature.
If a transaction has been flagged as RBF, as long as it is unconfirmed, it can be replaced by a new transaction spending a same input and having a higher fee rate and higher absolute fee.