Ordinary people aren't very tech savy and don't care about satoshi/byte. I moved funds lately. I believe I spent $5 to move approximately $55 and a higher fee to move a bigger number.
5$ for a relatively standard transaction with 2 inputs and 2 outputs (legacy) is more than 140 sat/B. And if you used segwit, 180 sat/B. You heavily overpaid.
Currently 1-2 sat/B are enough to get your transaction confirmed within a few blocks. That's roughly 0.02 USD for segwit and 0.03-0.04 USD for legacy.
Which wallet did you use?
Either your wallet estimated the fee poorly (which almost always is the case) or you chose a way too high fee by your own.
Even when the fees spike (like 1 - 2 weeks ago), a fee of 20-30 sat/B was enough to get your transaction confirmed in a reasonable timeframe.