For such a large amount, the $ 5 fee does not mean anything. It's still very cheap and affordable. Maybe if the price of bitcoin is expensive and reaches $ 100,000 it is likely that the fee will increase.
The only justification to what you said is, having a bigger transaction with a smaller transaction fee but the point here is; let assume you have just $10 to transact, what can be the possible transaction fee for this type? possibly, one will see a bigger transaction fee instead of a smaller transaction fee.