You are absolutely righf cause this need go be discussed to help the newbies and one of the disadvantages cancel and replace a stuck transaction with another one on ETH network is fhat the previous transaction fee wont be refunded.
The fee paid for the first transaction (stuck transaction) does return to you. But since for canceling a transaction, you need to make a new transaction to yourself, you have to pay fee for a new transaction.
This is same as bitcoin when you double spend a stuck transaction to yourself.