I know there's no such thing as "frozen coins" on a non-custodial wallet but my question is, if the non-custodial wallet you use doesn't accept Replace-by-fee (RBF), can you open a new wallet that supports RPF with your seed phrase and replace the stuck transaction with a higher fee in the new wallet?
You can't turn a non-RBF transaction to RBF transaction just with importing your seed phrase into a wallet that supports RBF.
Of course, this doesn't mean there is no solution if you have a non-RBF unconfirmed transaction and you want to replace it with a new one.
By default, nodes drop transactions from their mempool if they stay unconfirmed for 14 days.
If your transaction has been broadcasted more than 14 days ago, many nodes have dropped your transaction from their mempool and you can import your wallet into electrum, connect to a server that doesn't have your transaction and broadcast a new one.
Note that if the fee rate you used for your transaction is too low, some nodes may drop your transaction due to their mempool size limit even earlier.
You can also replace your transaction with a new one paying higher fee with the help of nodes that have enabled full RBF.
Any node that has enabled full RBF, treat all transactions as RBF-enabled.