I'm more or less in the same situation as in yours, but I consider the mortgage the cheap credit I get to buy BTC. So I'm still having my mortgage now...

But I will definitive pay it off when it took 5% of my total holdings to do so; 20% is still a bit much, my personal opinion.