Should I rather get another hardware wallet and try to get my inventory over there?
What is your exact model of ledger hardware wallet?
Old and retired model S is slower than newer models, but I don't think this change would speed anything in your case, and your computer hardware and processor is more than enough.
it's not faster yet. I'll get a new hardware wallet on Tuesday, then I'll try to get everything over and will then reset the ledger.
Did you choose any other manufacturer for hardware wallet?
I would suggest getting something that is open source, and if you are only dealing with Bitcoin than Passport by Foundation could be good option.
As for other hardware wallets, I never heard someone had problems with mining rewards, but you can check the list I made for open source wallets available today:
https://bitcointalk.org/index.php?topic=5288971.0