Question is what's the best strategy to now go to the desired end state:
Singlesig (offline-wallet) --> Multisig (offline-wallet)
In that case, you don't really have any option other than sending the UTXOs from one wallet to the other and paying the transaction fee each time.
Why do you want to move the coins to a multi-sig wallet? Is your offline wallet no longer secure? Is leaving the coins where they are not an option?
If you do still want to send them across, then you can either wait until fees are lower or just bite the bullet and pay the fees. You will have to spread the transactions out over a period of time so they are not obviously linked together. You could potentially consolidate two or three of the UTXOs together each time to save on fees if you are happy with the privacy implications of doing so.