I would agree. Since you are sending it to a Ledger hardware wallet, then your coins will be pretty much just as secure there as they were on the paper wallet, and it will make your life much more straightforward in the future when you want to spend them again. If you do want to put some of them back on to a paper wallet, then you should create a brand new paper wallet from scratch (which is a whole other complicated multi-step process, but also involving many of the same steps regarding airgapped computers as above).