As long as you secure you pass phrase and make sure not to leave it plugged-in and active in your computer all the time, your funds in your hardware wallet will be safe.