There is no guarantee that the device which has been online before hasn't been already infected.
Also, when you delete a file from your computer, it's not completely removed from your hard drive. That's why there are some tools that can recover deleted files.

So, if you want to be completely secure, you should use an air-gapped devie. This means that you should use a device which has been always offline and will be never online.