This is not 100% safe. Probably 99%. You forget about the BIOS,MBR and Viruses that can Infect your Graphics Bios. These will Infect your CD/USB distro after reboot.
If you're worried about those writing to your sd-card, then you can set it to read-only when you don't need to write to it. If you're worried about it reading data on the card...any virus could do that already (so if you're infected, you're SOL). If you have one of those on your computer, then you're in pretty bad shape already and I don't know how to help you.
