wouldn't the hardware wallet open up potential avenues for fraud where the fraudulent 'card' or hardware wallet looks legitimate at face-value, and people only realize that they've been defrauded when they go and try to initialize the wallet and it does not work?

The balance can be verified using an NFC enabled smartphone. That's why you don't need trust, you can verify the card you have been given is funded.