In general, if you seriously delve into this aspect, then the first exchange that announced its crypto debit card was the Bitfinex exchange, it was back in 2015, but the first exchange that not only announced, but was able to issue a physical plastic card was precisely Coinbase, yes.
The first exchange with their plastic cards was Bitcurex in August 2012:
https://web.archive.org/web/20121021142209/http://shop.bitcurex.com/product.php?id_product=10 Obviously also no KYC at the time.
The next one after them was BitInstant, I think, although I don't remember if they actually managed to launch it.