No, the PIN is only for the physical access to the hardware wallet. Else they would have asked you also backup that on the card.
The backup card is there so anything happens, from the Ledger getting destroyed and Ledger company getting bankrupt to you losing all your memories, you or somebody from your family can access the money.