Yes, serial numbers would be quite silly as a private key. They are predictable, they are sequential and anybody who handles a bill has the serial number and could empty out the bitcoins.

Hence the old mafia movie saying "bills, used, non-sequential".