Of course he does.  You know the private key, since you put it on the CD.  Using that private key, you could still perform the exact same fraud as you would if you sent him the bitcoins over the network using the private key instead of him doing it.  The fact that he knows the private key too doesn't keep you from using it.  There is even more risk of fraud if he accepts it from you before he transfers the bitcoins away from that private key.  If you leave and he hasn't yet transferred the coins, you could use the private key and steal the coins before he can transfer them.