It isn't an actual theft. He didn't broke into his house or hacked his computer. I think that the "thief" can keep the funds legally.
Breaking and entering or illegal access is not a prerequisite to stealing. It is absolutely theft and is absolutely illegal.
What if I tried to log in to an online banking service with a random username and password, which happened to match the account of a real customer? Can I legally take all their money?
What if I tried to buy something online, entered some random details and a random credit card number, which happened to match the details of a real card holder? Can I max out their credit cards?
Why are these situations any different to someone randomly entering your private key?