You can't verify that a specific person sent the money to you by matching addresses. That's just the way the system works. See this discussion:
http://forum.bitcoin.org/?topic=2273.0The workaround to this problem is that the recipient sets up a new receive address for every client. That way, when money shows up in the account, the recipient knows who it's from.
Good point, I straight away assumed the received address meant the 'senders' address, rather it is just the address those BTC were at previously.
The workaround you said is a viable idea, however I'm not sure what the theoretical limit of number of unique addresses per machine is, but I suppose for transactions like these they probably own many computers.