What email service do you use?  Can you check your login/logout histories or IP Addresses?  Gmail allows this, but I'm not sure about others.  Clearly this person had access to your email as well, to confirm the withdrawals.