I thought about that, but would be strange that the guy doesn't even change the password afterwards... And where would he have found my e-mail address?
That's too much to be true...
Email provider should let you check last few IP addresses which accessed your account so you can double-check. (Usually in a "tools" or sometimes "settings" tab - though with smaller providers, you' may have to email support)