I get what he's saying, but the way he says it makes it sound like there's no such thing as a reasonable, responsible, honest woman. (Or maybe, just maybe, such women tend to avoid him.)
Also the misuse of the word "whom" bothered me more than it should have.
woman avoiding him for what reason?

Personally, I didn't think about it until I met the love of my life. I had a lot of bad luck with girls and then fate gave me one beautiful person that I met on my way.
So are you married now? since you already met the love of your life?