As long as you have enough entropy, I do agree brain wallets are perfectly fine. The reason why many people including myself advise against it is that in general people are too lazy to properly choose random words instead substituting them for words they believe to be random or ones from famous films/songs/poems et al.
It's not just that. Everyone can work on cracking a brainwallet from anywhere. To crack a wallet.dat file they have to have a copy of it.
Anyone can also work on cracking a private key from anywhere too
without the wallet file.Bitcoin private keys have 160 bits of security, maximum.
So, you just have to make sure your brain wallet has enough
bits of security (128-bit or more).