As mentioned early, it is quite secure, if you use a 4 digits PIN there are 10000 possibilities and only 3 chances. And you can use much longer PINs. Remember, even if the mnemonic passphrase is compromised that is not enough to steal your funds as long as your two factor authentication is also not compromised.
If you are not using a hardware wallet, is the encrypted mnemonic stored? If so, a 4 digit PIN should be trivial to crack since with overwhelming probability only one of the 10000 combinations will yield a valid mnemonic. I assume you use a different method, but it would be good to be sure and to know some of the details!