The most critical problem of PGP is the lack of forward secrecy, IMO.
Briefly speaking it is about the risk of an exposed private key situation that compromises a secret message in the future. Implementing forward secrecy in current PGP implementations is possible though: peers can agree on a shared session key but it is not considered as a feature for PGP as long as it is not a part of the core key management protocol.

As of using bitcoin public keys for encryption of messages, it looks rather a bad practice both because of address re-use and lack of key management and no support for fame and mapping keys to real-world persons and entities. Note that implementing such a system on bitcoin would be worse because of dangerous privacy implications.