I had that problem on one of my machines a while ago. I believe I had to disable IPv6 in ufw for it to start working again. Let me see if I can find out how to do that again, I can't remember.
Edit: try this:
To disable ipv6, edit /etc/default/ufw and change IPV6 to no (without quotes)
# Set to yes to apply rules to support IPv6 (no means only IPv6 on loopback
# accepted). You will need to disable and then enable the firewall for
# the changes to take affect.
IPV6=no
Amazing. That fixes my ufw issue. That also fixed my other connections issues. For example, connecting from ssh took 20-30seconds to connect after ufw is enabled. that is also fixed.
Thank you very much Morblias.