Thanks guys, finally it connected! And I didn't change anything.
I've also successfully connected from another PC on the same LAN to my electrs via SSL.

It wasn't about the ports, and Bitcoin Core was indeed fully synced.
I think the issue was electrs still compacting the DB.

When electrs indexed, the DB took 60 GB of space, then it started compacting and the DB kept shrinking. Last night it was 44 GB and I still couldn't connect. Now it's 42 GB and I can connect.
The `starting txid compaction` message in electrs logs was the telltale sign, but it could have been clearer as there was no indication of progress and it took longer than expected to complete. My Bitcoin Core synced in under 72 hours, I was expecting electrs to be done within hours, and that most of it would be the indexing and not the compacting.