I would disable passwords in ssh and generate a private/public key pair to login from the outside world.  I run 2 copies of sshd on my box.  One on port 22 that uses passwords but is only available on the internal LAN and 1 that is mapped to a different port that I only know that only allows access through keys.  My router disables port forwarding until I run knockd to enable port forwarding to my hidden sshd port.  Knockd is a program that listens on the Internet interface on your router and will allow you to enable port forwarding on demand when a combination of certain tcp/udp ports are accessed in the correct order that only you will know.  Basically a secret knock that will then allow you to ssh into your systems behind your router.  Also for this level of security you would have to load different firmware onto your router.  Something like DD-WRT, Tomato or OpenWRT.  This is not for the novice but secures your system much tighter.

-Dukejer