The problem you describe sounds a lot like this:

https://superuser.com/questions/183982/ubuntu-server-9-04-computer-freezes-the-entire-network-periodically

As the link says, I think this would be an interesting test:

"Next time this happens, it would be interesting to see if the problem goes away simply by disconnecting the Ubuntu server's Ethernet link to the network, and if it does, does the problem come back when you plug the Ethernet cable back in?"