Thanks, but I'm 1000000% sure the aren't any other instances running.
Try to update your system and Java:
apt-get update && apt-get upgrade
Java is up to date
ii oracle-java9-installer 9.0.1-1~webupd8~0 amd64 Oracle Java(TM) Development Kit (JDK) 9
ii oracle-java9-set-default 9.0.1-1~webupd8~0 amd64 Set Oracle JDK 9 as default Java
09:55:57.804 [INFO] PeerClient Starting IP refresh thread
09:55:57.937 [INFO] NodeManager Node manager started
09:55:58.141 [INFO] PeerServer Starting peer server: address = 0.0.0.0:5161
09:55:58.202 [INFO] SemuxBFT Consensus started
09:55:58.204 [INFO] SemuxBFT Entered new_height: height = 1, # validators = 4
09:55:58.239 [INFO] SemuxP2pHandler Received DISCONNECT msg, reason = DUPLICATE_PEER_ID
09:55:58.608 [INFO] SemuxP2pHandler Received DISCONNECT msg, reason = INVALID_HANDSHAKE
09:55:59.363 [INFO] SemuxP2pHandler Received DISCONNECT msg, reason = INVALID_HANDSHAKE
09:56:00.097 [INFO] SemuxP2pHandler Received DISCONNECT msg, reason = INVALID_HANDSHAKE
09:56:00.587 [INFO] SemuxP2pHandler Received DISCONNECT msg, reason = INVALID_HANDSHAKE
As I'm saying RC5 is broken for me. RC3 works fine. I can go to my rc3 dir, start it and it works. When I stop it and go to RC5 dir, start it, then it's stuck with the above error.
Start fresh with RC5. Download a new copy, delete the old database folder, restart pc and start resync from block #0.