No you don't need to log in to the site to use runsim. Can you give me the error for the crashes please? It might just be the site you're running on that I have not implemented the roll verifier for yet. Have you tried switching sites and running then sim then?
It wasn't giving a specific error. It just said DiceBot has stopped working.
I tried switching to another site and it did work correctly so looks like it was due to not having roll verifier implemented on the site that was selected.