Ooh, possibility to notice. Though I'm actually planning on having the server generate a fresh address and private key for the arbitrator to use, so there wouldn't be a chance of a multsig address repeats. (I have the server generate the arbitrator's key and give it to him so that the server can immediately sign the failsafe transaction with the arbitrator's signature. Then the seller would have no reason not to sign the failsafe transaction, so his signature is guaranteed. This means that the failsafe transaction will work even if the buyer and arbitrator refuse to sign any further transactions. It does mean that a malicious server could act as the arbitrator, but I think that's a small acceptable amount of risk. If enough people found this too debatable, I could add an option to allow the arbitrator to provide his own address, and I'll make sure that the arbitrator's address isn't already in the system in order to address the possibility you bring up.)

Ha, funny thing is I've been implementing this on a coincidentally similar (even to the price) bounty given by someone else.
(Bit of criticism of knowitnothing's solution: in his, the server generates the private key that the payment goes to. It's just a regular single-key address, not a multisig address. The users have to trust the site to not run away with the funds. The server then splits the private key that it knows with a secret-splitting algorithm allowing any two users to create a copy of the private key. There's no way for the users to verify that the split-key values that they get from the server are valid until after the funds have been sent and they attempt to recombine the key to claim its funds.)