That makes sense.

Still, it's hard to get something so simple to say "Honors their contracts" into something like a number. Like if I see 6/7 what am I to think? Do I really think that 11/11 is the same or even likely at all to be similar to another 11/11 I did business with?

Some contracts require nothing more than not being a total scumbag, for example, "I have $100 on dwolla and will send it after you send me 19 bitcoins". Unless I'm a total liar I'm going to uphold it and when you rank me perfectly it means a lot less than something like, "I can deliver XYZ software with A, B, C, D, and E features by time T" That requires proper estimating of the task, ones skill, avoidance of bad luck like illness etc etc. If it is completed it ought be a large boon to reputation and if it only partially fails it doesn't mean the person is untrustworthy with smaller or even similar jobs.

It makes me think of the doctor who has a great record because he never takes risky cases.

I don't mean to imply that work should not be done on this, just that a complete solution (even for something as simple as "does this person honor contracts" is probably going to evade us forever and tools for managing the information we can get about potential trading partners is a good place to start.