I don't doubt that theymos is continuing to assess the extent to which the actual behavior is playing out as a failure or a success, and likely the assessment is going to come out somewhere in the middle rather than your seeming presumption that the system has been a failure and that some goals have not been reached.
A bunch of assumptions there. But let me ask you to assume something real quick
do you believe they set metric based goals that they have been assessing all along and seeing how things are going? Meaning, they said We expect X% of Y Account will do Z 20% more often after the deployment of Merit? Thats how I would have managed this.
Well, I would imagine that there is an attempted measurement of account farming and shit posting, so these kinds of measures might not be precise to verify whether the merit system implementation is causing movement in the preferred direction. Furthermore, there could be issues with merit sources not spending their smerits or engaging in borderline abusive use of their source merits that might need to be measured to decide whether to change merit source incentives. Some behaviors are going to be easier measured than others, and there could even be some considerations about changing the measurement tools based on kinds of behaviors that are detected.
I am guessing you are saying they may have set vague goals? I really don't see an answer to my question. Since you are setting yourself up as some sort of Merit system defense counsel, and you don't know what metric based goals they set, it seems they may not have set any. If that is the case, that's a botched job.
This is a really an important point as the likelihood of success or failure absolutely is impacted by goal setting. First, if they have no metric based goals, they will never be able to determine if they actually achieved their goals. Second, if they have no metric based goals, they will not know how to make course corrections (which are pretty much always needed because with people, things never go perfectly as planned).
When you launch some major intervention impacting thousands of people and don't have any metric based goals, you know what that is called? It is called guessing. If they are guessing, they are playing with fire.