I have not thought it through, wanted to have feedback! 
From my background the intuitive thing is to use F1 measure as a score. Or area under ROC curve. There are good wikipedia articles about these.
Precision is not always a good thing. It leads to over-fitting.
There is a book which seems pertinent, "Evidence-based Technical Analysis" although I have yet to read it -- it's next to my bed, but in another state.
http://www.amazon.com/Evidence-Based-Technical-Analysis-Scientific-Statistical/dp/0470008741/ref=sr_1_1?ie=UTF8&qid=1397582403&sr=8-1&keywords=evidence+based+technical+analysis