Great!
I think the issue with the methodology you are proposing to study the quality of MA´s system suffers from the fact, that the results are influenced by the trading skills of individuals. So at the end you will still not now if the results are due to Socrates or do the superb trading skills. Everyday in the markets there are plenty of winners that are using their own systems. Does technical analysis work ? Sure! but there are a lot of traders loosing money with that system. Is that a fault of technical analysis ?
The accuracy of Socrates can be only tested against the predictions the system is generating and not for specifics trades that are on the responsibility of individuals. It predicted a high in Nasdaq in May. We have a high in May so far. Bitcoin reversals predicted an uptrend in Bitcoin: we are having a bitcoin uptrend. No weekly bearish reversals have been elected so far. And the next high / low is supposed to be in June. We will see.
Between May first and today the system did not predict any specific weekly direction for Nasdaq as no weekly reversals have been elected (also not any monthly, daily I am not following). This is exactly what is happening now. If reversals are elected tomorrow , I will post it here and then we can see the results one week after (where we should consider that reversals act similar as support / resistance levels) So far, I can not complain in any aspect about the prediction of the system as long as I am following it.
I disagree with that notion. You said that some people make money and some people lose money, with the technical analysis systems. That is like saying some people exercise and get better health while others do the same and get injured. Each system is different. My system is different, and the way I pick price points is possible unique. I don't use Socrates at all anymore, all the forecasts I make are my own. I am simply demonstrating that clear forecasts can be made without ambiguity.
See, the idea that what Socrates claims and what actually happened is where I have a problem. The claims are NOT exact, so you can say in hindsight it was right. So if it went the other way, you could say it was right too. It picked a February high, and that did NOT happen. There was a Panic Cycle in August for the Dow and turning point. That didn't happen either. It picked a November low in gold. That did not happen. I can go on and on about these claims that didn't work. Sometimes it does work- I've given credit where it is due and he picked the low in December and the January high. However, he made a LOT of mistakes too.
Yes, making a CLEAR prediction in what Armstrong is saying beforehand is important, because otherwise it is hindsight bias and means nothing. Anyone can say they were right after the fact happens already.