Something to keep in mind whenever someone bring up past prediction performance is that distinguishing between luck and skill is very difficult.
http://www.ffscambridge.com/blog/post/taleb_on_distinguishing_between_investing_skill_and_random_chanceConsider a group of 10,000 investment managers whose investment success is comparable to that of a fair coin toss. Each manager has a 50-50 chance of making or losing 10% in any year. At the end of each year the losing managers are thrown out of the game.
After the first year 5,000 managers have made money and the rest have bailed out. After seven years there will still be 78 managers who have made money continuously, purely on the basis of random chance. Their portfolios are up a total of almost 95% after 7 years. If these folks then go off and start their own hedge funds, theres no way to know that their apparent genius was merely luck, and theyll probably attract a lot of investors.
Now suppose that instead of giving the managers an even chance of making or losing money, we start with a pool of 10,000 relatively incompetent managers. These individuals are more likely to lose money than to make it: they each have a 45% chance of making a profit and a 55% chance of losing money. At the end there are still 37 fund managers who have made 10% a year for seven years. Even though this group is more likely to lose money than to make it, a few will still run up impressive gains and develop reputations as hot managers.
This demonstrates that if there are lots of fund managers, a certain number of them can look like geniuses when in fact they are merely lucky, even if, in reality, theyre moderately incompetent! As Taleb notes, knowing that a manager has an extraordinary investment track records doesnt tell you anything: without knowing how many managers out there have tried and failed, we will not be able to assess the validity of the track record. If the manager population were small, a good track record would be impressive. But in fact, if you count all the money managers in the U.S. alone, 10,000 underestimates the reality.