Quote from: chodpaba on June 30, 2011, 04:46:03 PM

Quote from: KaptainBlaZzed on June 30, 2011, 02:12:24 PM

Quote from: chodpaba on June 29, 2011, 08:24:12 PM

Quote from: KaptainBlaZzed on June 29, 2011, 02:22:24 PM

Quote from: mmortal03 on June 27, 2011, 10:56:05 PM

Looks good. Now we need someone to write one that does do a conservative estimate of additional difficulty increases. Difficulty increases aren't completely predictable, but writing an estimator that uses the exponential curve that best fits previous increases would be a good start.

I would gladly write that in if somebody would help me with the mathematical calculations to do this.

Anybody have an idea on how to do this?

Take a look at this.
https://forum.bitcoin.org/index.php?topic=13339.msg183351#msg183351

The thing is, it is not like figuring out an exact Difficulty in the future, but a probability distribution. This is what you need to determine risk vs. reward for an investment. Figure out what your risk tolerance is and then decide how much you are willing to risk.

It is not OK to do this: "Price and Difficulty will be X two months from now so I can calculate my return from that."

But it is OK to do this: "There is a 50% probability that Price and Difficulty will be above/below level X two months from now. That is/is not an acceptable risk to me."

The numbers you linked to look interesting, but could you explain the calculations and how you got the numbers?

A good place to start is to look at the correlation between a moving average of price with Difficulty. I think you will find that a 10-12 week moving average will give you a fairly linear correlation. You can apply Pearson to give you an objective measure of the degree of correlation. http://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient

Once you have that you will find that Difficulty can be approximated by multiplying the moving average of price (the one you found to have a high degree of correaltion) with some factor. Let's say you use a factor that is in between all the ratios of Difficulty to the moving average of price going back as far as your data goes. You will find that these ratios fall on a more or less normal probability distribution. http://en.wikipedia.org/wiki/Normal_distribution So, you can characterize the likelihood that the actual Difficulty will fall so far from your projection.

There are several methods you can use from there to dial in results that give you less variance and a more reliable outcome. One method I use is to max out the degree to which the moving average of price is correlated with Difficulty by applying a custom weighting to the average. You could try to go for accuracy and optimize it by minimum average deviation. It will occasionally give you some spectacularly close hits, but this is statistically fragile. That is, it can produce more outliers. So, what I prefer to do is to minimize the expected maximum deviation. This method specifically targets outliers to help ensure your projection falls within a certain range of probability. It won't always give you the most accurate result, but it will give you more reliable results with in an advertised range.

All this can give you pretty reliable results for a projection that is one re-target out, like I have been getting. But the real trick is to try and figure out what Difficulty is likely to be going further out. This is because future Difficulty is highly dependent on what the exchange rate is going to be. What I have been experimenting with to try and go out farther with the projection, without trying to determine what price is going to be, is to simply drop terms from the moving average going forward. But the shorter the average gets the less correlated it gets and the more variance it produces. So I have to use a bunch of tricks to try and keep the variance down. One of those is to apply a weighting to the truncated average, another is to optimize it to reduce outliers as described above. Yet another is to eliminate outliers by discarding earlier data samples that do not correlate well. But this is a slippery slope, because it can make your data more statistically fragile and actually produce more outliers.

The next step, which I am still working on, is to come up with meaningful way to do actual price projections. And more importantly, to attribute a meaningful probability distribution to price projections. This is a work in progress.

I wish there were some kind of simple formula to use but as soon as you put something like that together it breaks. I continually re-evaluate and update the models I use as more data comes in.

An interesting project, and I feel one that has had some impressive results, but I wonder if your goals aren't a bit too lofty. Modelling human greed/ambition/deceit and so on has a pretty poor track record (consider the billions poured into the biggest brains on wall street). If you enjoy it though, more power to you, and I'll be subbing your projection thread to watch with interest your progress.