Quote from: tagada on May 11, 2013, 12:32:55 PM

Quote from: TobbeLino on May 11, 2013, 11:15:33 AM

But the problem with fetching historical trade data at given points in time with MtGox API is that the bot only wants ONE trade (the first one) from each chunk of trades. The rest of the data is usually not relevant since we need samples at the correct points in time (if the sample interval is very short, there COULD be more than one useful trade in the chunk, but in most cases there are probably more than 1000 trades between each sample time, so only the first trade is useful. So in most cases we have to make one call for each sample). If there was a way to specify how many trades to fetch with each call, I would set it to 1, but as far as I know that's not possible with any API version...

But caching data should cure some of the problem. Then you could at least restart the bot/browser without a new storm of calls to MtGox. But as you said, it's probably ok - while developing, I need to restart the bot all the time, causing 144 calls to be made, and I don't think I've been blocked...

Good point. But if the bot expects each request to be roughly equivalent to one sample and uses only one price from each request (looks like it uses the opening price of each sample), that seems like a pretty rigid and ineffective way to go. It seems to expect each reply to be equivalent to a sample size.

So maybe instead of making n requests and pick the opening price from each, it would be more effective to use all those trades to build your own candles from it? Take a look at goxtool's slot_fullhistory() and OHLCV() functions here https://github.com/prof7bit/goxtool/blob/master/goxapi.py as this is pretty much the way it does it, building its own Open-High-Low-Close-Volume candles from all the fetched history. That could give you a relative independence from MtGox's response size since you'd be fetching for a total history duration rather than making a precise number of requests in which only one price is picked from each. Does that make sense?

The API's the bot currently uses allow it to specify from which time we want the chunk, so the first trade in each reply is always "on time". But we can't specify how big the chunk should be (in our case we only need it to be 1 trade).