Post
Topic
Board Service Announcements
Re: [ANN] Bitcoin-Analytics.com - price correction, additional currencies
by
Masqurin
on 17/04/2013, 04:27:15 UTC
Without a detailed rundown of your system my ability to evaluate your performance capabilities is limited but I'll try.

1) Server-side performance shouldn't be a problem for these filters, they only take O(n) additional arithmetic operations per update (n = number of orders in the orderbook) and the same amount of memory as whatever you're using to store the volume data right now.

2) When sending data to the client during an update, do you send all data or just the update delta? I'm not sure what sort of data the client keeps track of, but assuming you have all the calculations done server side the update delta should be very small, for each graph you should only need to send one additional datapoint per line per update.

3) It seems the client is receiving data for all currencies and exchanges, even the ones that aren't shown. You can avoid some data transfer by sending it selectively. For example, I only care about mtgox data in USD,  there are 13 other tabs containing data I don't need. What I would do is have an option in the user's preferences for the default exchange and currency, and only load and display that data by default. Request and subscribe to the other tabs only when the user requests it.

4) Similar to 3), you don't have to show all charts by default and the client doesn't need data for hidden charts. As I suggested before, add a control to hide/show each chart, and have an option for a default set of charts in user preferences. Only request and subscribe to data for a chart if and when it's shown.

For each series you'll have 24*60*4 = 6240 datapoints. Each datapoint is two floats, i.e. 8 bytes, yielding about 49kB of raw floats per series (not sure how the data is structured, e.g. if you're using JSONs there will be more overhead). I'm not sure what kind of compression ratios you're getting, a cursory search suggests gzip doesn't do a very good job of compressing floats. Still, it shouldn't be a problem for most people and you may be able to reduce it further by using a compression algorithm that's optimized for floats.

With regards to the filtering I suggested, I think there's no real point sending the raw data to the client; as it is the data is only useful for showing relative change in volume for either bids or asks. Seeing as orders that are very far from where the trades are happening don't really affect the prices, I don't think you lose anything by reducing the weights assigned to them. In fact by discounting spam orders and far-out orders it would better link relative change in volume with change in price since the orders that matter are given more importance.