Quote from: peedee on June 23, 2011, 11:20:04 AM

Quote from: gigabytecoin on June 23, 2011, 04:18:29 AM

Quote from: peedee on June 22, 2011, 10:47:35 PM

Its actually like this :

$fiveMinutesAgo = time();
$fiveMinutesAgo -= 60*5;

$workers = mysql_query("SELECT count(*) FROM `stats_userMHashHistory` WHERE `timestamp` >= $fiveMinutesAgo AND `mhashes` > 0");

// Verify it worked
if (!$workers) echo mysql_error();

$nrworkers = mysql_fetch_array($workers);

Then where ever you want to display the amount of workers :

echo "$nrworkers[0]"

Btw the stats_userMHashHistory table is filling up pretty quickly, especially if the number of workers that have been created increases, I think this will be become a scalability bottleneck.

Just don't use MySQL then. CouchDB or another database type optimized for large databases.

What about purging / summarizing? Some stats don't need to be available up to the lowest timeslice level forever.

For example :
Keep timeslice level data for 1 week
Keep hour data for 1 month
Keep day data for 3 months
Keep month data for 5 years

Can make nice graphs out of that and save A LOT of space. A bottleneck will be reached someday, keep adding resources gets expensive / risky so it's better to start summarizing while it's still the early stage than to find out you'll need hour long running jobs to purge / summarize your data later.

Or am I missing something and do we need to keep everything we collect ?

This all sounds pretty good idears, I think most of the data wont be neccesary.