Hi, about a month ago a set up a bitcoind 0.9.2.1 + coinsparkd node. Everything went pretty well until yesterday when i decided to upgrade the bitcoin layer. I installed bitcoin 0.11 and now coinsparkd crash every 30mins.
Coinsparkd uses shared memory to communicate with bitcoind
coinspark@cfengine:~$ coinsparkd -asset=369721-162192-13158 start
Starting coinsparkd...
coinspark@cfengine:~$ Cannot start database server, probably server with shared memory key 12595 is already started
If you are sure there is no other server using this shared memory key, please run
ipcrm -m 12595
and try starting the server again.
This is a log where you can see the coinspark error:
coinspark@cfengine:~/.coinspark/assets/369721-162192-13158$ tail -f ./tracker.log
2015-09-04 14:00:15.445 R-5E9B86B0 R C-0061 Mempool update successful Time 0.151s, Iterations: 3, Accepted: 161 txs, Remaining: 0 txs, Total: 161 txs
2015-09-04 14:00:20.651 R-5E9B8807 R C-0061 Mempool update successful Time 0.165s, Iterations: 3, Accepted: 166 txs, Remaining: 0 txs, Total: 166 txs
2015-09-04 14:00:25.845 R-5E9B8966 R C-0061 Mempool update successful Time 0.145s, Iterations: 3, Accepted: 170 txs, Remaining: 0 txs, Total: 170 txs
2015-09-04 14:00:31.062 R-5E9B8AD6 R C-0061 Mempool update successful Time 0.176s, Iterations: 3, Accepted: 178 txs, Remaining: 0 txs, Total: 178 txs
2015-09-04 14:00:36.271 R-5E9B8C54 R C-0061 Mempool update successful Time 0.160s, Iterations: 3, Accepted: 185 txs, Remaining: 0 txs, Total: 185 txs
2015-09-04 14:00:41.457 R-5E9B8DD8 R C-0061 Mempool update successful Time 0.150s, Iterations: 3, Accepted: 188 txs, Remaining: 0 txs, Total: 188 txs
2015-09-04 14:00:46.585 R-5E9B8F6A R C-0061 Mempool update successful Time 0.091s, Iterations: 3, Accepted: 195 txs, Remaining: 0 txs, Total: 195 txs
2015-09-04 14:01:21.000 R-5E9B8F6E FTL! C-0002 Timeout
2015-09-04 14:01:21.001 R-5E9B8F6E FTL! C-0057 Get mempool failed Error: 00000006
2015-09-04 14:01:21.001 R-5E9B8F6E FTL! C-0019 Error while processing mempool Error: 00000006
Before the crash:
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x00003133 98304 coinspark 666 6308096 1
After the crash
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x00003133 98304 coinspark 666 6308096 0
After searching through all the source code i found the point where the "Timeout" error have been sent out.
Github
linkThe method is "TrackerProcessAsyncMessage" in "tracker/tracker.cpp:63", this is the code:
/*
* --- Asynchronous message processing. Send message, wait for [count] valid responses.
*/
cs_int32 TrackerProcessAsyncMessage(cs_int32 count)
{
cs_int32 StopTime;
cs_int32 err;
err=CS_ERR_NOERROR;
StopTime=(cs_int32)cs_TimeNow()+CS_DCT_MSG_TIMEOUT;
cs_int32 size;
size=g_Message->GetSize();
cs_LogMessage(g_Log,CS_LOG_MINOR,"C-0001","Bitcoin message request: ",(cs_char*)(g_Message->GetData())+4);
g_ProcessMessage=count; // Setting response count, if 0 receiving thread ignores all messaged
err=g_Connection->SendNow(g_Message->GetData(),size); // Sending request
if(err)
{
return err;
}
while(g_ProcessMessage) // Waiting for response, receiving thread decrements g_ProcessMessage
{
if((cs_int32)cs_TimeNow()>=StopTime)
{
cs_LogMessage(g_Log,CS_LOG_FATAL,"C-0002","Timeout","");
return CS_ERR_TIMEOUT;
}
__US_Sleep(1);
}
return g_ErrorCode;
}
So when "cs_TimeNow()" is greater or equal "StopTime", the "timeout" error will be fired.
The "StopTime" variable is
StopTime=(cs_int32)cs_TimeNow()+CS_DCT_MSG_TIMEOUT;
And he we are, CS_DCT_MSG_TIMEOUT is defined in "coinsparkd/tracker/bitcoin.h:38"
#define CS_DCT_MSG_TIMEOUT 30
The "Get mempool failed" error is triggered in "
coinsparkd/tracker/block.cpp:900" into the "ProcessMemPool" method.
and the "Error while processing mempool" error is triggered in the "cs_Tracker" method into the
"tracker/tracker.cpp" file at row 965:
if(g_State->m_AssetDB->m_lpHandlerShMem[CS_OFF_DB_SHMEM_HEAD_SIGNAL] == 0)
{
if(block>g_State->m_BlockChainHeight)
{
sprintf(msg,"Blockchain height: %d",g_State->m_BlockChainHeight);
cs_LogMessage(g_Log,CS_LOG_MINOR,"C-0056","Processing mempool",msg);
err=RemoveBlock(1);
if(err)
{
sprintf(msg,"Error: %08X",err);
cs_LogMessage(g_Log,CS_LOG_FATAL,"C-0018","Cannot clear memory pool",msg);
goto errorlbl;
}
for(mempool_attempt=0;mempool_attempt<1;mempool_attempt++)
{
err=ProcessMemPool(new_op_return_file);
new_op_return_file=0;
}
if(err)
{
sprintf(msg,"Error: %08X",err);
cs_LogMessage(g_Log,CS_LOG_FATAL,"C-0019","Error while processing mempool",msg);
// goto errorlbl;
err=RemoveBlock(1);
}
}
}
I'm digging into the coinspark and bitcoind 0.11 source code to understand why this is happening. Can someone help me?