Quote from: TierNolan on January 19, 2014, 07:25:27 PM

I am getting this spam. File corruption or something?

Code:

2014-01-19 19:22:51.214727 > File "/home/miner1/p2pool/p2pool/util/memoize.py", line 28, in b
2014-01-19 19:22:51.214885 > res = f(*args)
2014-01-19 19:22:51.215046 > File "/home/miner1/p2pool/p2pool/util/skiplist.py", line 21, in __call__
2014-01-19 19:22:51.215200 > self.skips[pos] = math.geometric(self.p), [(self.previous(pos), self.get_delta(pos))]
2014-01-19 19:22:51.215357 > File "/home/miner1/p2pool/p2pool/util/forest.py", line 18, in previous
2014-01-19 19:22:51.215510 > return self.tracker._delta_type.from_element(self.tracker.items[element]).tail
2014-01-19 19:22:51.215695 > exceptions.KeyError: None
2014-01-19 19:22:51.226186 > Share check failed:
2014-01-19 19:22:51.226606 > Traceback (most recent call last):
2014-01-19 19:22:51.226840 > File "/home/miner1/p2pool/p2pool/util/deferral.py", line 233, in it
2014-01-19 19:22:51.227076 > res = gen.send(cur) # external code is run here
2014-01-19 19:22:51.227285 > File "/home/miner1/p2pool/p2pool/util/deferral.py", line 284, in _worker
2014-01-19 19:22:51.227495 > self.func(*self.args, **self.kwargs)
2014-01-19 19:22:51.227727 > File "/home/miner1/p2pool/p2pool/node.py", line 304, in clean_tracker
2014-01-19 19:22:51.227955 > best, desired, decorated_heads = self.tracker.think(self.get_height_rel_highest, self.bitcoind_work.value['previous_block'], self.bitcoind_work.value['bits'], self.known_txs_var.value)
2014-01-19 19:22:51.228188 > File "/home/miner1/p2pool/p2pool/data.py", line 452, in think
2014-01-19 19:22:51.228440 > if self.attempt_verify(share):
2014-01-19 19:22:51.228659 > --- ---
2014-01-19 19:22:51.228866 > File "/home/miner1/p2pool/p2pool/data.py", line 431, in attempt_verify
2014-01-19 19:22:51.229093 > share.check(self)
2014-01-19 19:22:51.229293 > File "/home/miner1/p2pool/p2pool/data.py", line 317, in check
2014-01-19 19:22:51.229516 > share_info, gentx, other_tx_hashes2, get_share = self.generate_transaction(tracker, self.share_info['share_data'], self.header['bits'].target, self.share_info['timestamp'], self.share_info['bits'].target, self.contents['ref_merkle_link'], [(h, None) for h in other_tx_hashes], self.net, last_txout_nonce=self.contents['last_txout_nonce'])
2014-01-19 19:22:51.229820 > File "/home/miner1/p2pool/p2pool/data.py", line 162, in generate_transaction
2014-01-19 19:22:51.230052 > 65535*net.NEW_SPREAD*bitcoin_data.target_to_average_attempts(block_target),
2014-01-19 19:22:51.230262 > File "/home/miner1/p2pool/p2pool/util/memoize.py", line 28, in b
2014-01-19 19:22:51.230498 > res = f(*args)
2014-01-19 19:22:51.230725 > File "/home/miner1/p2pool/p2pool/util/skiplist.py", line 21, in __call__
2014-01-19 19:22:51.230955 > self.skips[pos] = math.geometric(self.p), [(self.previous(pos), self.get_delta(pos))]
2014-01-19 19:22:51.231171 > File "/home/miner1/p2pool/p2pool/util/forest.py", line 18, in previous
2014-01-19 19:22:51.231380 > return self.tracker._delta_type.from_element(self.tracker.items[element]).tail
2014-01-19 19:22:51.231589 > exceptions.KeyError: None
2014-01-19 19:22:51.241558 > Share check failed:
2014-01-19 19:22:51.241947 > Traceback (most recent call last):
2014-01-19 19:22:51.242221 > File "/home/miner1/p2pool/p2pool/util/deferral.py", line 233, in it
2014-01-19 19:22:51.242466 > res = gen.send(cur) # external code is run here
2014-01-19 19:22:51.242717 > File "/home/miner1/p2pool/p2pool/util/deferral.py", line 284, in _worker
2014-01-19 19:22:51.242987 > self.func(*self.args, **self.kwargs)
2014-01-19 19:22:51.243234 > File "/home/miner1/p2pool/p2pool/node.py", line 304, in clean_tracker
2014-01-19 19:22:51.243521 > best, desired, decorated_heads = self.tracker.think(self.get_height_rel_highest, self.bitcoind_work.value['previous_block'], self.bitcoind_work.value['bits'], self.known_txs_var.value)
2014-01-19 19:22:51.243855 > File "/home/miner1/p2pool/p2pool/data.py", line 452, in think
2014-01-19 19:22:51.244122 > if self.attempt_verify(share):
2014-01-19 19:22:51.244400 > --- ---
2014-01-19 19:22:51.244653 > File "/home/miner1/p2pool/p2pool/data.py", line 431, in attempt_verify
2014-01-19 19:22:51.244937 > share.check(self)
2014-01-19 19:22:51.245229 > File "/home/miner1/p2pool/p2pool/data.py", line 317, in check
2014-01-19 19:22:51.245493 > share_info, gentx, other_tx_hashes2, get_share = self.generate_transaction(tracker, self.share_info['share_data'], self.header['bits'].target, self.share_info['timestamp'], self.share_info['bits'].target, self.contents['ref_merkle_link'], [(h, None) for h in other_tx_hashes], self.net, last_txout_nonce=self.contents['last_txout_nonce'])
2014-01-19 19:22:51.245820 > File "/home/miner1/p2pool/p2pool/data.py", line 162, in generate_transaction
2014-01-19 19:22:51.246085 > 65535*net.NEW_SPREAD*bitcoin_data.target_to_average_attempts(block_target),
2014-01-19 19:22:51.246394 > File "/home/miner1/p2pool/p2pool/util/memoize.py", line 28, in b
2014-01-19 19:22:51.246914 > res = f(*args)
2014-01-19 19:22:51.247158 > File "/home/miner1/p2pool/p2pool/util/skiplist.py", line 21, in __call__
2014-01-19 19:22:51.247407 > self.skips[pos] = math.geometric(self.p), [(self.previous(pos), self.get_delta(pos))]
2014-01-19 19:22:51.248801 > File "/home/miner1/p2pool/p2pool/util/forest.py", line 18, in previous
2014-01-19 19:22:51.254732 > return self.tracker._delta_type.from_element(self.tracker.items[element]).tail
2014-01-19 19:22:51.255015 > exceptions.KeyError: None
2014-01-19 19:22:51.269010 > Share check failed:
2014-01-19 19:22:51.272994 > Traceback (most recent call last):
2014-01-19 19:22:51.273263 > File "/home/miner1/p2pool/p2pool/util/deferral.py", line 233, in it
2014-01-19 19:22:51.273532 > res = gen.send(cur) # external code is run here
2014-01-19 19:22:51.273801 > File "/home/miner1/p2pool/p2pool/util/deferral.py", line 284, in _worker
2014-01-19 19:22:51.274067 > self.func(*self.args, **self.kwargs)
2014-01-19 19:22:51.274313 > File "/home/miner1/p2pool/p2pool/node.py", line 304, in clean_tracker
2014-01-19 19:22:51.274594 > best, desired, decorated_heads = self.tracker.think(self.get_height_rel_highest, self.bitcoind_work.value['previous_block'], self.bitcoind_work.value['bits'], self.known_txs_var.value)
2014-01-19 19:22:51.274892 > File "/home/miner1/p2pool/p2pool/data.py", line 452, in think
2014-01-19 19:22:51.275147 > if self.attempt_verify(share):
2014-01-19 19:22:51.275400 > --- ---
2014-01-19 19:22:51.275682 > File "/home/miner1/p2pool/p2pool/data.py", line 431, in attempt_verify
2014-01-19 19:22:51.275944 > share.check(self)
2014-01-19 19:22:51.276187 > File "/home/miner1/p2pool/p2pool/data.py", line 317, in check
2014-01-19 19:22:51.276442 > share_info, gentx, other_tx_hashes2, get_share = self.generate_transaction(tracker, self.share_info['share_data'], self.header['bits'].target, self.share_info['timestamp'], self.share_info['bits'].target, self.contents['ref_merkle_link'], [(h, None) for h in other_tx_hashes], self.net, last_txout_nonce=self.contents['last_txout_nonce'])
2014-01-19 19:22:51.276791 > File "/home/miner1/p2pool/p2pool/data.py", line 162, in generate_transaction
2014-01-19 19:22:51.277054 > 65535*net.NEW_SPREAD*bitcoin_data.target_to_average_attempts(block_target),
2014-01-19 19:22:51.277306 > File "/home/miner1/p2pool/p2pool/util/memoize.py", line 28, in b
2014-01-19 19:22:51.277561 > res = f(*args)
2014-01-19 19:22:51.277820 > File "/home/miner1/p2pool/p2pool/util/skiplist.py", line 21, in __call__
2014-01-19 19:22:51.278084 > self.skips[pos] = math.geometric(self.p), [(self.previous(pos), self.get_delta(pos))]
2014-01-19 19:22:51.278357 > File "/home/miner1/p2pool/p2pool/util/forest.py", line 18, in previous
2014-01-19 19:22:51.278616 > return self.tracker._delta_type.from_element(self.tracker.items[element]).tail
2014-01-19 19:22:51.278884 > exceptions.KeyError: None

I am seeing same.

Code:

2014-01-19 12:34:09.334941 > --- ---
2014-01-19 12:34:09.334967 > File "/Users/ageo/src/p2pool/p2pool/data.py", line 431, in attempt_verify
2014-01-19 12:34:09.334994 > share.check(self)
2014-01-19 12:34:09.335020 > File "/Users/ageo/src/p2pool/p2pool/data.py", line 320, in check
2014-01-19 12:34:09.335047 > raise ValueError('share_info invalid')
2014-01-19 12:34:09.335074 > exceptions.ValueError: share_info invalid
2014-01-19 12:34:09.359392 > Share check failed:
2014-01-19 12:34:09.359472 > Traceback (most recent call last):
2014-01-19 12:34:09.359516 > File "/Users/ageo/src/p2pool/p2pool/util/deferral.py", line 233, in it
2014-01-19 12:34:09.359555 > res = gen.send(cur) # external code is run here
2014-01-19 12:34:09.359603 > File "/Users/ageo/src/p2pool/p2pool/util/deferral.py", line 284, in _worker
2014-01-19 12:34:09.359652 > self.func(*self.args, **self.kwargs)
2014-01-19 12:34:09.359789 > File "/Users/ageo/src/p2pool/p2pool/node.py", line 304, in clean_tracker
2014-01-19 12:34:09.359856 > best, desired, decorated_heads = self.tracker.think(self.get_height_rel_highest, self.bitcoind_work.value['previous_block'], self.bitcoind_work.value['bits'], self.known_txs_var.value)
2014-01-19 12:34:09.359916 > File "/Users/ageo/src/p2pool/p2pool/data.py", line 452, in think

In periods 5 to 10 minutes. bfgminer fails over to secondary pool then returns once p2pool errors subside.

Any ideas as to cause?

Thanks,
-alan

EDIT: It's actually happening again as I type. P2pool filled with the errors above and miners failing over. 1:19PM PST. Anybody else seeing same?

EDIT 2: Looks like something local.

Code:

n' + ''.join(traceback.format_stack())
2014-01-19 13:16:21.051978 > ########################################
2014-01-19 13:16:21.052079 > >>> Warning: LOST CONTACT WITH BITCOIND for 4.6 minutes! Check that it isn't frozen or dead!
2014-01-19 13:16:21.052118 > ########################################
2014-01-19 13:16:21.052153 P2Pool: 17395 shares in chain (17600 verified/20953 total) Peers: 0 (0 incoming)