Quote from: hmage on June 07, 2016, 07:57:26 PM

For the future, gcc has a flag: -fstack-usage

when compiling it generates *.su files that have info how much bytes a function would need for stack.

After adding that, recompiling cpuminer-opt on core2 and then printing all *.su files, sorted by size:

Code:

$ find . -iname '*.su' -print0 | xargs -0 cat |sort -k2n|tail|column -t
x17.c:212:6:x17hash_alt 3904 static
cpu-miner.c:2689:5:main 4144 static
x17.c:87:13:x17hash 4256 static
hodl-wolf.c:28:5:scanhash_hodl_wolf 4304 static
scrypt.c:696:12:scanhash_scrypt 7680 dynamic,bounded
hmq1725.c:143:13:hmq1725hash 7744 static
scrypt.c:648:13:scrypt_1024_1_1_256_24way 9088 dynamic,bounded
m7mhash.c:195:5:scanhash_m7m_hash 12464 dynamic,bounded
api.c:511:13:api 17136 dynamic,bounded
cryptonight.c:172:6:cryptonight_hash_ctx 2097648 static

You might want to increase max stack size in makefile to 3mb or more, setting it to 2MB isn't enough because you need 2097648 bytes just for that function.

Thanks for the info. I increased the stack size in Makefile.am to 3 MB but it made no difference.
AES still produces rejects and non-aes still crashes.

There is apparently another problem with the AES version other than the stack overflow (that is a huge stack
compared with the other algos) because I solved that by reducing the local variables.

So the situation now for AES seems the same code with no superficial Windows hooks works on Linux but produces rejects
on Windows. By superficial I mean checks for Windows in cryptonight code. There may be some low level hooks
in common code also used by other algos.

The core2 build still crashes after moving ctx to global and increasing the stacksize to 3 MB in Makefile.am. It's either
the same crash or a different one. I haven't followed up because my focus is on AES first.