Post
Topic
Board Gambling
Re: Proving that my gambling script works.
by
o_e_l_e_o
on 03/03/2018, 20:59:57 UTC
It is also worth pointing out that 20 runs is statistical noise.

I set up a contigency table, played with the numbers and ran some chi-squared tests on them. I calculated for a p-value of 0.05 (This means that the results we achieve would be achieved by random chance 5% of the time i.e. we are 95% confident the script is working as advertised. This is a common minimum standard required across most fields of scientific.)

Even if the script could completely eliminate the house edge (which it can't), we would need around 6000 runs to obtain a p-value of <0.05. If the script could half the house edge from 1% to 0.5%, we would need around 23000 runs to obtain a p-value of <0.05.
5% still doesn't prove a script works: this would lead to the false result that 1 out of 20 gambling scripts is EV+. That can't be right.
It can be a good method to disprove 95% of the gambling scripts, but it doesn't prove that the remaining 5% works. Run the same test again, and another 95% will fail.

You are absolutely correct, but we can change the p-value to anything we want. 0.05 (i.e. 5%) is a commonly used minimum standard as I said, but we could equally decide to be more rigorous by increasing the number of runs thereby resulting in a smaller p-value.