In my last post (7NTxx gains over 74% popularity) I wondered why learning results are so bad. But, I understood that the underlying idea was not correct.
My RNN started from random bidding, then is was learned to improve its performance. So, instead of comparison to proper bidding in human sense, we should compare resulting biddings statistics to random bidding statistics.
Looking at the results beloew, I see (no surely, since there is no statistical error analysis), that RNN does better than the random player. Apart from increasing 7NTxx probability, in case of RNN usually the probabilities of high contracts are lower than the probabilities of “normal” ones.
I run 100.000 of random bidding trials and got the following results:
| Contract | Probability | Count |
|---|---|---|
| 7NTxx | 52.72% | 52724 |
| 7NT | 23.38% | 23375 |
| 7NTx | 17.65% | 17653 |
| 7S | 2.68% | 2681 |
| 7Sx | 1.06% | 1058 |
| 7Sxx | 0.94% | 941 |
| 7H | 0.69% | 691 |
| 7D | 0.26% | 256 |
| 7Hx | 0.19% | 190 |
| 7C | 0.09% | 94 |
| 7Hxx | 0.09% | 88 |
| 6NT | 0.06% | 60 |
| 7Dx | 0.05% | 49 |
| 6S | 0.04% | 35 |
| 7Cx | 0.02% | 22 |
| 6H | 0.02% | 21 |
| 7Dxx | 0.02% | 17 |
| 6C | 0.01% | 8 |
| 6D | 0.01% | 5 |
| 6NTx | 0.01% | 5 |
| 6Sx | 0.00% | 3 |
| 6Cx | 0.00% | 3 |
| 5H | 0.00% | 2 |
| 5NT | 0.00% | 2 |
| 4S | 0.00% | 2 |
| 5S | 0.00% | 2 |
| 7Cxx | 0.00% | 2 |
| 4H | 0.00% | 2 |
| 5C | 0.00% | 2 |
| 6NTxx | 0.00% | 1 |
| 4C | 0.00% | 1 |
| 4NT | 0.00% | 1 |
| 6Dx | 0.00% | 1 |
| 1C | 0.00% | 1 |
| 6Hx | 0.00% | 1 |
| 4Sx | 0.00% | 1 |
We can calculate the probability of final contracts in some simple cases, directly. E. g.
- the probability of PASS (4 passes) is: (1/36)^4 (the same for each bidding like: some_bid, pass, pass, pass)
- the probability of 7NT contract after 7NT bid has happened is 0.25
- the probability of 7NTx (doubled) contract after 7NT bid is 0.1875
- and so the probability of 7NTxx (redoubled) contract after 7NT bid is 1-0.25-0.1875 = 0.5625
So the probability of 7NTxx contract (without assumption that 7NT was bid) is not greater than 0.5625.
After nightly run, on 1.320.000 trails the results are:
| Contract | Probability | Count |
|---|---|---|
| 7NTxx | 52.67% | 695222 |
| 7NT | 23.44% | 309436 |
| 7NTx | 17.60% | 232270 |
| 7S | 2.72% | 35908 |
| 7Sx | 1.05% | 13823 |
| 7Sxx | 0.92% | 12209 |
| 7H | 0.69% | 9127 |
| 7D | 0.25% | 3254 |
| 7Hx | 0.19% | 2443 |
| 7C | 0.11% | 1492 |
| 7Hxx | 0.09% | 1216 |
| 6NT | 0.06% | 750 |
| 7Dx | 0.05% | 673 |
| 6S | 0.03% | 455 |
| 7Cx | 0.02% | 280 |
| 6H | 0.02% | 250 |
| 7Dxx | 0.02% | 222 |
| 6D | 0.01% | 185 |
| 6NTx | 0.01% | 97 |
| 6C | 0.01% | 96 |
| 5NT | 0.01% | 88 |
| 7Cxx | 0.01% | 73 |
| 6Sx | 0.00% | 61 |
| 5S | 0.00% | 58 |
| 5H | 0.00% | 42 |
| 5D | 0.00% | 38 |
| 6Hx | 0.00% | 34 |
| 5C | 0.00% | 22 |
| 6NTxx | 0.00% | 21 |
| 6Dx | 0.00% | 16 |
| 4NT | 0.00% | 15 |
| 4S | 0.00% | 13 |
| 6Sxx | 0.00% | 12 |
| 4D | 0.00% | 11 |
| 5NTx | 0.00% | 9 |
| 6Cx | 0.00% | 9 |
| 5Sx | 0.00% | 8 |
| 3NT | 0.00% | 6 |
| 4H | 0.00% | 5 |
| 5Hx | 0.00% | 5 |
| 3D | 0.00% | 5 |
| 4C | 0.00% | 4 |
| 5Cx | 0.00% | 4 |
| 6Hxx | 0.00% | 4 |
| 2H | 0.00% | 4 |
| 3C | 0.00% | 4 |
| 4NTx | 0.00% | 3 |
| 3H | 0.00% | 2 |
| 2NT | 0.00% | 2 |
| 1C | 0.00% | 2 |
| 6Dxx | 0.00% | 2 |
| 2S | 0.00% | 2 |
| 2C | 0.00% | 2 |
| 1D | 0.00% | 1 |
| 5Dx | 0.00% | 1 |
| 4Sx | 0.00% | 1 |
| 1NT | 0.00% | 1 |
| 3S | 0.00% | 1 |
| 1H | 0.00% | 1 |
| 4Dx | 0.00% | 1 |