Accidentally Cheating at Backgammon

Why players perceive unfairness in Backgammon software

On a regular basis, inexperienced Backgammon players voice opinions about how a certain computer program or game server cheats.  The stochastic nature of the game lends itself to this kind of perception on the part of human beings.  Generally, there are a few primary reasons for this type of belief.

First, novice players often fail to recognize the complexities of the game of Backgammon, so what they perceive as an unnatural number of “lucky rolls” are not (necessarily) due to luck, but rather due to skillful play on the part of the opponent.  Expert players tend toward positions where a greater number of rolls would be considered good (i.e., “lucky”).  A higher percentage of good moves tends to make the dice appear biased in ones favor, and it is also key to good checker play.

In many cases, players also fail to understand the nature of truly random numbers.  It is often stated that, say, a certain number of doubles in a row indicates…  excuse me…  “proves” that the virtual dice are unfair when, in fact, a truly random number generator would have to produce any arbitrary sequence (whether or not a pattern is perceptible) given enough rolls.  Of course, we are talking about pseudo-random number generators (PRNG), so they are, by their very nature, not truly random.  However, one would have to do an actual study/count of the dice rolls to make any conclusion about any particular PRNG.

The reason for this need to analyze a PRNG scientifically, rather than anecdotally, seems fairly obvious.  Human beings have selective memory, which means that we tend to recall things that are out of the ordinary, so a number of doubles in a row stands out, whereas a statistically identical sequence of rolls that do not seem to show a pattern are not reported.  Likewise, a few very good (or very bad) rolls are more memorable than many run-of-the-mill rolls.

Related to this is the concept of apophenia, which is the human “experience of seeing patterns or connections in random or meaningless data.” [from Wikipedia]  Our minds have evolved to recognize patterns, so we can sometimes perceive things that are not there.  This is how people see images in clouds, hear music or sounds in white noise, and imagine divine imagery in oil stains or burnt toast.

All of these factors make it very easy for an average person to perceive unfairness in Backgammon software or servers (even in games against other human beings), and even trained experts can be fooled.

How experts demonstrate that Backgammon software is fair

There are a few key points that are usually made by experts when arguing that a particular Backgammon program does not cheat.  First, of course, one generally describes some of the aspects of the perception problem, as listed above.  In particular, reports are almost always anecdotal, so they can be dismissed quickly as having no scientific validity until somebody does an actual count and statistical analysis.

To dismiss accusations of manipulated dice (by software), the suggestion is to manually input dice rolls, which most (decent) programs allow, according to the rolls of physical dice recorded meticulously, or by changing to an alternative PRNG.  If the results stay statistically consistent, that argues against the idea that the rolls are manipulated.  Another common argument is that programs can “look ahead” to see which rolls are upcoming and make moves based on this prior knowledge, and manual input of dice rolls also removes this possibility.

Another method to test if dice rolls are being artificially manipulated is to switch sides and look for discrepancies.  In other words, start a game (or save one in progress) with a particular random number seed and play the rest of the game, recording the dice rolls for each side.  Then, restart (or load) the game and play the opposite side.  If the dice rolls remain the same, then no manipulation was done to bias the outcome.

A final, less scientific, approach is the simple “Why?” method, wherein one looks at the reasons why (and how) a programmer might decide to write a biased program.  Speaking as the primary programmer for MVP Backgammon Professional, from MVP Software, I can assure you that cheating would add a whole extra layer of (unwanted and unnecessary) complexity, so I certainly did not and would not include such code.  In fact, accusations of unfairness were troubling enough to MVP for the first version of the program that our version has a replaceable PRNG library so one can write ones own (with whatever extra checking is desired).

Possibility for Backgammon software to cheat without malice aforethought

This whole topic was reinvigorated when yet another thread appeared on recently, entitled “Jellyfish.  Cheating or just Lucky” [links to Google groups].  Through dozens of messages, some people suggested/argued that the Backgammon program Jellyfish seemed to cheat, while two other popular programs, GNU Backgammon and Snowie, did not.

Interestingly (and, n.b., anecdotally), when testing MVP Backgammon, I had a similar experience.  I was simply testing relative strength with a series of 25-point matches between my program and these others.  Whereas the strength of my neural network was comparable to the others, it got beaten significantly by Jellyfish when it rolled the dice.  When MVPBG rolled, it was much closer.  As a final test, I played one match with manual rolls, and it was again close.  At this point, I figured out the likely problem (leaving alive the possibility that it was just sheer chance).

The whole purpose of a neural network is to discover connections and patterns in provided data, and the conclusions are affected by the design of the inputs (essentially, which raw data is supplied) and, of course, the requested output(s).  In our design, we basically supplied the number of checkers on each point (in a special format), the number on the bar, and the number borne off.  This specifies a pure position in the game (with no knowledge about moves or rolls), and our outputs were designed to estimate the probability of each potential game outcome (win, loss, or winning/losing either a gammon or backgammon).  The neural network was only used for evaluation; the selection of moves was based on the evaluation of the resulting position (and cube decisions were calculated mathematically from the neural network outputs).

Theoretically, we could provide irrelevant inputs (e.g., outside temperature) and during training, their influence on the network would tend toward zero.  However, providing somewhat related data, such as the last game move, could give the neural network just enough information to begin to anticipate an outcome and bias the outputs.  More directly, providing the current dice roll, or perhaps designing the neural network to rate individual moves based on that roll, gives the network additional information that could be used to actually predict the next pseudo-random roll, especially if the particular PRNG is not very good.  After all, guessing what the next roll would be based on the position and previous roll is exactly the kind of task that neural networks are designed to solve.

Based on this observation, I suggest that it is possible that the programmers of Jellyfish may have inadvertently, and with no malicious intent whatsoever, provided their neural network with just a little too much information, and it may have taken that information to (at least partially) figure out the random number sequence and then draw conclusions that were not intended.

This would be a very interesting (and perhaps slightly startling) example of emergent behavior in a computer system.  It would, however, explain why a program could pass all of the tests to “prove” it is not cheating, but still have an observable bias when using its own dice.  I suppose we could call it “computer intuition“.  Of course, without more scientific study, it could still just be called “luck“.

13 thoughts on “Accidentally Cheating at Backgammon

  1. Horse hockey! In a statistical analysis of 320 doubles rolls, 65% went to the computer, 35% to the player. That is well skewed toward the computer, and well OFF the balance norm for 320 rolls left truly random. It CHEATS!

  2. Giving you the benefit of the doubt:

    What is “It”? Are you talking about Jellyfish or one of the other programs I mentioned?

    Just throwing out numbers is not proof. Do you have a link with the raw data, or at least describing the testing methodology? Has the experiment been replicated?

    By definition, any stochastic system in going to “skew” in some direction. If a statistical analysis was actually done, then what was the degree of statistical significance in a sample size that small?

    I can only state with certainty that (my version of) MVP Backgammon Pro does not cheat. It is always possible that another program or server *may* manipulate dice rolls, but the burden of proof lies with the accuser.

    In more practical terms, if you think that a program cheats, drop it and buy a different one. I was merely explaining the reasons for anecdotal issues and proposing a theory whereby a programmer may think that a neural network backgammon is fair when it may not be. Deliberate manipulation of dice rolls is a separate matter (and if I get a better description of the experimental methodology for your quoted results, I may take a shot at reproducing them).

  3. I have been playing JF for years Sometimes 10 to 15 games a day and I have always thought it cheated (had some sort of unfair advantage). I have read every blog and comment and have felt very frustrated by what I consider the internet bias towards JF As if we are all stupid and can’t tell a perception from an anomoly. I think your ideas fit exactly with I have noticed, especially a predilectionto double before a rescue roll

  4. Who ever wrote this article is from planet I live in a shell. Evidently they are not average as everyone else.(B***s***) I play all the time they do cheat and there are people who can manipulate the game. I played a little while ago and rolled 8 doubles in a row. The designer of the software evidently doesn’t play all the time . The game if your behind will throw you right back in line with the other player and then start helping the other player When you catch up. Bad creation . No room for strategy.
    I would love to find a site where it allowed you to win fairly and a site that could not be manipulated.

  5. Interesting. Did you make a conscious effort to prove my point, or did you not actually read the article and just want to spout off? Methinks it was the latter.

    A truly random number generator would have to generate “8 doubles in a row” occasionally, or else it would not be properly random. Because that is a pattern that even a simple mind can recognize, it sticks in your head. You also only provide one anecdote, and no evidence. I directly addressed each of these fallacies in this article: 3 strikes; you’re out.

    As a final note, the gist of the article is to present a possible scenario whereby a neural network could learn to “cheat” inadvertently. I make no claims (and have no direct knowledge) about the fairness of online backgammon servers.

  6. The program cheats. It is as simple as that. Some examples: to get your two men out from the start of a game the best starts are double 6, double 4, double 3 or 5 & 6. The PC threw these 5 times as often as I would get them. Over the course of 100 games the PC threw 6 & 1 (needed to help block your opponent from getting out at the start of a game) 6 times as often as I did. And don’t get me started on how many times the PC gets off the bar in one throw (even if it only has one spot to aim at), whereas it usually takes me 5 throws to escape. Another dead giveaway is the willingness of the PC to place two or three checkers alongside each other as singletons. It does so in the knowledge that I will be unlikely to throw the required numbers to get them even though I have several to aim at! On the other hand, whenever I leave a singleton exposed there is a 75% chance that the PC will throw what is needed to get me. And don’t get me started on the number of times it throws exactly what is needed to get two of my singletons, whereas this almost never happens for me. The program cheats!

  7. I am convinced that the GNU program is psychic. It always seems to know when to double just before it is about to roll the perfect dice sequence to win the game.

    I know that it is immensely skillful when it comes to understanding the science of probability. But its success with this respect seems to be uncanny.

    And it shows this tendency not just on the grandmaster level but even on the intermediate level of play.

  8. What I find interesting in all these comments; they never complain when dice rolls seem to favor them too much. E.g. double 6, double 6 then double 5! But when the computer gets these rolls, then the dice are rigged. Have you considered that maybe you’re not as good as you think you are?

  9. I think reading between the lines I see a rationalization for a higher percentage of “good” rolls for the program when the skill level is set higher. One thing I have noticed in a lot of games is that when you increase the difficulty level, the program gives itself better rolls, cards, etc. I have stopped counting the number of games where the program got 3 doubles in a row while bearing off. The record for the program is 8 doubles in a row. I had 3 once.

  10. When the CPU is losing it starts rolling excessive amounts of doubles, no sorry but it cheats big time haz nothing to do with strategic moves, or numbers.

  11. I have played 4000+ games on Backgammon Pro and have been on the cusp of breaking through to Master II level for sometime now (ie. joining the top 100 global players in the program’s database). I have tried very hard to give the program the benefit of the doubt in relation to fair play. However, all the points made by GK are things that I have independently noted too. This goes beyond bias and selective attention on the part of the player. I have amassed over 400 gammons compared to less than 200 by the computer. Yet, every time I am close to breaking into Master II level, the computer seems to go into overdrive as described by GK and I suffer another reversal. Weird.

Leave a Reply

Your email address will not be published. Required fields are marked *