My Jeopardy! Strategy Advice To IBM - Part 8

Ed Toutant Report on Watson Jeopardy! Visit – September 17, 2010

MISCELLANEOUS OBSERVATIONS

It appears to me that Watson’s biggest disadvantage is his lack of any speech recognition or audio sensing.  There are many occasions in a typical Jeopardy! match when it is essential to know what is being said.  The most common example is during a rebound opportunity.  Since Watson doesn’t know what his opponent’s answer was, he might buzz in and give the same wrong answer.  Occasionally there are clues or categories that are multiple choice, usually with three different choices.  If the category is Executive, Legislative, or Judicial and they ask which branch a particular government official serves in, Watson will look stupid if he can’t pick up the free money after his opponents both answer incorrectly.  There are also many times when Alex Trebek pauses to think or to get a ruling on an answer that might be acceptable.  That’s a huge hint to the other players that the answer is something close to what the contestant said, but Watson won’t be able to capitalize on those opportunities with his current design.

I observed one match in which Watson had a big lead going into Final Jeopardy! and didn’t even attempt to answer the question.  I thought that was poor form and not very sporting, even though it didn’t change the outcome of the match.  The whole point of the Jeopardy! Challenge is to see how well Watson can play, so the public is going to want to see his best effort, even when his confidence is low.

In two of the six matches I saw, Watson led going into FJ, but did not have a lock game.  Alan Bailey was in second place both times.  In both cases, Watson’s FJ wager would have conceded a tie to Alan if he was right and went all in on his bet.  This is a very unorthodox strategy that seems ill-advised.  Alan learned from experience and bet it all in the last match when he saw that Watson is programmed to play for a tie in non-lock games.  The IBM strategists apparently think that a tie is as good as a win.  I don’t think the public will see it that way.  A tie will be seen as a moral victory for the human.  Someone on the IBM team told me that playing to allow a tie slightly improves Watson’s chances of not losing outright, but I have not seen a plausible hypothetical case where that would be true.  In fact, if the word leaks out (which seems likely) that Watson offers ties, contestants will make unconventional FJ wagers to take advantage of that weakness and Watson will win less often than he would with a more traditional wager.

I assume the IBM team must be familiar with the J! Archive.  It includes a FJ wagering calculator for the conventional strategy that most good contestants agree with and try to follow.

http://www.j-archive.com/wageringcalculator.php

The site also has a glossary of unofficial terms and strategies, summarizing the consensus thinking of a lot of past contestants and serious analysts of the game. 

http://www.j-archive.com/help.php

I believe the most important of these is the Two-thirds Rule, which explains how to bet FJ when you are within two-thirds of the leader’s total.  Almost every week, contestants lose games they should have won, simply because they did not know this betting tip.  It happened again today (10/07/10), when the second place contestant should have won on a triple stumper, but she lost by betting too much.  There are also some worthwhile post-game discussions every day on the official Jeopardy! message board.  Very little escapes the attention of those devoted and generally knowledgeable fans, and any unusual incidents or strategic errors are usually thoroughly analyzed. 

http://boards.sonypictures.com/boards/forumdisplay.php?f=11

It occurred to me that Watson could test the value of different strategy variations by partitioning himself into three separate Watsons, each running a different strategy and then playing many simulated Jeopardy matches against different versions of himself.  Reset his memory each time so he doesn’t remember seeing the clues before.  If one version significantly outperforms the others after hundreds of simulations, that would tend to prove which approach works best.

Many of the tips I have described here could be used not only by Watson, but also by human contestants.  But for a human, the game progresses so fast that it seems like a blur.  Any plan to use a wide array of incremental tactics would become impossible for a human to implement without distracting from the main game.  One of Watson’s strengths is that he can be programmed to take advantage of many small cumulative factors that may add up to a meaningful edge.  Unlike humans, Watson doesn’t get distracted.  Of course, any changes should be thoroughly tested prior to the Jeopardy! Challenge, to assure that no new bugs have been introduced.