My Jeopardy! Strategy Advice To IBM - Part 2

Ed Toutant Report on Watson Jeopardy! Visit – September 17, 2010

WATSON’S SELF-KNOWLEDGE

All good Jeopardy! contestants must have a broad range of knowledge, but one of the most important requirements is to know what you know, and to know what you don’t know.  Without awareness and quantification of one’s own strengths and weaknesses, no strategy will be very effective.  With a general understanding of the probable outcome of different choices, one can begin to play the game in a way that will maximize the expected result.  Back when I was trying to become a contestant, I transcribed two years of Jeopardy! episodes from my VCR and kept my personal accuracy statistics for each topic, each clue value, each Daily Double, each Final Jeopardy!, and the location of each Daily Double.  I also kept statistics for all contestants on the show, to see how I compared.  By the time I got the call, I had a good awareness of my probable chances on each clue.  I tried to bet accordingly and choose the most favorable clues, depending on the status of the game at the time.  I assume that Watson must have done this already, to an even greater extent than I was able to do.  There are thousands of Jeopardy! game transcripts available in the unofficial archives that could be used for that purpose.

What I found from tracking my own statistics was that the questions certainly got more difficult toward the bottom of the board, though not nearly in proportion to the dollar values of the clues.  I found that the clues in the fourth and fifth rows were almost equally difficult.  (This was 20+ years ago, so I don’t know if that is still true.)  It is therefore possible to generate an expected value for each clue on the board, which can be (over)simplified as: clue value x % accuracy x % chance of being first on the buzzer.  Each expected value could be further adjusted based on the clue’s subject and the probability of it being a Daily Double.  When this is done, it becomes obvious that the game is won or lost in the bottom half of the board, since that’s where most of the points are.

The clues are written with the intention of being increasingly difficult as you move to the bottom of the board.  I think that is true for all human contestants, since the clues are written from a human perspective and the clue values are assigned based on what humans would know.  I can’t say for sure, but I suspect that it is not as true for Watson, since he uses different thought processes to generate his responses.  In many cases, Watson might find a $2000 clue just as easy as a $400 clue.  For example, if Watson’s database includes the lyrics to all songs by Prince, identifying an obscure song is no harder than identifying a big hit.  If my theory is correct, and Watson does not notice the intended difficulty level of the question as much as humans do, that would make the high value clues even more important to Watson than they are to human contestants.