Ed Toutant Report on Watson Jeopardy! Visit – September 17, 2010
WATSON’S KNOWLEDGE OF HIS OPPONENTS
If IBM knows who Watson’s potential opponents will be, it may be possible to study each player’s performance history and other biographical details, to assess their strengths and weaknesses and develop a customized strategy for each opponent. A lot of information can be found online about the legendary Jeopardy! players. I assume IBM is very familiar with the amazing online database at the J! Archive, which can be found at http://www.j-archive.com/. Every game that Ken Jennings and most other top contestants have ever played is meticulously transcribed there. Watson can prepare for his upcoming matches by analyzing each contestant’s knowledge base and style of play. With that information, Watson can play the game slightly differently against each player. If Watson determines that Ken Jennings is especially good at wordplay categories and especially weak in sports, he could hunt for Daily Doubles in Ken’s strongest categories to prevent Ken from cashing in on those, then switch to sports questions after taking the DD out of the game. Similarly, if Watson knows that Ken grew up in South Korea, and that Brad Rutter grew up in Pennsylvania (and both of those details are true, by the way), then Watson could try to prevent them from scoring big in categories about “The Korean War” or “The Keystone State.” Watson has the potential to know much more about his opponents than they know about him, so that could be used to gain an advantage.
Ken Jennings shares many details about his personal life online. You can get a pretty accurate picture of the guy by reading his daily blog posts from the past four years. It would be easy to compile a list of the places he has been, books he has read, his hobbies, interests and other insights into what he knows and cares about. He even blogged recently about how TV quiz shows never write questions based on what a specific contestant would or would not know. He admitted that his personal Achilles heel would be questions about Fashion.
One subject that I’m sure Ken excels in is Movies. If Ken were to play against Watson in a special all-movies theme episode, I’d bet the house on Ken. Here is an old list Ken posted online, showing all the films he had seen in his life, as of several years ago. Unfortunately, he hasn’t kept it updated, but it shows how dedicated he is to learning all there is to know about films.
There are several other web sites that have statistical data about some players’ strengths and weaknesses, since most of them participate for fun in a variety of different quizzing formats. For example, there is a spreadsheet with category results for each participant in the annual World Quizzing Championships, which is an intense written quiz that started in Europe and has attracted many top American game show contestants. Check it out here on this Belgian site: http://www.iqa.be/. If you look at the full results for 2010, you’ll see that Jerome Vered finished in 27th place, out of 1175 players worldwide. Brad Rutter finished in 140th place, and I finished in 153rd place. A quick look suggests that Brad is somewhat weak in Sports and Science, and relatively strong in Culture and Lifestyle. Jerome’s pattern is similar, with higher scores overall, and a notable strength in History. Ken Jennings did not register for the World Quiz, but he took it unofficially at home and reported his results on his blog: http://ken-jennings.com/messageboards/viewtopic.php?p=57985#57985
Ken’s score of 118 would have put him around 60th place. He showed weakness in the Sports and World categories, and relative strength in Culture and Entertainment.
Ken, Jerome, and Brad have taken the World quiz and similar written tests of overall knowledge several times, and their results are consistent – Jerome knows more than Ken, and Ken knows more than Brad. And yet, when they competed against each other in the finals of the Jeopardy! Ultimate TOC, the results were just the opposite. Brad won convincingly all three days, and Ken played consistently better than Jerome. A lot of people assume that Brad was just quicker on the buzzer, and there may be some truth in that, but I believe the main reason why Brad won was that Ken’s and Jerome’s knowledge bases overlapped each other a lot, whereas Brad knew more things that neither Ken nor Jerome knew. When there is a three-way competition, the two people who know the most might lose to the person whose knowledge is more unusual. The same principle holds in the Watson matches. When I spoke to the IBM strategy team, they acknowledged that Watson has a much better chance to win against two human opponents than it would against one, because Watson will get most of the questions that are computer friendly, and the two humans will have to split the questions that are human friendly. Since Watson fares best when his two opponents are most similar to each other, he may be at a slight advantage if his two opponents are of the same sex, or the same age range. That would tend to make their knowledge bases overlap the most, allowing Watson to answer a greater share of the questions.
No comments:
Post a Comment