The 25th International Joint Conference on Artificial Intelligence (IJCAI- 16) marked the first running of the Winograd Schema Challenge, sponsored by
Nuance Communications. The Winograd Schema Challenge was originally conceived by Hector Levesque
(Levesque 2011; Levesque, Davis, and Morgenstern,
2012) as an alternative to the Turing test that has clear
criteria for success and doesn’t rely on deception. Six systems were entered, exploiting a variety of technologies.
None of the systems were able to advance from the first
round to the second and final round.
The Winograd Schema Challenge is concerned with
finding the referents of pronouns, or solving the pro-
noun disambiguation problem. Doing this correctly
appears to rely on having a solid base of commonsense
knowledge and the ability to reason intelligently with
that knowledge. This can be seen from considering an
example of a Winograd schema. Like all Winograd
schemas, it consists of two halves:
( 1) John took the water bottle out of the backpack so that
it would be lighter.
( 2) John took the water bottle out of the backpack so that
it would be handy.
The referent of it in sentence 1 is the backpack; the refer-
ent of it in sentence 2 is the water bottle. A human can
FALL 2017 97 Copyright © 2017, Association for the Advancement of Artificial Intelligence. All rights reserved. ISSN 0738-4602
The First Winograd Schema
Challenge at IJCAI- 16
Ernest Davis, Leora Morgenstern, Charles L. Ortiz, Jr.
; The first Winograd Schema Challenge was held in New York, New York,
as part of the International Joint Conference on Artificial Intelligence. The
challenge was original conceived by Hector Levesque as an alternative to the Turing test. This report details the results of
this first challenge.