similar approach is pursued by Zhou, Dai, and Chen
(2015), where quadratic programming is exploited to
decide the best match between the text elements and
the equation template components. Templates, in the
form of equation tree structures, are exploited also by
Koncel-Kedziorski and colleagues (2015), where integer linear programming is used for generating the
space of trees and machine-learning techniques are
used to select the best matching tree.
Following a semantic approach, Morton and Qu
(2013) use a framework based on fuzzy logic and
ontologies to solve mathematical word problems
with the primary purpose of teaching users. To compute results, the search engine WolframAlpha and its
integration within Mathematica are used.
In the paper by Shi et al. (2015) a new representation language (DOL) has been designed to bridge natural language text and math expressions. The parsing
of natural language into DOL is performed using a
context-free grammar. Then, a reasoning step recognizes the text portions of interest in the resolution,
such as mathematical sentences, and these are subsequently turned into numerical expressions.
Inference (FOL) and statistical approaches for NLP,
are exploited by Liang et al. (2016) to transform the
text description in a tag-based form, and then into a
first-order logic program. Inference is then performed
to determine the solution.
Solving mathematical puzzles requires also comprehension of the role played by quantities in natural language. In the paper by Roy, Vieira, and Roth
(2015) a series of features are described to support reasoning on quantities expressed in natural language.
Two different numerical reasoning tasks are investigated and addressed: quantity entailment, and the
problem of automatically understanding and solving
elementary school math word problems.
The increasing interest in solving mathematical
word problems is also witnessed by the creation of
several data sets. Recently, Koncel-Kedziorski and colleagues (2016) proposed a framework for unifying all
these sets into a single repository, with the possibility of extending it with new problem types and
The Todai Robot Project24 aims at creating an artificially intelligent agent obtaining a high score in the
Japan National Center Test for University Admissions
(McGoogan 2015, Strickland 2013), and passing the
entrance exam of the University of Tokyo in 2021.
Natural language comprehension is particularly
stressed in order to support query answering and algebraic problems.
Some mathematical puzzles can be viewed as math
word problems, while others can be assimilated to
logic puzzles. An obvious way to solve logic puzzles is
the use of theorem provers for first-order logic. The
translation of a problem (expressed in natural language) to a logic semantics (exploitable in automated
reasoning) is a hard and challenging problem for AI.
In the paper by Lev et al. (2004) a method is proposed
that uses an intermediate language called Semantic
Logic, a general-purpose language with events,
groups of variables, modal operators such as “
necessary” and “possibly,” and generalized quantifiers. The
representation in the semantic logic can be translated in first-order logic and then solved by using a suitable theorem prover.
In a paper by Mitra and Baral (2015) the system
LOGICIA is introduced. The authors claim it is the first
system able to solve logical grid puzzles in a fully automated manner. Puzzles are addressed by translating
them to answer set programming (ASP; Gelfond and
Lifschitz ), and then solved by an ASP solver.
A different, more general approach is presented by
Forbus, Klenk, and Hinrichs (2009), where a central
role is given to the analogy reasoning capability typical of human beings. Analogy is exploited in the
Companions cognitive architecture for matching,
memory retrieval, and generalization tasks. The
approach has been applied in different domains, and
in particular in the test-taking setting for physics
problems. The interesting aspect is that the Companions architecture does not have, initially, any specific
knowledge about physics, but exploits analogy to
retrieve relevant solutions by looking into previously
accumulated examples and extrapolating/adapting
existing solutions to solve the new problem.
In table 1 we report some related works, classified
with regard to important dimensions of the proposed
challenge: the domain they address; whether they
take as input only natural language descriptions or
diagrams as well; whether they support problem
description using “everyday language” or rather
restrict problem description to a specific, limited language; and the solver used to solve the problem model (once a model has been extracted). We do not
report the AI technique used by each system to identify or extract a model from the natural and diagram
problem description: roughly speaking, the majority
of the works exploit rule-based, with or without logic inference, and statistical approaches.
Our challenge differs from the mentioned works in
many aspects. In order to address our challenge, in
contrast with some works focused on SAT, no specific
deep knowledge on some disciplines is required (for
example, math, physics, geometry). Instead, com-monssense reasoning capabilities are needed. Indeed,
mathematical puzzles often require the ability to reason about space, time (qualitative and quantitative),
causality, and events.
Moreover, a number of different puzzle categories
should be addressed within the challenge: some categories can be assimilated to math word problems,
some are more similar to logical puzzles, and some
ask for a different variety of skills and reasoning abilities. Therefore, the challenge also requires taking
into account the problem of establishing the appropriate modeling and problem-solving technique. In