locutor (Nirenburg and McShane 2016b; McShane,
Blissett, and Nirenburg 2017).
How long will it take for LEIA applications to come
to fruition? About the same amount of time as it takes
to learn to play the violin: anywhere from a year to a
lifetime, depending on time spent, resources available, and target quality and coverage.
This research was supported in part by Grant
#N00014-16-1-2118 from the US Office of Naval
Research. Any opinions or findings expressed in this
material are those of the author and do not necessarily reflect the views of the Office of Naval Re-search. Thanks to Micah Clark for his insightful comments on a draft of this article.
1. Church (2011, p. 18) writes: “[John] Pierce objects to
attempts to sell science as something other than it is (for
example, applications), as well as attempts to misrepresent
progress with misleading demos and/or mindless metrics
(such as the kinds of evaluations that are routinely performed today).” Evaluating NLU systems using the metrics
imposed by the NLP community is not only impossible, it
falls squarely into the category of mindless metrics.
2. Cited from en.wikipedia.org/wiki/Detroit on December
3. Some annotations have been removed for concise presentation.
4. It does, however, follow a typical NLP practice of isolating
a particular phenomenon, such as reference resolution or
word-sense disambiguation, annotating a corpus for a subset of its realizations, then staging a competition among
machine-learning systems trained using that corpus. The
utility of these task-specific competitions, whose results, to
my knowledge, are rarely incorporated into application systems, remains to be seen.
5. The reasoning strategies presented here are selective, not
comprehensive. For discussion of yet another reasoning
strategy — reasoning by analogy — see Forbus and Hinrichs
6. The OS ontology is language independent. The names of
concepts look like English words only for the benefit of the
humans who acquire the knowledge resources and test and
evaluate the system. For the system’s purposes, concept
names could as easily be randomly selected sequences of
7. There have been two main implementations of the theory of OS. Implementations prior to 2015 carried out sen-tence-level analysis, meaning that they considered whole
sentences at once, an approach typical for syntactic parsers.
By contrast, the implementation under development since
2015 pursues incremental (word-by-word) analysis, which
more closely emulates what people do. See McShane and
Nirenburg (2016) for an in-depth juxtaposition of these
8. We currently use select outputs from the Stanford
CoreNLP tool set (Manning et al. 2014).
9. A more complete lexicon would include many more
phrasal senses, such as eat one’s hat, eat one’s heart out, eat
someone alive, and so on.
10. Yes, a lion can eat a human … and a car can be hot pink,
and some dogs have no tails — none of which is covered by
our current ontology, which is intended to provide agents
with knowledge of how the world typically works.
11. [e] indicates an empty category — that is, ellipsis. The
italics indicate the antecedent.
12. Although this might seem like the obvious approach, it
is actually not typical in mainstream NLP, where entities of
interest in a corpus — so-called markables — tend to be
selected manually prior to system training and evaluation
(for the example of coreference resolution, see Hirschman
and Chinchor ).
13. See Tanenhaus et al. (1995) for a discussion of grounding linguistic references in the real world as early as possible.
14. Of course, full language understanding far from
exhausts such a robot’s responsibilities, as discussed by
Matthias Scheutz (Scheutz 2017) with respect to ethics.
15. This example arose in conversation with Selmer
Bringsjord about the division of labor between language
processing and general reasoning.
16. In some cases iplementations are partial. Most
microtheories include aspects ranging from simple to
extremely complex. Consider the example of nominal compounds: in the simplest case, they are recorded explicitly in
the lexicon; in the most complex case, the agent doesn’t
know the meaning of either of the words; and there are
many eventualities in between. In developing microtheories, we attempt to flesh out the full problem space even if
we cannot immediately achieve high-quality results for the
more difficult component phenomena.
Church, K. 2011. A Pendulum Swung Too Far. Linguistic
Issues in Language Technology 6( 5): 1–27.
Du Bois, J. W.; Chafe, W. L.; Meyer, C.; Thompson S. A.;
Englebretson, R.; and Martey, N. 2000–2005. Santa Barbara
Corpus of Spoken American English, Parts 1–4. Philadelphia,
PA: Linguistic Data Consortium.
Forbus, K. D., and Hinrichs, T. R. 2017. Analogy and Relational Representations in the Companion Cognitive Architecture. AI Magazine 38( 4). doi.org/10.1609/aimag.v38i4.
Graff, D., and Cieri, C. 2003. English Gigaword.
Philadelphia, PA: Linguistic Data Consortium.
Hirschman, L., and Chinchor, N. 1997. MUC- 7 Coreference
Task Definition. Retrieval Group, Informational Access
Division, National Institute of Standards and Technology,
Version 3. Gaithersburg, MD: US Commerce Department.
Jackendoff, R. 2007. A Whole Lot of Challenges for Linguists. Journal of English Linguistics 35( 3): 253–262.
Lindes, P., and Laird, J. E. 2016. Toward Integrating Cognitive Linguistics and Cognitive Language Processing. Paper
presented at the 14th International Conference on Cognitive Modeling (ICCM). University Park, Pennsylvania, 4–6