weapons systems. These applications require AI systems that are highly robust, and yet our current systems fall far short. To create the level of robustness
required for such high-risk applications, we need systems that are robust both to known unknowns (the
uncertainty they represent explicitly) and to
unknown unknowns (unmodeled aspects of the
In this article, I’ve made an (incomplete) catalogue
of the ideas and methods that the AI community has
developed for achieving robustness. To manage the
known unknowns, we can build on our existing
methods for representing uncertainty using probability distributions or uncertainty intervals. We can then
define robust optimization problems in which we
search for the optimal solution when competing
against an adversary that is given a fixed budget. An
important idea for achieving robustness in machine
learning is regularization. We saw that it is possible to
reformulate regularization in terms of finding a
robust optimum against an adversary who can perturb the data points. A second important idea for
achieving robustness is to optimize a risk-sensitive
objective, such as the conditional value at risk
(CVaR). Again we saw that it is possible to formulate
CVaR optimization as finding a robust optimum
against an adversary who can perturb the transition
probabilities of our dynamical model of the world.
This tells us that acting conservatively can confer
robustness. Finally, we explored how to make inference itself robust and discussed work on robust probabilistic and diagnostic reasoning.
I then turned to cataloguing our ideas about the
unknown unknowns. The first idea is to develop
methods for detecting when our model is inadequate
before our AI system makes a mistake. I discussed
work on anomaly detection and change-point detection that can protect a system against changes in the
data distribution. A second idea is to learn causal
models, because they have been proven to be more
transportable and therefore more robust to changes
in the context of decision making. I spent a long time
discussing the third idea, which is to employ ensembles or portfolios of methods. I looked at algorithm
portfolios for SAT solving as well as knowledge-level
portfolios in which the AI system models multiple
facets (such as structure, function, and appearance) of
objects in the world. Finally, I discussed the idea of
continually expanding the knowledge that our systems possess. While it is impossible to know everything, we can hope that “on the average, and in the
long run, more knowledge is better than less” (
Herbert Simon, Harry Camp Lectures at Stanford University, 1982).
I wish to thank the many people who provided sug-
gestions and pointers to relevant research: David Ack-
ley, Stefano Albrecht, Juan Augusto, Randall Davis,
Pedro Domingos, Alan Fern, Boi Faltings, Stephanie
Forrest, Helen Gigley, Barbara Grosz, Vasant Honavar,
Holgar Hoos, Eric Horvitz, Michael Huhns, Rebecca
Hutchinson, Mykel Kochenderfer, Pat Langley, Srid-
har Mahadevan, Shie Mannor, Melanie Mitchell,
Dana Nau, Takayuki Osogami, Don Perlis, Jeff Rosen-
schein, Dan Roth, Stuart Russell, Tuomas Sandholm,
Rob Schapire, Scott Sanner, Prasad Tadepalli, Milind
Tambe, Brian Williams, Zhi-hua Zhou. I apologize
that I was not able to weave in all of the great work
that folks described. I also thank the AI Magazine edi-
tor Ashok Goel for his suggestions and editorial
This work was partially supported by the Future of
Life Institute ( futureoflife.org) FLI-RFP-AI1 program,
grant number 2015-145014, and by NSF grants
0705765 and 0832804.
Adams, R. P., and MacKay, D. J. 2007. Bayesian Online
Changepoint Detection. arXiv Preprint. arXiv:0710.3742
[ stat.ML]. Ithaca, NY: Cornell University Library.
Antonucci, A., and Zaffalon, M. 2007. Fast Algorithms for
Robust Classification with Bayesian Nets. International Journal of Approximate Reasoning 44( 3): 200–223. doi.org/10.
Arkin, R. C. 2009. Governing Lethal Behavior in Autonomous
Robots. London: Chapman and Hall/CRC. doi.org/10.1201/
Bareinboim, E., and Pearl, J. 2012. Transportability of Causal
Effects: Completeness Results. In Proceedings of the 26th
National Conference on Artificial Intelligence (AAAI-2012),
698–704. Menlo Park, CA.: AAAI Press.
Barry, D., and Hartigan, J. A. 1993. A Bayesian Analysis for
Change Point Problems. Journal of the American Statistical
Association 88(421): 309–319. doi.org/10.1080/01621459.
Ben-David, S.; Blitzer, J.; Crammer, K.; Kulesza, A.; Pereira, F.;
and Vaughan, J. W. 2010. A Theory of Learning from Different Domains. Machine Learning 79( 1): 151–175.
Ben-David, S.; Blitzer, J.; Crammer, K.; and Pereira, F. 2007.
Analysis of Representations for Domain Adaptation. In
Advances in Neural Information Processing Systems 19, 137.
Cambridge, MA: The MIT Press. doi.org/10.1007/s10994-
Bendale, A., and Boult, T. 2015. Towards Open World Recognition. In 2015 IEEE Conference on Computer Vision and Pattern Recognition, 1893–1902. Piscataway, N.J.: Institute for
Electrical and Electronics Engineers. doi.ieeecomputersoci-
Bertsimas, D., and Thiele, A. 2006. Robust and Data-Driven
Optimization: Modern Decision Making Under Uncertainty. In Tutorials in Operations Research: Models, Methods, and
Applications for Innovative Decision Making, 95–122.
Catonsville, MD: The Institute for Operations Research and
the Management Sciences (Informs).
Betancourt, M. 2017. A Conceptual Introduction to Hamiltonian Monte Carlo. arXiv Preprint. arXiv:1701.02434
[stat.ME]. Ithaca, NY: Cornell University Library.
Blitzer, J.; McDonald, R.; and Pereira, F. 2006. Domain Adap-
tation with Structural Correspondence Learning. In Proceed-
ings of the 2006 Conference on Empirical Methods in Natural