requires careful consideration of incentive structures,
learning, and communication.
The symposium brought together researchers
from different fields to discuss the current state of
multiagent learning, as well as future directions and
roadblocks. The main topics of discussion concentrated on questions such as the following. What are
the right models for multiagent learning in different
situations (for example, Dec-POMDPs, I-POMDPs,
other decision-theoretic or game-theoretic models)?
What are the best benchmarks to use and how can
we create new, high-quality ones? What is the role of
deep learning in multiagent learning? What are the
best metrics for evaluating multiagent learning performance? How do we ensure that multiagent learning is applicable to real-world problems?
The invited speakers spoke on many of these
issues. Ann Nowé (Vrije Universiteit Brussel) discussed learning properties with simple learning rules
such as learning automata. Igor Mordatch (OpenAI)
talked about recent methods for deep reinforcement
learning for communication in cooperative multiagent systems. Mac Schwager (Stanford University)
talked about multiagent learning for multirobot
coordination. Mykel Kochenderfer (Stanford University) spoke about multiagent learning for applications ranging from aircraft collision avoidance to
autonomous vehicles and drones. Pradeep Varakan-tham (Singapore Management University) described
combinations of game theory and optimization in
order to balance resource demand, and Emma Brunskill (Stanford University), in a joint session with the
Symposium on Integrated Representation, Reasoning, and Learning in Robotics, talked about progress
in model-based reinforcement learning by using
ensembles of neural networks as models.
Ten contributed talks were also given on many of
the topics previously mentioned. These talks
described work on new game-theoretic and decision-
theoretic methods in scenarios that are partially
observable, human-interactive, ad-hoc, imperfect
information, multi-task or nonstationary. The scope
of work represents the broad set of approaches and
situations in which multiagent learning applies.
Overall, there was much enthusiasm about the
future of multiagent learning. A number of topics
relevant for the progress of the field, such as scala-
bility, evaluation, and properties that are relevant for
real-world applications, were deliberated in a discus-
sion session and analyzed in further detail in break-
out sessions. One point that resonated particularly
well with the audience was the idea of constructing
a suite of benchmark problems for multiagent learn-
ing. It was agreed that future symposia and work-
shops will be held to continue discussion and the
progress that has been made. Concrete goals include
an industry and academe partnership to develop a
website with papers and benchmarks, as well as
widening the participation in the discussion by
including additional senior and junior researchers.
This symposium was organized by Chris Amato
(Northeastern University), Thore Graepel (
DeepMind), Joel Leibo (DeepMind), Frans Oliehoek (Delft
University of Technology and University of Liverpool), and Karl Tuyls (DeepMind). Christopher Amato, Frans Oliehoek, and Karl Tuyls prepared this
report. The papers of the symposium were published
in the AAAI digital library.
Christopher Amato is an assistant professor at Northeastern University.
Haitham Bou Ammar is the head of reinforcement learning at PROWLER.io, Cambridge, UK.
Elizabeth Churchill is the director of UX at Google.
Erez Karpas is a senior lecturer at Technion, the Israel Institute of Technology.
Takashi Kido is a researcher in Japan. He had been a visiting researcher at Stanford University.
Mike Kuniavsky is a user experience designer at Parc.
W. F. Lawless is a professor at Paine College.
Frans A. Oliehoek is an associate professor at TU Delft and
the University of Liverpool.
Francesca Rossi is a distinguished research scientist at the
IBM T. J. Watson Research Center, and professor of computer science at the University of Padova, Italy.
Stephen Russell is a researcher at the US Army Research
Siddharth Srivastava is an assistant professor at Arizona
Keiki Takadama is a professor of the University of Electro-Communications in Japan.
Karl Tuyls is a research scientist at Google DeepMind.
Philip Van Allen is a professor in the Media Design Practices department at the Art Center College of Design.
K. Brent Venable is a professor of computer science at
Tulane University and a research scientist at the Florida
Institute for Human and Machine Cognition (IHMC).
Peter Vrancx is a senior machine learning researcher at
PROWLER.io, Cambridge, UK.
Shiqi Zhang is an assistant professor at Cleveland State