team had to complete at least one qualifier. Initially,
over 60 teams registered, with 10 teams qualifying
for the final competition.
All of the competing teams were able to complete
some aspect of the finals. No team, however, was able
to achieve a perfect score. Additonally, although the
organizers intended for all teams to use planning or
AI approaches, this did not prove to be the case. Of
the four teams that submitted entries for the finals,
one used neither a conventional planning nor an AI
approach. This team performed an in-depth analysis
of the scenario and its patterns, which allowed them
to design a system with minimal sensors and thus
boosted their score quite a bit. Future iterations of
ARIAC will discourage an approach like this by making the environment harder to predict. The other
teams all instituted planning approaches, where their
robotic systems would receive sensor data and then
plan based on the current state of the world.
The winners of ARIAC 2017 were asked to present
at a workshop held at the 2017 International Conference on Intelligent Robots and Systems, where
travel was provided for one member of each winning
team. The workshop also included talks from industry and academe that addressed industrial robot agility, as well as the future of ARIAC.
ARIAC 2018 will have a completely new scenario
and setting. The organizers are considering an order
fulfillment scenario, where orders are submitted and
filled by a team’s robotic system. The 2018 competition is also expected to provide cash prizes for the
first-, second-, and third-place teams. The next round
of ARIAC will likely be held in May of 2018.
Certain commercial/open source software, hardware,
and tools are identified in this article in order to
explain our research. Such identification does not
imply recommendation of or endorsement by the
authors or NIST, nor does it imply that the software
tools identified are necessarily the best available for
1. National Institute of Standards and Technology — Agile
Robotics for Industrial Automation Competition,
2. Competition video results: vimeo.com/224134238.
3. The Robot Operating System, Open Source Robotics
4. Gazebo, Open Source Robotics Foundation, www.open-
Downs, A.; Harrison, W.; and Schlenoff, C. 2016. Test Meth-
ods for Robot Agility in Manufacturing. Industrial Robot: An
International Journal 43( 5): 563–72.
National Institute of Standards and Technology. 2016.
Canonical Robot Command Language (CRCL). Gaithersburg, MD: National Institute of Standards and Technology.
Anthony Downs is a mechanical engineer who has worked
at NIST in the Intelligent Systems Division since 2005. He
has worked on designing test methods for various projects,
including Agility Performance for Robotic Systems, Collaborative Performance for Robotic Systems, and Emergency
Response Robots, and on other list projects such as ARL
UWB Testing, DARPA ASSIST, DARPA TransTac, DARPA
TransApps, and DARPA BOLT. He has a BS in mechanical
engineering from University of Maryland at College Park.
Craig Schlenoff is the associate program manager of the
Robotics Systems for Smart Manufacturing Program, the
project leader of the Agility Performance of Robotic Systems
project, and the group leader of the Cognition and Collaboration Systems Group in the Intelligent Systems Division
at the National Institute of Standards and Technology. His
research interests focus on knowledge representation and
ontologies, intention recognition, and performance evaluation of autonomous systems and industrial robotics. He
previously served as the program manager for the Process
Engineering Program at NIST and he is the director of
ontologies at VerticalNet. He received his bachelor’s degree
from the University of Maryland, his master’s from Rensselaer Polytechnic Institute, and his PhD from the University
of Burgundy (France).
William S. Harrison III is a mechanical research engineer
in the Department of Commerce’s National Institute of
Standards and Technology (NIST). Harrison’s specialty within the project is virtual fusion, which is the mix of simulated and real components for process validation and training.
His interests include virtual reality, game engines, augmented reality, and CG modeling. He received his bachelor’s degree from the University of Michigan, his master’s
from the University of Florida, and his PhD from the University of Michigan.