Ambiguity: Experiences with an Image Annotation
System”
6 in the context of a use case for creating and
editing rich metadata descriptions for images. The
authors discussed the roles of ambiguity, disagreement, and subjectivity in knowledge formation and
their implications for the design of a system for
semantic annotation of images.
Lora Aroyo, Anca Dumitrache, Praveen Paritosh,
Alex Quinn, and Chris Welty served as cochairs of
this workshop. The papers of the symposium were
published jointly with the HCOMP2018 CrowdBias
workshop as part of the CEUR workshop proceedings series.
Work in the Age of
Intelligent Machines
The Work in the Age of Intelligent Machines work-
shop explored ways in which human work and occu-
pations will be changed as artificial intelligence
becomes increasingly prevalent in the workplace.
While much media and academic attention has
focused on forecasts of the displacement of workers,
less attention has focused on ways AI might change
the workplace, and in particular, ways AI might gen-
erate new jobs or mitigate the displacement of work-
ers. By doing so, AI may help address a large-scale soci-
etal problem, a shift in skills needed in the workplace.
This workshop, part of a series of workshops supported by the National Science Foundation, brought
together researchers from academia, government and
industry. The workshop’s goal was the generation of
key research questions on the topic that merit further
study. Research questions were generated in discussions around two topics: “AI-Human Team Dynamics,” facilitated by Kurt Luther (Virginia Tech), and
“New Jobs, Education and Training, Unemployment,” facilitated by Matthew Lease (University of
Texas, Austin).
The AI-Human Team Dynamics group posed the
following questions: How can human and AI agents
participate in competitive interactions in order to
obtain better outcomes? How can AI and human
agents effectively interact through negotiation? How
can humans represented by AI agents (with poten-
tially different value systems) effectively interact
with each other through negotiation? How can we
combine them? The first question inverts the com-
monly held view that humans and machines should
complement each other. Instead, organizing friendly
competition between humans and machines may
help us better understand human and machine capa-
bilities. Such competitions could be the precursor to
negotiation, given that it becomes easier to figure out
how to trade or align cognitive effort in the service of
a shared goal once the comparative advantages
between humans and machines are better under-
stood. Answering these questions collectively might
help to produce new techniques for product design,
The New Jobs, Education and Training, Unem-
ployment group addressed complementary issues.
While AI may displace workers, it may also help
retrain people to work in jobs that require skills in
short supply. That is, AI may substitute for humans
in some tasks, but conversely it can help build
human capabilities that work in concert with
machine intelligence. The discussion of this group
centered around questions such as the following:
How can we use AI to reduce skill barriers to jobs,
thereby growing job opportunities and the scalabili-
ty of labor? How can we combat a potential skill-
technology gap and so reduce labor market frictions?
How can AI be used to simplify highly skilled jobs to
make them accessible to a larger part potential work-
force?
These questions, in turn, led to a series of questions related to the labor force: What new jobs
and/or transformation of existing jobs will come
from the advent of intelligent technologies? What
job descriptions are emerging on job boards related
to AI? Do some industries hire more people as
automation increases? In discussing these scenarios,
the group considered a potential impact of
autonomous vehicles on restaurants: less expensive
and more ubiquitous transportation might encourage more nights out.
This discussion branched out into questions about
how AI might, in fact, enhance jobs, the leading
question being, How can we integrate AI alongside
human workers in such a way as to enhance (in some
balanced way) productivity, satisfaction, and career
growth?
The group addressed what is known about intelligent tutoring, about predicting student failure in
advance and providing interventions, and about peer
assessment and feedback, especially research
informed by MOOCs and crowd-based approaches to
skill building and work. A metaquestion was also
posed: To work successfully in the age of intelligent
machines, what do people typically need to know
about AI — and what don’t they need to know?
Overall, the workshop raised a variety of important
questions that necessitate further research, so that we
can be proactive in addressing human work and
occupational changes as AI becomes increasingly
ubiquitous in the workplace. With the series of NSF
workshops on the future of work, and the growing
interest in the topic, the community can expect these
ideas to be further discussed and explored not only in
these workshops but in other community events as
well. This workshop was supported by the National
Science Foundation under grant IIS-1745463.
