38 AI MAGAZINE
systems like self-driving cars and manufacturing
and surgical robots. Such systems typically gen-
erate significant amounts of data, but it is not clear
how industry designers can account for context and
leverage analytic tools like ML to gain insight from
this data into the actual use of intended designs or the
influence from external, potentially problematic cues.
ML has been used extensively for modeling
user-choice preferences, but little attention has been
paid to how to use such techniques to gain new
design insights into user behaviors, particularly in
terms of understanding contextual cues, or to connect user behaviors to system performance. We propose that, given the large amounts of data that such
autonomous systems typically generate, ML could
be useful if the algorithms were accurate and stable,
learned relationships relatively fast, and were interpretable in a design context and explainable.
A case study was presented that highlighted the
difficulties in selecting the best algorithm for the
contextual cue analysis. Determining that CART
was the best algorithm for this analysis, we demonstrated that applying ML techniques to the design
data analysis can lead to interesting and potentially
useful results that are very different from traditional
hypothesis-driven statistical experimental designs.
We are not suggesting that ML approaches should
replace such scientific methods, but rather that they
should be used to augment analyses.
Future related work should include determining
what makes some ML algorithms better suited for
design problems and what core characteristics define
such utility. Moreover, given long-standing problems
with people understanding probabilistic reasoning
algorithms (Tversky and Kahneman 1974), are there
representations that make some ML algorithms more
interpretable and explainable for industry users?
Explainability of ML techniques is likely a multidimen-
sional construct, and a future area of inquiry should
be describing how and why various ML approaches
may be more or less useful in the design context.
This research was supported by National Science
Foundation Grant 1548417 to Duke University. The
technical monitor was Jordan Berg. The views and
opinions expressed are those of the authors and do
not necessarily reflect the views of the National
Amershi, S., and Conati, C. 2007. Unsupervised and Super-
vised Machine Learning in User Modeling for Intelligent
Learning Environments. In Proceedings of the 12th Interna-
tional Conference on Intelligent User Interfaces (IUI’07), Hono-
lulu, HI. New York: Association for Computing Machinery.
Andre, D., and Wickens, C. D. 1995. When Users Want
What’s Not Best for Them. Ergonomics in Design doi.org/
Boussemart, Y.; Cummings, M. L.; Las Fargeas, J.; and
Roy, N. 2011. Supervised Versus Unsupervised Learning
for Operator State Modeling in Unmanned Vehicle Settings.
Journal of Aerospace Computing, Information, and Communication 8( 3): 71–85. doi.org/10.2514/1.46767.
Bredereke, J., and Lankenau, A. 2002. A Rigorous View of
Mode Confusion. In Computer Safety, Reliability and Security, SAFECOMP 2002. Lecture Notes in Computer Science,