Table 3. Checklist for Builders of AI Applications.
Select problems with a solid business case.
Minimize changes required in existing work flows.
Identify domain- or task-specific knowledge and data for the problem.
Select appropriate knowledge representations and data sources.
Develop knowledge and data acquisition and maintenance plans.
Select appropriate reasoning/learning strategies.
Develop a set of test cases and performance metrics.
Add safeguards and opt-out capabilities.
Test with real data from users or operating environment.
community, the need to bring in outside experts.
Over the years, AI application developers have
made a major mind shift. We have learned the hard
way that success starts with solving problems important to the customer.
One caveat: Although public interest in AI is on the
rise, do not add an AI component to an application
just for the sake of it. AI can introduce complexity,
and systems should always only be as complicated as
is necessary to model the domain and task. Again,
focus on customer over technology.
Minimize Changes Required
in Existing Work Flows
Think about the integration of AI with other tools
and parts of the larger system. It is rare to completely replace an existing work flow. Thus, it is prudent to
build new systems so that they can slot into the
approaches already used by the customers as much
as possible. Few new AI systems solve stand-alone
problems that require no user interaction. Most are
used as “intelligent assistants” and the amount of
change management required to succeed in adoption
is directly proportional to the magnitude of the
changes required in existing work flows. Ease of use
is the “license to operate.”
Identify Domain- or Task-Specific
Knowledge and Data for the Problem
The history of successful AI applications shows that
“the power is in the knowledge,” both expert-pro-vided knowledge and knowledge extracted out of
data with the appropriate preprocessing, feature
selection, learning techniques, and parameter tuning. Devote up front the effort needed to acquire
enough of the knowledge and data so that you will
better understand how to design the rest of the system to best match the domain and task.
Select Appropriate Knowledge
Representations and Data Sources
Depending on the nature of the domain- and task-
specific knowledge, choose a knowledge representa-
tion that most closely models the world while still
supporting efficient reasoning strategies. Prefer
declarative knowledge since it is easier to understand,
explain and change than procedural knowledge. For
machine-learning approaches, select high-quality
data sources (for example, data with expert-verified
ground truth) when feasible or develop a strategy for
learning and reasoning with noisy data sources.
Develop Knowledge and Data
Acquisition and Maintenance Plans
Consider knowledge/data acquisition and maintenance to be an ongoing process. Make the process
iterative: repeatedly evaluate if the knowledge and
data are appropriate for the reasoning/learning strategies and domain/task and refine accordingly.
Select Appropriate Reasoning
and Learning Strategies
Most large AI systems will require various kinds of
reasoning and learning strategies for various subproblems. Design a system architecture that supports
decoupling of these disparate components so that
refinements in one component will not require drastic changes in other components.
Depending on the constraints dictated by the
domain and task, select an approach and components that are data driven or model driven, or use a
Develop a Set of Test Cases
and Performance Metrics
Due to the complexity of most AI systems, testing
and performance evaluation are critical.
The word performance encompasses a variety of
concerns, including run-time speed and use of
resources plus adequacy of the knowledge and reasoning components. Run-time speed and use of
resources are standard computational concerns that
must be addressed in any system that is to be
deployed and scaled. They are not specific to AI appli-