and computes performance metrics that are combined into a final ranking. The diagnostic framework
specifies a text-based communication protocol consisting of time-stamped events. Its implementation is
on top of the transmission control protocol and supports Linux and Windows.
Participants in the Diagnostic Competition submit
diagnostic algorithms that implement the framework
API. A diagnostic algorithm must implement a call-back function that is called when new sensor data is
available. We have provided dummy diagnostic algorithms written in C and in Java. For other languages,
participants have to write their own code that communicates with the remaining framework components. Participants also submit diagnostic algorithms
written in Matlab and in LISP.
The main goal of the software components in the
diagnostic framework is to perform diagnostic experiments, or to play scenarios to the diagnostic algorithms. The scenarios are stored in text files. The scenario data source framework component reads the
scenario files and sends commands and sensor data
to the diagnostic algorithms. Both the scenario data
source and the diagnostic algorithm send data to the
scenario recorder. The latter is in charge of creating
output scenarios that contain all events in the original scenarios but also the diagnoses computed by the
diagnostic algorithms. These output scenarios are
evaluated by a scenario evaluator to compute several
performance metrics. All framework components are
managed (started, stopped, monitored, and so on) by
the scenario loader. The framework is designed with
security in mind. The fault injection events, for example, are not submitted to the diagnostic algorithms, to prevent cheating.
We envision the diagnostic framework as a step toward the adoption of a wider diagnostic standard
and, we hope, one day, the design of a universal industrial protocol for exchange of diagnostic information.
Diagnostics has deep roots in the aerospace industry.
Users from this industry require fast and accurate rea-
soning in order to achieve autonomy or to provide
decision support to human flight operators. The sys-
tem in this track is the electrical power system (EPS)
test bed in the ADAPT lab at the NASA Ames Research
Center. The system is a hardware test bed of an elec-
trical power and distribution system of a satellite and
illustrates many of the design properties that present
in real-world satellites. These properties are control-
lability and observability as well as redundancy. A
subcircuit of ADAPT is called ADAPT-Lite and is used
as a lightweight competition track. Diagnostic algo-
rithms in the ADAPT and ADAPT-Lite tracks are
benchmarked with a large number of diagnostic sce-
narios, including single and multiple faults. Some of
these scenarios are created by hardware sensor data,
others programmatically by using a software simulator.
The synthetic track consists mainly of the ten IS-CAS- 85 logic circuits that were initially used for automated test pattern generation (ATPG). These are
net lists of (old) real-world integrated circuits (ICs)
such as arithmetic-logic units (ALUs), error correction codes (ECCs), adders, and multipliers. We have
also added four smaller circuites from the 74XXX IC
family as simpler cases for the DAs. The modeling of
the systems in this track is trivial, and there are several easy algorithms that can be used for diagnosis.
The challenge in this track, however, is not modeling, but globally optimizing the computational performance vs. diagnostic accuracy. The largest of these
circuits (c7552), for example, has 3512 logic gates,
which presents problems for many DAs. Further,
there is no restriction on the number of faults that
we inject. They may mask or may not mask, making
the optimization of computational performance versus diagnostic accuracy nontrivial. The DAs must report results within 30 seconds of CPU time.
FACT (Roychoudhury, Biswas, and Koutsoukos 2009)
is a model-based diagnosis system that uses hybrid
bond graphs, and models derived from them, at all
levels of diagnosis, including fault detection, isola-
tion, and identification. Faults are detected using an
observer-based approach with statistical techniques
for robust detection. Faults are isolated by matching
qualitative deviations caused by fault transients to
those predicted by the model. For systems with few
operating configurations, fault isolation is imple-
mented in a compiled form to improve performance.
Fault Buster, uses a combination of multivariate
statistical methods for the generation of residuals.
Once the detection has been done, a neural net-workperforms classification for computing isolation.
GoalArt Diagnostic System (Larsson 1996) is based
on multilevel flow models, which are crisp descriptions of flows of mass, energy, and information. It
performs fast root-cause analysis with linear computational complexity. Its main advantage is that it is
very efficient to engineer a model. The algorithm has
been proven in several commercial applications.
The hybrid diagnosis engine (HyDE) (Narasimhan
and Brownston 2007) is a model-based diagnosis engine that uses consistency between model predictions and observations to generate conflicts that in
turn drive the search for new fault candidates. HyDE
uses discrete models of the system and a discretiza-tion of the sensor observations for diagnosis. HyDES uses the HyDE system but runs it on interval valued
hybrid models and the raw sensor data.
LYDIA is a declarative modeling language specifically developed for model-based diagnosis (MBD).