the curves. The ROC describes the system’s behavior
for a range of confidence threshold settings and
enables the cardiologist (the end user) to decide on
weighting the false positives (FPs) and false negatives
(FNs) unequally (a very desirable property according
to our expert).
The overall accuracy results, averaged over 35
folds, are shown in table 2. As can be seen from all of
these results, our approach has excellent accuracy for
all three plaque types. In fact, across the 35 folds, the
median accuracy for all three plaque types is 100 percent, indicating that our classifiers are able to perfectly separate the plaque types using the features we
designed. In a few folds, the accuracy is lower than
100 percent. We conjecture that this is because some
pullbacks have many more images associated with
them than others. When such a pullback is held out,
the training set size decreases in size and yields a classifier with lower accuracy.
In the second experiment, we ran our trained classifier on the cryo-images. We also ran a baseline
approach following Ughi et al. (2013). This approach
uses beam-attenuation estimates from a layer model
applied to single A-lines and 2D texture and geometric
measures as features for classification with the added
requirement of manual region of interest selection for
analysis. These results are shown in table 3. Here the
“Other” row corresponds to pixels in these images that
belong to none of the three plaque types. The accuracy of the approach in this case is lower, possibly
because these are ex vivo images, which have somewhat different characteristics from the training set.
However, our approach still outperforms the state of
the art. Further, these values are still at a very useful level according to our expert. In particular, cardiologists
now divide an image into quadrants and simply state
whether a quadrant contains a certain plaque type. If
we use this as a performance measure, our current
approach has perfect accuracy on the cryo-images.
The results also indicate that in some cases some
plaque types may be confused with others. For example, the average intensity of a lipid region may be
very close to that of calcium. However, they may still
be separable due to the fact that the lipid’s attenuation coefficient is much higher.
To confirm our intuitive understanding of the
plaques’ characteristics we performed a leave-one-fea-ture-out experiment. In this experiment, we ran the
classifier using all of the features and noted the accuracy measures (as shown in table 2). We then
removed each feature at a time to see the impact on
the accuracy. We found that removing the attenuation parameter had the biggest impact on the lipid
accuracy, reducing it down to 92. 4 ± 8. 87 percent,
while removing the average intensity feature had a
significant effect on the fibrous’ accuracy and uncertainty (down to 95. 2 percent ± 10. 75).
In addition to high accuracy, our approach is also
efficient at classification. Each test fold (on average
Figure 5. Results of the Back-Border Segmentation.
An illustration of back-border segmentation (yellow line) along with lumen
segmentation (red line) in a typical clinical image in both views. (a) is the
polar image and (b) is the x-y image. The yellow line is broken due to view
conversion. Asterisk marks the guide-wire shadowing artifact.
Table 1. Performance Measures.
Area under ROC and the accuracy, sensitivity, and specificity at the optimal
operating point on the ROC curves.
Calcium Lipid Fibrous
Accuracy 92. 2 ± 6.28% 96. 95 ± 2.79% 96. 17 ± 4.0%
Sensitivity 93.0 ± 2.58% 98. 95 ± 2.35% 94. 28 ± 5.23%
Speci;city 96. 5 ± 3.39% 93. 65 ± 2.77% 95. 89 ± 2.18%
AUC 0.9837 0.9947 0.9959