56 AI MAGAZINE
Another set of techniques focuses on nonpara-
metric statistics from labeled data to include classifi-
cation and connectionist approaches. Classification
methods seek to organize the data into categories
by determining a decision boundary between the
categories (for example, support vector machines).
These methods have grounded mathematical theory
for repeated results, but provide limited robust oper-
ations in complex scenarios.
The current set of AI techniques follow from DL
rooted in connectionist neural networks. Minsky and
Papert (1969) highlighted that the original perceptron
(one layer) could not perform the exclusive OR (XOR)
function. The challenge was quickly solved by adding
another layer in the network (two layers) such that
the middle layer learned the XOR function, while the
final layer did the classification. An example is finding
the XOR bounding box (inside versus outside the classification boundary box). Adding the middle layer for
the four sides of the box (top, left, bottom, right) supported XOR learning. Extensions of these ideas have
resulted in the convolutional neural network (CNN).
The CNN includes choosing a kernel (for example,
5 × 5 bounding box in an image), learning the classification, and typically using backpropagation with
stochastic gradient descent to learn one layer at time.
By connecting all of the partial layer results, reasonable
classifications are obtained — subject to the appropriate choice of the number of layers. More exemplar
data increase the likelihood the system would be able
to perform well with increasingly complex situations.
The advantage of the CNN is that the kernels do not
need the full transformation equations, and with an
autoencoder as an artificial neural network, neighborhood functions can be used to filter the results for
parametric reduction. The number of layers in deep
NN is part of ongoing work; for example, hyperpara-metric learning has replaced the sigmoid function in
traditional NNs to include the rectified linear unit for
Multimodal DL (Ngiam et al. 2011) has focused on
leveraging the different learning approaches over the
Figure 3. ML Approaches Based on Available Data.
Prior Information Complete Bayes’ Decision Theory
Labeled Data Unlabeled Data
Supervised Learning Unsupervised Learning
Symbolic Classifiable Connectionist Propositional Evolutionary