tion. The early algorithms for bootstrapped semantic
lexicon induction exploited the observation that
semantically related nouns often co-occur in conjunctions (for example, lions and tigers and bears), lists
(for example, lions, tigers, bears ...), appositives (for
example, stallions, male horses), and compound
nouns (for example, tuna fish). These bootstrapping
algorithms began with a few seed nouns for the
semantic category of interest and identified nouns
that co-occurred with the seeds, in close proximity or
in specific syntactic constructions (Riloff and Shepherd 1997; Roark and Charniak 1998). The resulting
dictionaries were far from perfect, but they showed
that bootstrapped learning of semantic dictionaries
was possible and that this approach was a promising
direction to pursue.
The mutual bootstrapping algorithm described in
the next section brought together these seemingly
different tasks of learning information extraction
patterns and learning semantic dictionaries. Mutual
bootstrapping showed that both types of knowledge
could be learned at the same time, and that doing so
was mutually beneficial.
Mutual Bootstrapping
The mutual bootstrapping algorithm in our 1999
AAAI conference paper hinged on two key ideas: ( 1)
words and contextual patterns can be used independently to identify instances of a semantic category, and ( 2) multiple knowledge sources can serve as
the foundation for bootstrapped learning when
played off each other. The process of simultaneously
learning two different types of knowledge by alternately leveraging one type of knowledge to learn the
other was called mutual bootstrapping.
In our work, we identified two types of knowledge
that are often sufficient to identify the semantic category of a noun phrase (NP) in context: ( 1) the head
noun of the NP itself, and ( 2) the context surrounding the NP. For example, consider the sentence: The
brown dog barked at the cat. We can infer that The
brown dog is an ANIMAL in two ways: ( 1) because we
know that the word dog commonly refers to ANIMALS, or ( 2) because barking is an action usually performed by ANIMALS. Of course, there are exceptions,
for example a hot dog often refers to FOOD, and people can also bark when they are angry. But either type
of knowledge, lexical or contextual, is usually sufficient by itself to make a strong inference about the
meaning of a phrase in context.
Figure 1 illustrates the mutual bootstrapping learn-
ing process. The bootstrapping cycle begins with a
text corpus and a small set of manually defined seed
words for the targeted semantic category. For exam-
ple, suppose you want to learn words and patterns
for the semantic category DISEASE. The input would
consist of a large collection of texts that frequently
mention diseases and a small set of seed words that
refer to diseases, such as cholera, flu, listeria, measles,
and tuberculosis. The seed words are used as the initial
semantic lexicon, which is iteratively expanded dur-
ing the bootstrapping process.
The AutoSlog pattern generator (Riloff 1993) was
then applied to the text corpus in an exhaustive fashion to produce an extraction pattern for literally
every noun phrase (NP) in the corpus. This process
produces an enormous set of patterns paired with the
NPs that they extract, which collectively represent all
of the noun phrase contexts in the corpus. The mutual bootstrapping algorithm uses this data, along with
the initial semantic lexicon, both to induce a pattern
dictionary for the semantic category and to grow the
semantic lexicon. The learning process has two alternating steps. First, all of the patterns are scored based
on their strength of association with the terms in the
semantic lexicon (for details, see Riloff and Jones
1999). The highest-scoring pattern is added to the
pattern dictionary. Second, in a leap of faith, all of
the NPs extracted by the newly added pattern are
assumed to belong to the targeted semantic category
and their head nouns are added to the semantic lexicon. The process then iterates: all of the patterns are
rescored based on the expanded semantic lexicon, a
new pattern is selected and added to the pattern dictionary, its extracted head nouns are added to the
lexicon, and so on.
To return to figure 1, imagine that the pattern
infected with <NP> is the highest-scoring pattern
because many of the seed terms occur in that context. This pattern would be added to the pattern dictionary, and the head nouns of all NPs that occurred
in this pattern context are assumed to be DISEASES
and added to the semantic lexicon. In the example,
the newly added terms would be ebola, malaria,
plague, pneumonia, and tularemia. The expanded
semantic lexicon then serves as a larger set of seed
terms for the next iteration of bootstrapping.
While this approach worked well in many cases,
one problem was the leap of faith that all NPs extracted by a pattern belong to the same semantic category. Even when strongly associated with a category,
contextual patterns can co-occur with NPs of different semantic categories because very few contexts
occur with a single semantic category 100 percent of
the time. Occasional incorrect lexicon entries typically will not change the course of bootstrapping
very much, especially if the errors are distributed
across different competing categories (that is, if the
errors represent a variety of different semantic classes). But a serious problem exists when many incorrect lexicon entries belong to the same (incorrect)
semantic class. This phenomenon arises when some
contexts systematically occur with multiple semantic categories. For example, locations and temporal
expressions both frequently occur with event phrases and the preposition in or on, such as happened in
(Boston/November) and occurred on (Wall Street/Mon-
