be considered to attempt to make the best decisions
possible, particularly when using units with varied
abilities (figure 3), but the problem space is not nearly as large as that of the full game, making feasible
exploratory approaches to learning domain knowledge (Weber and Mateas 2009). There appears to be
less research interest in this aspect of RTS game AI
than in the area of large-scale, long-term strategic
decision making and learning.
Reinforcement learning (RL) is an area of machine
learning in which an agent must learn, by trial and
error, optimal actions to take in particular situations
order to maximize an overall reward value (Sutton
and Barto 1998). Through many iterations of weakly
supervised learning, RL can discover new solutions
that are better than previously known solutions. It is
relatively simple to apply to a new domain, as it
requires only a description of the situation and pos-
sible actions, and a reward metric (Manslow 2004).
However, in a domain as complex as an RTS game —
even just for tactical decision making — RL often
requires clever state abstraction mechanisms in order
to learn effectively. This technique is not commonly
used for large-scale strategic decision making, but is
often applied to tactical decision making in RTS
games, likely because of the huge problem space and
delayed reward inherent in strategic decisions, which
make RL difficult.
RL has been applied to StarCraft by Shantia,
Begue, and Wiering (2011), where Sarsa, an algo-
rithm for solving RL problems, is used to learn to
control units in small skirmishes. They made use of
Figure 3. A Battle in StarCraft.
Intense micromanagement is required to maximize the effectiveness of individual units, especially spellcaster units like the
1311 965 160/200