shows the win ratio averaged over the open and the
hidden maps. As with the standard track, there are
substantial differences in the performance of the bots
between the two.
The same observations apply as in the standard
track: hard-coded bots perform well in standard situations, and not as well in nonstandard situations.
Adding nondeterminism did not change this fact.
POWorkerRush, PuppetSearch, and SCV performed
better in this track than in the standard track, whereas NaïveMCTS performed worse, being outperformed
by SCV. POLightRush and Strategy Tactics performed
slightly worse in this track, but that result follows
from the first three bots having performed better. The
lower performance of NaïveMCTS, in particular, was
to be expected, since its game tree search nature
assumes a deterministic game. Overall, however, it
seems that the low amount of nondeterminism present in this track, which is representative of commercial RTS games, did not affect the performance of
most bots excessively. The exception is NaïveMCTS,
which performed significantly worse in the open
maps.
Partial Observability Track
Five bots were used for the partial observability track:
four of the preexisting bots (RandomBiased,
POWorkerRush, POLightRush, NaïveMCTS) and one
competition entry, BS3NaïveMCTS. Each round
robin tournament consisted of 5 ; 4 = 20 games
(since we discarded self-play matches), and we per-
formed five full round-robin tournaments in each of
the eight maps, for a total of 5 ; 8 ; 20 = 800 games.
Figure 7 shows the win ratios achieved by each of
the bots in this track, organized by type of map. The
bot that achieved the highest win ratio over all maps
was POLightRush, with a win ratio of 0.670. The second-best bot in this scenario was BS3NaïveMCTS, the
competition entry, with a win ratio of 0.617. Figure 7
also shows the win ratio averaged over the open and
the hidden maps.
In terms of the observations to be made, the first
thing we see is that the random bot (RandomBiased)
performed much better in this track, which might be
due to the fact that, since there is partial observability, it is harder for the other bots to pinpoint where
the opponent is. The random bot also tends to create
a large number of workers, which can be hard to deal
with later in the game.
We see that BS3NaïveMCTS performs significantly
better than NaïveMCTS, which is to be expected,
since there is a direct improvement over it to handle
partial observability.
We can also see that BS3NaïveMCTS outperformed
both hard-coded bots in the hidden maps, whereas
this did not happen in the standard track. Hard-coded bots still have an advantage, though, on the open
maps.
Looking at the per-map results, we see that in this
partially observable setting, no bot managed to win
a single game in the very large map4, so all games
there ended in a tie. The main problem is that in
such a large map, it was difficult for the bots to find
each other, which highlights the importance of
Figure 7. Win Ratios of the Bots in the Partial Observability Track, by Map Type.
The left plot shows the win ratios averaged over only the open maps; the center over the hidden maps; and the right plot the average of all
maps.
1
0.8
0.6
0.4
0.2
0
All Maps Hidden Maps Open Maps
RandomBiased
POWorkerRush
POLightRush
NaiveMCTS
BS3NaiveMCTS
