WINTER 2018 85
ing adversarial collaboration for scientific advance-
ment. Our current system of reporting almost exclu-
sively positive results is akin to the machine learning
problem of training on positive data: we lack solid
negatives. Adversarial frameworks, such as the legal
system, are important in areas where truth is under-
stood to be unattainable. Adversaries, when properly
motivated, can be relied on to find the flaws in one
another’s arguments or evidence, and would not take
a contrary position to vacuous, irrelevant, or merely
incremental hypotheses. We believe that science, and
especially AI, needs to adopt such an approach, and
we propose to start simply — with bets.
Betting has a long history in human society, and
scientific bets are an old tradition: legend has it that
Newton’s Principia was motivated by a bet. Hawking
Guidelines for Bets
; The subject of the bet must be relevant to AI or a related field.
; We want to encourage bets that promote or advance the field, and discourage vague or nonscience bets such as “when will be the next AI winter” or “company X will be out of business by 2020”
; Bets are for reputation, not money.
; The period of the bet must be a stated length of time or until a particular event or situation occurs. In all cases, the conditions for time of adjudication must be clear at the onset.
; The bettors must provide an argument explaining why the subject of
their prediction is important and why they think they will be proved
right. All arguments will be in the public domain.
; Individual people or small groups make bets using their real names.
Large groups, formal organizations, and pseudonyms are not allowed.
; Every bet must have an adjudicator. Adjudicators must be agreed upon
by the bettors and can be assigned if needed. The adjudicator must
approve the operationalization of the bet.
; The outcome of predictions is usually decided through mutual agreement by the bettors; the adjudicator serves the role of being a neutral
party who can resolve disagreements and the bettors agree to live with
that resolution. Thus, while the two bettors have the goal in specifying
the bet to make their “side” as clear as possible, the adjudicator has the
goal of ensuring the operalization is as clear as possible. There are no
; Bets and predictions are always ultimately win-lose or true-false. There
are no partials. This is important. A bet must be formed such that it can
be adjudicated in this way.
; The data used to adjudicate a bet must be publicly available. There are
; Bets may not be revoked once final, unless through agreement by all
; The bet and outcome will be made public.