The study involved 61 first-year undergraduates students enrolled in a statistics and probability course
for engineering majors at Ben-Gurion University. The
study was carried out during the middle of the semester after the students had acquired basic knowledge
of probability theory and undergone a midterm. All
students were given a home exercise to familiarize
with the TinkerPlots software. In total 32 students
were presented with a plan solution, and 29 were presented with a list solution.
The study was conducted in a designated lab in
which each student was situated in front of a computer. In the first part of the study, all students were
provided with the COIN problem description in writing, and the expert solution to the problem was subsequently presented to them using the list or plan
visualization, depending on their assigned condition.
In addition, all students were shown an (identical)
snapshot of the TinkerPlots desktop following the
solution procedure. The students were asked several
comprehension questions about the solution, such as
whether (and why) the interaction shown to them
constitutes a correct solution to the COIN problem;
to indicate the role in the solution of one of the
actions in interaction; to explain the solution to a
friend using free text. There were two purposes for
these questions: First, to confirm that the students
comprehended the solution; and second, to compare
between students’ self-explanations of the solution to
the COIN problem in the two experimental conditions. Students were allocated up to 20 minutes to
complete this part of the study.
In the second part of the study, students were
asked to solve two new problems in sequence using
TinkerPlots. Students were allocated up to 30 minutes
to complete this part of the study. The problems were
taken from the curriculum of an introductory course
in probability. We wanted the problems to be nontrivial but still possible to solve by the majority of
students in the allocated 30 minutes. We chose problems whose solutions exhibited similar concepts (
reasoning about combinations and events in sample
space). The test problem, called DICE, did not mention explicitly the notion of expectation, and it
required students to reason about the disjunction of
complex events that relate to the sample space.
John and Mary compete in a dice-tossing game. They
take turns tossing a die, and sum the result of each
toss. The winner is the first to accumulate more than
10 points. Compute the probability that ( 1) John will
win after two rounds of the game. ( 2) Either John or
Mary will win after two rounds of the game.
We collected the TinkerPlots log from each student’s
interactions as well a snapshot of their desktop
recording during the activity.
We hypothesized that students assigned to the plan
condition would exhibit better performance than
students assigned to the list condition when using
TinkerPlots to solve the test problems. We measured
student performance using the following metrics: the
length of interaction (in minutes); the total number
of actions in an interaction; the ratio of redundant
actions in an interaction, which represent mistakes;
and exogenous actions that do not play a part in the
We provide a description of students’ performance
for the DICE problem. All of the results we report
below were statistically significant p = 0.04 using a
nonparameterized two-tailed Mann-Whitney test.
We found that the plan visualization significantly
improved students’ performance across all measures.
These results are summarized in table 2.
Specifically, the average interaction length of the
students in the plan condition (AVG = 9.01 minutes,
SD = 3. 92) was significantly shorter than the average
interaction length for students in the list condition
Figure 8. List Visualization of the Expert
Solution to the COIN Problem.
Add device 1 to sampler1
Change the label of element 0 in device 1 in sampler 1 to “0”
Change the label of element 1 in device 1 in sampler 1 to “ 1”
Set the repetitions in sampler 1 to “1500”
Set the draws in sampler 1 to “ 3”
Run sampler 1 and generate columns Draw1,Draw2 and Draw3
Add an attribute named "Sum" to table 2
Edit the formula of attribute "Sum" in table 2 to Draw1+Draw2+Draw3
Add plot 3
Drag the attribute "Sum" to plot 3
Divide the values in plot 3
Average on plot 3 of attribute "Sum"
Table 2. Performance Measures on DICE Problems
for Students in Plan and List Conditions.
Time (Minutes) Number of Actions Redundancy
Plan 9.01 39. 65 28%
List 12. 47 57.06 46%