groups. Once the unbalanced strata were identified
by the human raters, then we filtered the posts to the
corresponding comparable subpopulations. We then
modified our method to compute a local average
treatment effect only over the strata deemed to be
balanced by human assessments, so as to assess the
effects of specific linguistic features of comments in
future risk to suicidal ideation
With the help of these human assessments and as
shown in table 2, we found that the effects of getting
a token in a comment may not be homogeneous.
Certain users may see little effect of getting a token
(low-propensity strata), while others see a large effect
(higher-propensity strata). By employing human
raters in this task, we showed how the outputs of
causal inference methods can be amalgamated with
expert feedback to improve results.
The fact that we obtain better results by incorpo-
rating human feedback in an AI task like the one pre-
viously described is further clarified while investigat-
ing the context of use of specific linguistic tokens in
comments, and situating those tokens in theoretical
framework of social support. In the users who show
reduced likelihood of suicidal ideation in our dataset,
we found comments on their posts to contain greater
expression of esteem ( 31 percent) and network sup-
port ( 23 percent), followed by emotional support ( 16
percent). Informational support ( 9 percent) and
acknowledgments ( 5 percent) were relatively lower
for comments containing tokens that decrease the
likelihood of posting about suicidal thoughts. Over-
all, this distribution indicates the positive impact of
esteem and network support in reducing one’s future
risk of suicidal ideation expression, a result which
can be accurately inferred by filtering the outcomes
of causal inference based on human feedback.
Table 2. Qualitatively Assessed Post Pairs and Associated Comment Tokens.
Post pairs and associated comment tokens qualitatively assessed to correspond to balanced and imbalanced treatment and control groups.
Text has been slightly paraphrased to protect the identities of the users.
Token Strata Treatment Post Control Post
High Propensity Strata
6 a reason behind my depression is how
small by body frame is. i’ve never cared
much about muscle but it’s obviously one
of the reasons i’ve been alone (friendships
and relationships) for my whole life.
i’m aware there’s no way to avoid pain
100%, which is why i’m attempting to go
for the least painful way. we’ve talked in
detail about exactly why our issues are
troubling for each of us, so he knows that
6 i don’t even know what all i feel. ashamed,
angry, at myself and at the family that
never did a thing to support me before.
i’m seriously thinking about just pulling
out i’m tired of trying, and failing, over
and over again.
feeling like shit but noone to talk to, just
need a friend who can cheer me up.
noones online on facebook that i can talk
to so just alone right now ...
Low Propensity Strata
seek 2 i realize that i’m having depression. i have
not showered for a week now, unable to
sleep and always thinking negative about
i noticed during the livestream, even
though that he wasn’t using their (i’m
assuming) condenser microphone, i felt
that his volume and the tones of his voice
sounded much more “comfortable” with
1 an american football fan but i am
intrigued by the world cup. i remember
watching 4 years ago and was fascinated.
does anyone know of a quality app i can
get on my phone that i can use to keep up
greetings people, greetings people, i am a
worthless nobody. i guess i want to take
more of your time in the vain hopes that
you’ll somehow be able to make me feel