Wikipedia:Wikipedia Signpost/2020-08-02/Recent research

Recent research

Receiving thanks increases retention, but not the time contributed to Wikipedia

A monthly overview of recent academic research about Wikipedia and other Wikimedia projects, also published as the Wikimedia Research Newsletter.

Receiving thanks increases retention of new editors, but not the time contributed to Wikipedia

The Citizens and Technology (CAT) Lab at Cornell University recently published two preprints studying the effects of Wikipedia's inbuilt "thanks button" on participation. These experiments were conducted in collaboration with community members from the Arabic, German, Persian and Polish language Wikipedias.

In the first study, all editors who had made a "thankable edit" (defined e.g. as receiving a "good-faith" and "non-damaging" rating from the automated ORES classifier) in the last 90 days were eligible to be randomly assigned into a test and a control group. Edits from the test group were then presented to experienced volunteers (334 overall) who decided whether to send a thanks for them. The researchers measured the effect on three preregistered main outcomes:

No significant changes were found in the daily labor hours contributed (comparing the six weeks before and after receiving the thank). The authors write that "this finding may result from an effect that is inconsequentially small, from the high variance in labor hours between participants, or from measurement error in the method for inferring time contributed" (which was based on grouping edits into "sessions" as proposed by Halfaker and Geiger, cf. our earlier coverage).
The retention of editors increased by 2% on average (an editor was defined as retained if they made "at least one edit to Wikipedia in a five week period starting at the beginning of the second week" after receiving the thank).
As the biggest effect, the study found that "receiving an expression of gratitude caused contributors to send 1.6 times more thanks" (according to the preprint; in an accompanying blog post, the researchers give a slightly different figure of a 43% increase).

These results are somewhat in contrast with a study by Goel, Anderson and Zia (conducted in collaboration with the Wikimedia Foundation's research team and presented at last year's WWW conference) which found "that receiving a thank has a strong positive effect on short-term editor activity", but had to base that conclusion on correlations as it was a merely observational study, instead of a randomized controlled experiment.

CAT's second study, titled "Expressing Gratitude and Feeling Emotionally Drained on Wikipedia" randomly assigned experienced editors from the German, Polish and Persian Wikipedias (recruited via banners to participate in the study) to either "review [edits by other contributors] and send personal expressions of gratitude to four Wikipedia contributors", or, in the control group, "to carry out common, routine activities on Wikipedia". Before and after this, they were asked to fill out a survey that asked whether they "feel positive about" their Wikipedia contributions, and whether they found contributing is "emotionally draining." Contrary to the researchers' preregistered hypothesis, the experiment "failed to find an effect from expressing gratitude on differences in socially supportive activity (p=0.74) [measured in article edits, article talk page edits, and thanks sent] or positive feelings about one’s contributions to Wikipedia (p=0.065)".

In general, the researchers found that (as summarized in an accompanying blog post) "People who spend more time mentoring and people who do more to monitor Wikipedia for vandalism report feeling more emotionally drained than others. Yet people who do more monitoring also feel more positive about their contributions".

The CAT team emphasizes that both studies were conducted in collaboration with community members from the involved Wikipedias, not unlike an earlier study that found a positive effect of barnstar-like awards on the German Wikipedia. In contrast, a quite similar project where researchers from Carnegie Mellon University had planned to study "How role-specific rewards influence Wikipedia editors’ contribution" was withdrawn in early 2019 after being met with resistance from editors on the English Wikipedia.

See also this Facebook discussion with two of the CAT researchers

Briefly

See the page of the monthly Wikimedia Research Showcase for videos and slides of past presentations.

Other recent publications

Other recent publications that could not be covered in time for this issue include the items listed below. Contributions, whether reviewing or summarizing newly published research, are always welcome.

"Openness, inclusion and self-affirmation: Indigenous knowledge in open knowledge projects"

From the abstract:

See also by one of the authors: "Ethics and responsibilities of open access. Lessons learned from the Wikipedia project of the Atikamekw First Nation" (abstract and 15min video)

Librarians "Reverting Hegemonic Ideology" by editing Wikipedia

From the abstract:

"From MDMA to Lady Gaga: Expertise and Contribution Behavior of Editing Communities on Wikipedia"

From the abstract:

"Does Copyright Affect Reuse? Evidence from Google Books and Wikipedia"

From the abstract:

See also university press release: "Wikipedia Readers Get Shortchanged by Copyrighted Material" and our coverage of an early presentation of the results (five years before the publication of this paper): "How Wikipedia articles benefit from the availability of public domain resources"

"Knowledge Graphs on the Web -- an Overview"

From the abstract:

How Wikipedia illustrates coup d’états in Soviet Russia and Egypt

From the abstract:

"Wikipedia's Network Bias on Controversial Topics"

From the abstract and paper:

References

← Previous "Recent research"

Next "Recent research" →

In this issue

2 August 2020 (all comments)

Discuss this story

These comments are automatically transcluded from this article's talk page. To follow comments, add the page to your watchlist. If your comment has not appeared here, you can try purging the cache.

Maybe I'm missing something but I don't quite understand the conclusion of this study. ("Receiving thanks increases retention, but not the time contributed to Wikipedia.") Give and editor thanks if one feels like doing so but it's really not going to make any difference? Maybe giving thanks and barnstars should be just considered a form of civility for its own good. Blue Riband► 23:03, 2 August 2020 (UTC)[reply]
- From reading the summary about it, I think the idea is that the editor doesn't spend longer on the site per day, but does stay for a longer number of days. But this title does somewhat conflict with that. There are caveats in that retention was defined as makingat least one edit to Wikipedia in a five week period starting at the beginning of the second week.
  My intuition and personal reaction would tell me that receiving "thanks" makes me more productive and increases the proportion of my future interactions which are of positive sentiment rather than negative sentiment (i.e. I find it easier to be cordial and my writing style may change from something that would have provoked tiresome argument into one that might lead to productive discussion), but doesn't increase the amount of time I spend or how long I will edit for. But I gather that the researchers focused on new editors rather than already long-term ones, and it would be (almost) impossible for them to systematically determine how sentiment is affected in an accurate manner. — Bilorv (talk) 22:25, 3 August 2020 (UTC)[reply]
Re: "In contrast, a quite similar project where researchers from Carnegie Mellon University had planned to study "How role-specific rewards influence Wikipedia editors’ contribution" was withdrawn in early 2019 after being met with resistance from editors on the English Wikipedia." To a researcher, a project on awarding Thanks may be quite similar to one on awarding Barnstars. To Wikipedians the two are very different, I must get dozens of times as many thanks as barnstars. Ϣere SpielChequers 15:46, 3 August 2020 (UTC)[reply]
- You're right about hinting on the absurd equation of 'Thanks' and barnstars. Researchers embarrass themselves if they don't understand differences. -- Just N. (talk) 21:31, 3 August 2020 (UTC)[reply]

I wrote this sentence and am very familiar with the similarities and differences, thank you - I am a Wikipedian as well. And nobody proposed an "equation". Also note that the German Wikipedia study that the preceding sentence refers to was about barnstar-like awards too.

Regards, HaeB (talk) 10:43, 4 August 2020 (UTC)[reply]

Confusing commentary of yours, HaeB. My remarks above weren't aimed at attacking anyone. Why you are complaining to be hit I have no idea. At what location do you think you were part of the scene ("I wrote this sentence")? Playing with ridfdles? I definitely just wanted to consent to Were's notes. Regards, Just N. (talk) 18:20, 4 August 2020 (UTC)[reply]

What is so hard to understand about the fact that I, as the author of this review, wrote the sentence that WereSpielChequers quoted above, including the comparison (not "equation") that you called "absurd"? Especially after I already told you above? Experienced Wikipedia editors are usually able to read a wiki page's revision history, or, failing that, should at least be capable of inspecting the byline on top. Regards, HaeB (talk) 08:28, 5 August 2020 (UTC)[reply]

"People who spend more time mentoring and people who do more to monitor Wikipedia for vandalism report feeling more emotionally drained than others. Yet people who do more monitoring also feel more positive about their contributions". I believe that mentoring is demanding: you're unavoidably focussed on someone else instead of your own preferences. That it is counted as just the same (as a stressing factor) as patrolling against vandalism is indeed irritating me. I'd guess that the above short summary is failing to give a better understanding of researching these quite different experience fields. If not so, the academic research designers would have delivered an embarassing approach due to not being prepared to go deep enough into differenciation of special Wp working fields. -- Just N. (talk) 21:20, 3 August 2020 (UTC)[reply]

Let me get this straight: If the Wikipedians who responded to this survey feel (on average) differently that you yourself feel or would expect to feel, then that must mean that the researchers have made "embarrassing" mistakes? Regards, HaeB (talk) 10:43, 4 August 2020 (UTC)[reply]

Hey HaeB, please read carefully! I said that it remains unclear which conclusion has to be drawn. It is unfair and not at all correct to ascribe (citation) *must mean* to me instead of correct 'could mean'. Regards, -- Just N. (talk) 18:07, 4 August 2020 (UTC)[reply]

That's not what you said. You were criticizing the researchers of either failing to summarize their own results correctly (as mentioned in the review, the quoted sentences were taken from their blog post), or else of "deliver[ing] an embarassing approach due to not being prepared to go deep enough". And you were basing this criticism on the argument that the survey results did not match what you personally "believe" about mentoring and patrolling.

I think that at this point it might be useful to read that blog post and/or the preprint itself, before launching into even further speculations and judgments about the study.

Regards, HaeB (talk) 08:28, 5 August 2020 (UTC)[reply]

For instance, we consider the topic abortion. The anti-abortion movement looks more represented than the abortion-rights one. It means that a user who randomly picks a Wikipedia’s page, has double the probability of reading an article related to anti-abortion than abortion rights. I was a little surprised to see this given the common wisdom that Wikipedia has—relative to America—a liberal bias. Perhaps that wisdom isn't so wise. Perhaps it's a case of "know your enemy". Or perhaps there is simply more to write about in terms of notable anti-abortion subjects. Answers on a postcard, please. — Bilorv (talk) 22:29, 3 August 2020 (UTC)[reply]
- The claim that Wikipedia has a "liberal bias" is an absurd over-simplification. Wikipedia has an educated white male American/European bias, per its demographics. When it comes to things like science and religion, that means it's "liberal", but when it comes to things like feminism or racial justice, it has a more conservative bias. Kaldari (talk) 20:59, 4 August 2020 (UTC)[reply]

Want the latest Signpost delivered to your talk page each month?

Home

About