Wikipedia:Wikipedia Signpost/2019-03-31/Recent research

A monthly overview of recent academic research about Wikipedia and other Wikimedia projects, also published as the Wikimedia Research Newsletter.

We hereby present
[User Name]
with the award

Edelweiss with Star
of the Portal Switzerland
for contributions to the German language Wikipedia.
sgd. The Project Edelweiss-Award

In a large-scale randomized experiment on the German Wikipedia, new editors who were presented with a barnstar-like award on their user talk page were 20% more likely to remain active during the following month. This statistically significant increase in the number of users coming back to contribute more persisted for a full year (four quarters). The effect also appeared when only considering article (mainspace) edits.

The "Edelweiss-Auszeichnung" (Edelweiss with Star) was awarded in a monthly process. All users who had just made their first article edit and at least one other edit, with at least five days between their first and last edit, were considered initially eligible for the award. This was followed by a semi-automated screening process, "developed in consultation with experienced community members", to remove e.g. blocked users, corporate accounts and "advertisers". Apart from this, the award (in its lowest level) was not based on an assessment of the quality of the user's contributions. Its "description does not contain any explicit performance criteria for getting the award, other than that the editors have made their first contributions to the German language Wikipedia in the previous month; it is mentioned that there were more than 4,000 newcomers as potential candidates in a given month." The award was handed out by the author, using a role account, to around 150 users per month. She notes that

"... randomly bestowing awards seems to be an almost impossible endeavor, because awards are designed to be given to individuals who excel in their tasks. However, this experiment shows that it can succeed if two important conditions are fulfilled. First, a basic preselection has to exclude obviously undeserving candidates, such as vandals. Second, subjects who by chance do not receive the award should be an unidentifiable group who ideally are ignorant of the award’s existence."

The screening process seems to have been reasonably effective in weeding out bad-faith contributors, with only 2% of the awarded users and 3% of the control group having been blocked after more than two years.

The paper also emphasizes that close coordination with the editor community, and the attachment to a thematic portal (Portal Switzerland, similar to a WikiProject on the English Wikipedia) were important to the award's success:

"... practitioners' endorsement is most likely to be vital for any such endeavor. The backing and trust of several highly reputable community members were central to this experiment. These contacts were established via telephone calls, which were followed up by regular roundtable meetings with a group of editors willing to tackle the retention problem with the help of the experiment. They became official founding members of the project, which was thus institutionalized under the umbrella of the Swiss national Wikipedia portal, providing the award with considerable repute and a formal character ..."

In contrast, a team of Carnegie Mellon researchers recently withdrew a similar research project proposal on the English Wikipedia due to community opposition. See previous coverage from The Signpost.

See also our earlier coverage of related research: "A Preliminary Study on the Effects of Barnstars on Wikipedia Editing", "Recognition may sustain user participation"

See the research events page on Meta-wiki for upcoming conferences and events, including submission deadlines, and the page of the monthly Wikimedia Research Showcase for videos and slides of past presentations.

Other recent publications that could not be covered in time for this issue include the items listed below. Contributions, whether reviewing or summarizing newly published research, are always welcome.

See also earlier coverage of a related paper by some of the same authors: "Emergent Role Behaviours in Wikipedia – The 'How' and 'Why'".