Wikipedia:Wikipedia Signpost/2023-01-01/Recent research

A monthly overview of recent academic research about Wikipedia and other Wikimedia projects, also published as the Wikimedia Research Newsletter.

This paper, presented earlier this month at the Empirical Methods in Natural Language Processing conference, applies a modified version of Graham's hierarchy of disagreement to classify talk page comments on the English Wikipedia. As explained by the authors:

The authors call these "rebuttal tactics", and distinguish them from a second category of dispute tactics, "attempts to promote understanding and consensus (referred to as coordination tactics)." Coordination tactics are classified with a separate set of "non-disagreement labels" which is combined from comment types identified in several previous research publications about Wikipedia talk pages (e.g. a paper by Ferschke et al. that was summarized in our March 2012 issue: "Understanding collaboration-related dialog in Simple English Wikipedia").

• "Bailing out" ("An indication that an editor is giving up on a conversation and will no longer engage.")
• "Contextualisation" (where "an editor 'sets the stage; by describing which aspect of the article they are challenging. This does not directly disagree with anyone")
• "Asking questions"
• "Providing clarification"
• "Suggesting a compromise"
• "Coordinating edits" to the article page ("This can signal that a compromise has been found.")
• "Conceding / recanting"
• "I don’t know" (i.e. "Admitting that one is uncertain. This signals that an editor is receptive to the idea that there are unknowns which may impact their argument.")
• "Other"

The authors provide a dataset "of 213 disputes (comprising 3,865 utterances) on Wikipedia Talk pages, manually annotated with the dispute tactics employed in the process of resolving a disagreement between editors", allowing multiple labels for each comment ("up to three rebuttal strategies and two resolution strategies per utterance", see examples below).

These discussions are drawn from the authors' own "WikiDisputes" dataset, which provides information "which is annotated according to whether the dispute was resolved without the need for a moderator." This allows the researchers to identify relations between specific dispute tactics and the risk of a conversation escalating. For example, they

In particular, they examine the effect of personal attacks, finding e.g. that conversations can still recover after a personal attack happens:

Furthermore,

The study proceeds to use machine learning for automatically classifying talk page comments with these multi-labels. A BERT-based model performed best (according to three different performance metrics), but still struggled with some of the labels:

Lastly, they apply this to the separate task of predicting whether a conversation will escalate, already examined in their earlier paper that gave rise to the "WikiDisputes" dataset. Namely, they use "multitask training with escalation as the main task and tactics as the auxiliary task, such that the features that are predictive of dispute tactics are incorporated in the escalation predictions." This improves upon their earlier prediction algorithm, "indicating that knowledge of these dispute tactics is useful for tasks beyond classifying the tactics employed."

The following table (adapted from Figure 1 in the paper) shows the labeling of several comments by two different users in one talk page discussion:

• See the page of the monthly Wikimedia Research Showcase for videos and slides of past presentations.
• The Wikimedia Foundation's Research team published its seventh biannual activity report.

Other recent publications that could not be covered in time for this issue include the items listed below. Contributions, whether reviewing or summarizing newly published research, are always welcome.

From the abstract:

From the paper's section on talk pages:

From the publisher's description:

From the abstract:

From the abstract and paper: