Wikipedia:Wikipedia Signpost/2021-12-28/Recent research

Recent research

STEM articles judged unsuitable for undergraduates below the first paragraph

A monthly overview of recent academic research about Wikipedia and other Wikimedia projects, also published as the Wikimedia Research Newsletter.

"The First Paragraph Is As Good As It Gets": study discourages STEM students from using the rest of Wikipedia articles

A study published last month in the journal College Teaching evaluated the suitability of English Wikipedia articles on STEM topics for undergraduate students' "opportunistic learning", defined as "informal, self-regulated study to learn, relearn, or be introduced to a concept".

The 28 articles were chosen from "six disciplines for which willing academics familiar with introductory STEM topics were available to participate in this study: Biology, Chemistry, Environmental Science, Mathematics, Physics, and Statistics", plus a "General" STEM category. Within each of these, the authors selected "four diverse introductory topics commonly encountered in STEM programs [... covering] topics commonly misunderstood or important in the discipline". The four "Statistics" articles had already been examined in a previous paper by three of the authors (see our review: "Evaluating Wikipedia as a self-learning resource for statistics: You know they'll use it").

Each article was evaluated in three components, based on a revision from 14 November 2019: "the entire article, the preamble (before the Table of Contents) [what Wikipedia's manual of style refers to as the lead section], and the preamble first paragraph". The focus on the latter two was motivated by the observation that they "are easily accessed on mobile devices with small screens, and [...] may be all that is read" (quoted from the earlier paper), an assumption supported by several data points and research results.

The articles were evaluated using what the authors call the "ACPD framework" (developed in their earlier paper), assigning a score from 1 ("Not suitable for opportunistic learning") to 3 ("Recommended for opportunistic learning") in each of four criteria:

The authors summarize the resulting ratings as follows:

In the Statistics category, the authors judged the first paragraphs "excellent" with the exception of histogram. But "the preambles and the entire articles were generally poor, with many A-qualifiers (errors). Some errors were basic..."

In "Environmental Science" (the other discipline where the evaluation had flagged accuracy concerns, in the articles extinction and greenhouse effect), the study criticized "uneven, vague, overly simplistic and/or imprecise" writing, highlighting an example from the article species which said "evolutionary processes cause species to change continually, and to grade into one another". The study also took issues with "paragraphs only tangentially related to the topic [...] For example, the 'Biodiversity' (Environmental Science) article states 'Biodiversity inspires musicians, painters, sculptors, writers and other artists', which is not useful for a learner seeking to understand the concept of biodiversity".

In "Mathematics", "articles were generally instructive from an encyclopedic viewpoint, but the fluid narrative was less useful for learners unless supplemented". While there were no accuracy concerns, "C-qualifiers were frequently applied because the development was less helpful for opportunistic learning".

For "Chemistry", the study criticized that "eight of the 12 article components lacked conceptual development (C). The articles introduced concepts at a level substantially above that expected of undergraduates or assumed knowledge that most would not have".

The authors emphasize that these evaluations were specific to the suitability of the Wikipedia articles for opportunistic learning, and that "a technically correct article may be a poor opportunistic learning resource. Of course, some criticisms (e.g., accuracy) may apply more generally".

Briefly

  • See the page of the monthly Wikimedia Research Showcase for videos and slides of past presentations.
  • A podcast interview with Heather Ford (author of various research publications about Wikipedia and of an upcoming book titled "Writing the Revolution: Wikipedia and the Survival of Facts in the Digital Age" covers "the power struggles and community governance that makes the site one of the most trusted information sources on the web".
  • The Wikimedia Foundation's research team published the fourth and fifth in a series of biannual reports about its work.

Other recent publications

Other recent publications that could not be covered in time for this issue include the items listed below. Contributions, whether reviewing or summarizing newly published research, are always welcome.

"Quality change: norm or exception? Measurement, Analysis and Detection of Quality Change in Wikipedia"

From the abstract:


"Digital Communication and Interactive Storytelling in Wikipedia : A Study of Greek Users' Interaction and Experience"

This master's thesis presents results of a survey asking readers of Greek Wikipedia how useful they found its "interactive storytelling tools (hyperlinks to other articles, navigation tables, page previews, photos, external sources of information, etc.)", and about improvements they would suggest.


"A Map of Science in Wikipedia"

From the abstract:

"Analyzing Race and Country of Citizenship Bias in Wikidata"

From the abstract:

References


Uses material from the Wikipedia article Wikipedia:Wikipedia Signpost/2021-12-28/Recent research, released under the CC BY-SA 4.0 license.