Summary
In Science Fictions, Ritchie argues that modern scientific research suffers from a number of issues – including outright fraud; publication bias and human bias; negligence and error; and sensationalism through media hype – that undermine its ability to be objective and accurate. He describes methodological issues involved in the replication crisis, such as p-hacking, as well as broader systemic issues, such as perverse incentives in the peer review process and the problem of null results being difficult or impossible to publish, leading to under-reporting of many important findings. He also details the ways in which technology such as Photoshop has made it easier for researchers to commit subtle but serious fraud.
The book focuses on the prevalence of fraud, bias, negligence, hype, and also perverse incentives and practices common in peer reviewed science. He notes that science is a practice that is built on trust and is not immune to human flaws and groupthink habits.
He extracts numerous examples from fields such psychology, medicine, origin of life, chemistry, biology, among others. And emphasizes that even top organizations and scientists have fallen prey to ill practices and scholarship. Impacting, at times, expert recommendations such medical practices & treatment, therapeutic or self help related matters; resulting in economic and resource wastage and even damage to the health of people, including death. He explains that journals often express bias towards new studies than they do in follow up studies that attempt to replicate or find the original studies to be null or insignificant.
Throughout the book, Ritchie offers examples of egregious fraud in scientific research, such as the misleading studies on trachea replacement published by Paolo Macchiarini that led to his patients dying and was covered up for years by the institutions supporting the research. But he also points out less obvious, pervasive fraud – for example, a survey that found evidence of manipulated images in 4% of the 20,000 scientific papers analyzed.
He alludes to a 2016 survey of over 1,500 researchers of the journal Nature – around 90% of chemists said that they’d had the experience of failing to replicate another researcher’s result; around 80% of biologists said the same, as did almost 70 % of physicists, engineers, and medical scientists.
Discusses the prevalence of "p-hacking" in many fields of science, in that scientists fudge their data or ignore data that nullifies the significance of their findings, leading to prevalence of bias. The desire for elegant looking theories befalls even physics where beauty of models is focused on more than testing if they are even true.
He argues that even well-intentioned researchers can be tempted to over-hype their results or repeat experiments until they receive the result they wanted simply by the desire to gain fame or advance their career. This occurs with scientific organizations as well. One of the examples of hype was NASA's announcement in 2010, of a bacteria that had altered the backbone of their own DNA, replacing the phosphorus with arsenic and being able to survive in phosphate free environment. The study had implications of alternate forms of life with different genetic material than life as we know it. After replication attempts, it turned out that arsenic contamination with phosphate is what accounted for bacterial growth, not arsenic itself.
In the final chapter, Ritchie offers possible solutions for improving the methodology, incentives, and outcomes of scientific research, such as encouraging researchers to pre-register their work to avoid the temptation to alter their conclusions to fit the outcome, and using technology to identify common errors that can creep into and distort statistical analysis.
