What is Science? A Book Review of Stuart Ritchie’s Science Fictions: Exposing Frauds, Bias, Negligence and Hype in Science. --- by Roman Krzanowski, Ph.D.
We are constantly puzzled by the question of “What is science?”
This comes despite the fact, or maybe because of it, that most of our greatest achievements and failures are directly or indirectly related to science.(1) In early modern times, people conducted science without realizing it (e.g., Wootton 2016), believing instead that they were following a kind of natural philosophy. (Take, for example, Newton’s Philosophiae Naturalis Principia Mathematica). Later on, philosophers would try to explain what science is, albeit not very successfully. We had Kuhn’s paradigm change and Popper’s falsification, among others.
Maybe we would fare better listening to scientists talking about science rather than philosophers talking about it—after all, they should know better. There is some danger in this, though, because we may encounter some really anti-philosophical personalities, such as Richard Feynman’s (in)famous quip about the philosophy of science being about as relevant to scientists as ornithology is to birds. Indeed, this dictum did not add anything significant to the discussion about the nature of science or philosophy, so we can classify it together with Kuhn’s and Popper’s musings. Thus, our puzzlement over science persists.
The chaos of the COVID-19 pandemic (2020–2022) also stirred up various anti-scientific sentiments, intensifying the confusion and deep apprehension about science. Nevertheless, whatever notions laypeople, journalists, bloggers, politicians, actors, and other influencers have been entertaining about science, they have not stopped or hindered scientific work, or at least so it seems. Science mostly just continued without paying too much attention to what people thought about it like it was somewhat oblivious to social moods.
Now Stuart Richie comes along with a claim: “Science is a social construct.” This is his opening statement in the book, but who is Stuart Richie, and what is he actually saying?
Stuart Richie lectures in Social Genetic & Developmental Psychiatry at King’s College London, and he has authored several publications for experimental studies in cognitive research, brain studies, intelligence, publication bias, meta-studies, and secondary data analysis, among other things.(2) Thus, he is an insider in that he works within the kitchen of science, so he actively practices it rather than just writing about it. He, therefore, seems qualified to comment on what things are cooking and how they are cooked.
But how is science a social construct, according to Stuart Richie? The usual claim that science is a social phenomenon (Social construction 2022) argues that there is no direct link between science and nature. This is not how Richie sees science. For Richie, science remains a search for objective truth, the pursuit of facts by relying on experiments, repetitions, and reproducible results. Richie’s social dimension of science means that science itself is a social construct made by society and embedded into our social structures, so it is subject to social ills, just like any human activity is. Thus, science suffers from being our own creation. The social aspect of science comes into play not in the scientific methodology per se but rather in everything around science. Science—as a complex of cultural systems comprising academia, papers, grants, tenures, hierarchies, competitions, and so on—is a social enterprise. This is what Richie means when he claims that science is a social construct, and the image of science from this perspective is not a pretty one.
Richie presents his account of science in three acts: Act One is the story of science’s dirty side, namely through case studies, methods, and evil ways. Act Two then discusses why this happens. Finally, Act Three ponders whether we can do anything about it. Richie ends his story with 10 commandments (one may well wonder whether this number is coincidental or deliberate) for how to read and interpret scientific papers. In the wake of the COVID-19 pandemic, this section is worth reading before we get beguiled by the hype about new drugs or breakthrough medical discoveries, regardless of whether it comes from the popular press, YouTube or other social media platforms, personalities, politicians, or respected journals.
Act one of Richie’s story, the dirty side of science, asks how does science make itself known? This generally happens through publications, with top scientific journals like Science, Nature, The Lancet, and such like carrying particular weight. These journals are supposed to present significant, top-quality research that is thoroughly vetted for coherence, verifiable results (which usually take the form of breakthrough discoveries or illuminating insights), and a strict adherence to accepted methodologies and the explicit and implicit ethical principles of science. Of course, there are hundreds of high quality journals with broadly similar publication standards, as well as many more where these rules are, let’s say, relaxed somewhat. Everyone knows this, so if you publish in one of the former journals, your career will flourish, while publications in other journals do not have such a stimulating effect. Indeed, publications in top journals gain recognition and citations, which act as a sort of pixie dust for authors in helping them to fly up the academic ladder and win grants. The names of these esteemed authors and their groundbreaking research may also enter into the lore of science as they are cited in textbooks that students will be tested on. These authors may become giants upon whose shoulders…
It therefore comes as a surprise that many of the studies published in top journals, when put under close scrutiny, turn out to be fraudulent. This was the fate of Zimbardo’s famous, or more accurately infamous, Stanford Prison Experiment, which showed how people take on roles imposed by circumstances, something that resurfaced more recently with the Abu Ghraib revelations. The same applies to the study of Stanley Milgram that was supposed to demonstrate that people will blindly follow orders that go against
their moral principles. You may say these are ancient history, having happened years ago, and we should move on. Well, we did, just not in the right direction.
A study from 2019 reported that out of 3,000 papers published in the top medical journals, 396 studies that advised a significant change in medical practices were completely unfounded. The findings of the studies were never replicated (i.e., confirmed), even though the golden rule is that scientific results should be replicable to be deemed reliable. Furthermore, the flawed claims these studies made were not trivial matters. For example, they suggested changes in practice for childbirth, allergy treatment, and the treatment of heart attacks and strokes, so these changes were at worst harmful to patients or ineffective at best.
A review published by the Cochrane Collaboration,(3) an organization dedicated to the quality of medical research, reassessed the published work and concluded that 45% of medical studies lacked sufficient evidence to support their conclusions. In other words, the findings of these studies could not have been replicated (i.e., verified).
The evil ways in which science gets distorted do not register as capital vices, but they still comprise an impressive collection. Among them, Richie lists fraud, bias, negligence, and hype. Fraud in science comes in many forms. It may be as simple as falsifying results for breakthrough surgical procedures, as was the case with Paolo Macchiarini of The Karolinska Institute, whose results were published in The Lancet as being based on solid evidence. As one may guess, following investigation into his work, his supposedly life-saving procedures turned out to be ineffective. Macchiarini eventually lost his job and The Lancet retracted his paper. Fraud also occurred in the case of Woo-Suk Hwang, who published a paper in Science about successfully cloning human stem cells. Unfortunately for him, further investigation revealed that the images used in his papers to prove his claim were doctored. On the more positive side, we could mention that he did successfully clone his pet dog, but this did not help him keep his job or academic reputation.
Richie reports that according to the Retraction Watch Leaderboard, which is a website tracking the authors of retracted scientific papers, just two percent of individuals are responsible for a quarter of retractions. This relatively small number of perpetrators is reflected in how the Leaderboard’s top achiever has 183 retractions to his credit! We can therefore talk about it being just a few bad apples, but are they spoiling the barrel? Indeed, the problem is not one of numbers, because these retracted papers have often reported flawed experimental results that give false hope to gravely ill patients or influence the procedures used by medical staff, causing (often irreparable) harm (e.g., the Macchiarini case). This is before we even consider the collateral damage, such as the money lost and the distrust of a shocked public.
Next on the list of evils is bias. In publications, bias refers to papers focusing on reporting positive, significant results, because these are generally the ones that get published. Papers reporting null or negative results are usually desk-rejected, although things are slowly changing here, as Richie points out. After all, who is interested in a seemingly unsuccessful study? Successful studies, meanwhile, report significant experimental results, and they attract citations and readership, so ratings go up. Who wants to read about null results from an unsuccessful study? Actually, some may. Studies with negative or null results can still provide valuable information, such as challenging the results of a successful study.
Anyway, what is a “significant result”? It’s generally regarded as one that confirms an experimental hypothesis with a p-value of less than 0.05.(4) This convention for evaluating the significance of experimental results was conceived by Ronald Fisher in the 1920s, but it has no deeper meaning. Indeed, it is arbitrary. It should not be compared with, for example, using a threshold for blood alcohol concentration (BAC) to distinguish drunk drivers from sober ones, because while it is still somewhat arbitrary, it does relate to a driver’s ability to safely drive a vehicle. The required p-value is arbitrary, yet it is a deciding factor in whether a paper gets accepted for publication or not. (Note that in high-energy physics a significant result must pass the five-sigma test, which is also a kind of p-test but much more stringent in that the required p-value is 3x10-7, which would be a killer outside physics.)(5) Acceptance of a paper can trigger a chain of events, resulting in recognition, grants, position, prestige, and maybe even an interview with Vanity Fair (e.g., the Macchiarini case). Who could resist such temptation?
Thus, an arbitrary number has become an arbiter of science, at least for studies depending on experiments in a field other than physics. Like any statistic, however, a p-value can be manipulated in many ways (i.e., p-hacking) by changing data sets, altering experimental hypotheses, and manipulating experimental data. Human creativeness can be unlimited for such things. A hacked p-value means hacked research, meaning that science has been hacked. If you need successful results from a study, and the p-value is not cooperating with you, simply change the statistics you use. You need to be expert at massaging data, which is what p-hacking requires, so only an expert can unravel these practices. Where does this leave us, the public, and science, though? How can the public trust experts when the experts cannot trust each other?
For another two sins committed when practicing science, negligence and hype, readers should consult Richie’s book directly.
Next comes the second act of Richie’s science saga. Everything has its causes, so fraud, bias, negligence, and hype in science must as well. The main cause is us, and the main victims are us, the public, scientists, trust in science, and science itself, or at least our idea of it. But why does this happen? In the scientific world, like everywhere else, everyone wants to get ahead. There is nothing wrong with this, and it is what makes us human, at least for many of us in some way. It is how we go about doing it that may cause problems. Richie says that the system of incentives in the scientific world “incentivisises scientists not to practice science but to meet its own perverse demand.” Simply put, they are perverse incentives. The scientific establishment imposes a simple measure of progress, namely the number of publications, which translates into the now infamous prime directive: publish or perish. It matters little what you publish as long as it is published in prestigious journals and in great quantity. Of course, there is a tacit assumption that these publications are the best stories produced by science, but Act One of Richie’s book showed that this is often not the case, maybe too often. But what does “too often” mean in this case, you may well wonder?
Incentives are originally provided to stimulate and support good research, and there is nothing wrong with that, but in reality, these incentives often work contrary to expectations. There are many ways in which incentives can lead to perverse outcomes. Grant-dispensing institutions, profit-oriented private companies, big and small pharmaceutical companies, and the chemicals industry tend to support research that benefits their bottom line. Studies showing anything contrary to this—such as harmful side effects, placebo effects, or poor efficacy—are rejected or filed away in a basement somewhere. Studies that align with founding agencies’ objectives receive support (i.e., incentives),(6) while other studies get nothing. It is therefore unsurprising when studies show the expected results (see also Strevens 2020). If you publish more, you get awarded more funding, so you do whatever it takes to publish more. The tenure-track system also favors those who publish frequently in high-flying journals, rightly so in principle but without considering the essence of what they publish. Thus, you publish as much as you can, whatever and wherever you can. To aid this effort, predatory journals will publish anything for those who can pay the publication fee. This in turn encourages the so-called salami-slicing of research. This has nothing to do with Italian sausages, but it relates to a practice were a large study whose findings should ideally be published as a single paper is instead divided into a series of shorter papers (i.e., salami-slicing) covering different areas of the main study. When the number of publications counts, this is the way to increase it, but if what counts are reporting significant findings, this is a counterproductive practice.
And now onto Act Three of Richie’s story: Can we do anything about it? With rather limited enthusiasm for the self-healing of humanity, Richie discusses ways to change our scientific practices. It seems that the chances of healing are, let’s say, slim. We are what we are, and our science practices reflect this. (We need to be careful to distinguish scientific practices from the methodology of science.) The whole system of science and its social construct—which is dependent on, and embedded in, the economic and political structures—does not bode well for the future. Science in this respect is an expression of our modern society with its drive for success and recognition, as well as its competitive culture of winner takes all. It does not mean that science will inevitably fail, though, because we will still have amazing discoveries and make progress in developing mind-boggling inventions. A sea of dirt will also be there, however, with tsunamis of hype swaying the public and feeding into all kinds of anti-whatever camps of less-sensible but highly vocal groups. Is it therefore a surprise that “people do not believe in science,”(7) as we have often heard during the pandemic? If people dismiss the Apollo moon landings as fake, why would they not question the COVID-19 vaccines? Are we surprised? However, do not blame science or the mythical “them,” because the fault lies with us, the ones doing the science. The repair work should start with us, with the scientific establishment.
Of course, there is much more detail to every story that Richie narrates. There are more examples of nuanced transgressions committed in scientific publications and research, fraud, and personal bias, both out of stupidity and arrogance—the list is long. Indeed, it reads like a crime novel, and it makes little difference that it is a white-collar crime. Again, you have to read about these in the book, but they certainly make for interesting and rewarding study.
So, what lessons does Richie’s story have for all of us? There are plenty. First, science is a survival game, and it must be played once you are in it. Some are good at science, some are good at games, and some are good at gaming science, but who wins? It’s not necessarily those who are good at science. But in the long run, science seems to possess some self-cleansing properties. When the fads, fears, and hype subside, diamonds are left in the dust. This is why we read classics. Second, do not read newspaper reports about miraculous new cures, foods, or whatever. They either do not exist, or there is only a slim chance that the popular press and trendy websites know about them, so they are merely confusing matters, as they are prone to do. Thirds, learn, learn, learn! Your whole life should be one of learning. This will help, but do not expect everyone to do it. Some people will never know any better, so you should avoid them. Fourth, the truth, despite many doubting its value or existence, always eventually prevails in science, philosophy, and humanity, just maybe not during your lifetime.
So, who should read Stuart Richie’s book? Anyone thinking about a career in science should certainly consider it. Indeed, young, idealistic scholars should realize they are not entering an idealistic field purely dedicated to the pursuit of truth and knowledge but rather a fierce neoliberal game of survival where anything is permitted as long as you do not get caught. In this world, you better learn the tricks of the trade or you will soon be gone unless you are independently wealthy. This is a perverse point of view, of course—it is infinitely preferable to invest in honest studies because they age much better, even if they are less spectacular.
Should anyone else read Stuart Richie’s book? The universities and their governing bodies certainly should before they turn the screw to tighten the publishing requirements of their staff. Indeed, they should think twice about it and maybe start promoting research quality rather than quantity. Unfortunately, there is little hope for this. Quality research requires time, money, patience and a tolerance of failure, something that managers have very little time for. The managers responsible for the rules usually have nothing to do with science and generally only care about the stats, because this is what they understand, if anything, and this is how they win the game that they are playing.
Who else should join the reading queue? The educated public is certainly a likely audience. With the threat of COVID-19 seemingly diminished, there is potential for similar events in future, so we better learn how to interpret the scientific reports of new treatments, cures, medications, and so on. We will be armed with some knowledge, even if we are still powerless. Nevertheless, it is better to be a dissatisfied human being than a satisfied pig, as the old dictum goes. A modern equivalent of this could be the choice between the blue and red pills in The Matrix. As you know, the choice is tough.
Now, does Richie’s book have any import for science outside of experimental research? The prime directive is real, and it works in every corner of academia, so the game of publications and grants is what today’s science is all about. Studies that do not rely on experiments unfortunately lack the truth serum of reproducibility, which is why it is difficult to spot the bogus research and ideas within them. Only time will ultimately reveal them.
And what about Richie’s original claim that science is a social construct? We believe that Richie proves his case. Science is embedded in societal structures, so it inherits all of their characteristics, both good and bad. This only applies to the operational aspects of science, however, and science in itself as the search for truth is unaffected in principle by this social dependency. The social dependencies hinder the reception of science in society, but the idea of science does not. This is why we still believe in it, at least some of us.
The optimistic message to take away is that science is still science, and it represents the greatest intellectual achievement of the human race so far, at least when done right. It is not a playground of changing moods and social trends, as Richard Rorty, Nelson Goodman, Stephen Shapin, Simon Schaffer, and to some extent Thomas Khun, Bruno Latour and many others would like to have us believe. In the absence of political and economic pressures, if such an ideal world could exist, or with sufficient time, science will refocus itself on its original objectives. This reflects what science was meant to be, and always will be: the search for truth (e.g., Grayling 2021). Remember this.
References and Footnotes -
Armstrong, M. 2021. The Most and Least Trusted People in the World. Statista. Available at https://www.statista.com/chart/26000/most-least-trusted-people-global/. Accessed on 04.04.2022. Boghossian, P. 2007(2006). Fear of Knowledge. Oxford: Oxford University Press. Grayling, A. C. 2021. The Frontiers of Knowledge. Dublin: Viking.
Strevens, M. 2020. The Knowledge Machine. Dublin: Penguin Books.
Social Construction. 2022. Social Construction of Science. Available at
http://sociology.iresearchnet.com/sociology-of-science/social-construction-of-science/ . Accessed on 24.03.2022.
Wootton, D. 2016(2015). The Invention of Science. London: Penguin Books.
(1) The term “science” as we understand it today is relatively recent, having entered into popular usage in the 19th century (Graylink 2021).
(2) See Stuart Richie on Google Scholar: https://scholar.google.com/citations?user=9TsCy3IAAAAJ&hl=en
(3) Cochrane is an independent, diverse, global organization that collaborates to produce trusted synthesized evidence, making it accessible to all: https://www.cochrane.org/.
(4) A statistical measurement used to validate a hypothesis against observed data.
(5) Switching between a p-value in particle physics and a p-value in medical and social sciences would probably result in minimal progress in the latter domain and in the profusion of elementary particles, which may be for the better.
(6) One can also read about the famous Eddington experiment for “confirming” Einstein’s general relativity theory to see how the human element enters science (Strevens 2020).
(7) Scientists still have better trust ratings than politicians, though (Armstrong 2021).
Thumbnail Credit: Michael Schiffer.com/@michael_schiffer_design