“As scientists,” West and Munafò write, in the editorial published in the Addiction journal, “we are supposed to aspire to be seekers of truth, led to whatever conclusions the data take us; but in reality we are also human, and therefore subject to the same array of biases that apply to everyone, scientists and non‐scientists alike. This includes a tendency to see patterns in our data that may not be there, the inclination to seek out evidence that confirms our preconceptions and a tendency to believe what we want to believe.”
Research looking at all aspects of electronic cigarettes increased globally as vaping was adopted by millions of smokers. The quality of these studies has varied in quality, from dispassionate analysis to blatant corrupted lies.
The scientific method ought to provide a method of checks and balances for the studies being carried out, but this has clearly failed. The pair say: “Concerns around the reproducibility of much published scientific research have led to questions about whether the scientific method is being applied sufficiently rigorously.”
West and Munafò highlight three problems with the research being published:
- The prioritization of novel ground‐breaking findings
- The emphasis on the data telling a story
- The tendency for results that conform to a dominant narrative
The majority of published research now includes a COI (conflicts of interest) statement, but this has failed to remove actual conflicts of interest in work. West and Munafò say: “These typically focus on financial COIs (and are themselves often imperfect and/or incomplete). There has been much less attention paid to the potential role of how cognitive and motivational factors shape the interpretation of evidence.”
Areas of concern the researchers highlight include studies claiming correlational evidence demonstrates a causal association of a youth gateway effect and researchers using correlational evidence to argue e‐cigarette use increases smoking rates.
Ultimately, evidence shouldn’t be open to interpretation based on preconceived ideal outcomes, and the pair believes the solution is not to continue to accumulate more similar studies.
Consequently, they call for a different approach: “‘Open research’ practices, in particular, where some or all elements of a research work‐flow are made publicly available, is one area where there has been a great deal of movement and innovation in recent years. This includes pre‐registration of study protocols, sharing of study materials, posting of study data and so on. This transparency can improve quality (for example, by requiring good data curation as well as increasing the level of checking that takes place before data are made public).”
They conclude: “Given the critical importance of understanding the epidemiology of vaping, and its possible impacts on tobacco use, together with the strong feelings that have built up within the tobacco control and research community, adoption of such open science practices is an urgent priority.”