Our series of posts for Open Access week kicks off today with a post from Marcus Munafò, Professor of Biological Psychology at the University of Bristol.
Open access publishing is an established and familiar part of academia, but other aspects of open science are less widely appreciated. Open science is an umbrella term describing efforts to make various aspects of the scientific research process accessible to anybody. This can include publications (i.e., the open access part of open science), but also study materials (open materials) and data (open data). The Facilitate Open Science Training for European Research (FOSTER) project has developed a full taxonomy of open science.
Over the last few years, my group, the Tobacco and Alcohol Research Group, part of the MRC Integrative Epidemiology Unit at the University of Bristol and the UK Centre for Tobacco and Alcohol Studies, has been gradually moving towards an open science model. Since we receive Research Council and Wellcome Trust funding we have been making our publications open access for some time. We also pre-register studies that involve collection of new data on the Open Science Framework, and share the resulting data via the University of Bristol Research Data Repository.
Why do we do this? One important reason is that we receive most of our research funding from either Research Councils (i.e., public money) or charities (such as the Wellcome Trust), and therefore have an obligation to be as transparent as possible, and also to maximize the efficiency and reach of our research output by enabling others to use our materials or data. But there’s another reason – we believe it improves the quality of our work, by introducing additional quality control procedures into our research pipeline.
There’s a great deal of interest in research reproducibility at the moment, particularly in the behavioural and biomedical sciences. We would hope that the majority of published research findings would be reproducible – in other words that a similar study would obtain similar results. However, there is growing evidence that this may not be the case. A recent attempt to replicate 100 psychology studies (the Reproducibility Project: Psychology) found that in only about 40% of cases did the replication study produce the same result as the original study.
How does open science improve research reproducibility? In our opinion, in a number of small ways, all of which may add up to a substantial improvement. For example, pre-registering study protocols (i.e., open materials) means we have to think hard about exactly what we’re expecting our study to show, and can’t move the goalposts after the data are in (a process known as HARKing – hypothesizing after the results are known). In principle, anybody could check what we report in our eventual published article against what we said we’d do in the first place.
There are other benefits. Making the data underlying our analyses available (open data) means that (again, in principle) anybody could check to see that they obtain the same results as us, or use a different analytical method that they think is more appropriate. This in itself is valuable, but it has resulted in unexpected benefits – researchers check their data files even more carefully when they know they’ll be available for public scrutiny, checking things just one more time to be certain. This introduces an extra level of quality control to the research process.
Of course, researchers are trained to be careful and meticulous. But researchers are also under enormous pressure, to publish articles and obtain grants, as well as often having to deliver teaching, complete administrative roles, and so on. Science is a human endeavour, and honest error is inevitable (witness typographical errors in published manuscripts, despite multiple rounds of peer review, proof reading, and so on). To protect against this, we need to embed processes in our work that help catch these errors. Open science has helped us to do this.
It’s easy to underestimate how much work is involved in adopting an open science model. For example, if we want to make the data we collection open, we need to ensure that participants in our studies have consented to this. As a result, we have had to modify our informed consent procedures, in consultation with our Faculty research ethics committee. The process has been a gradual one, and is still ongoing – we are now working to make our study materials open, beginning with our laboratory handbook. We feel the effort is worth it for the benefits.
Marcus Munafò
Professor of Biological Psychology
Tobacco and Alcohol Research Group
MRC Integrative Epidemiology Unit
School of Experimental Psychology
UK Centre for Tobacco and Alcohol Studies