Open Science promotes the quality of research

Oliver Genschow on his open science experiences

Was there a decisive experience that sensitised you to the topic of Open Science?

OG: Yes, that happened towards the end of my doctorate in 2012, when the so-called Stapel-Fall began. This was definitely a very formative experience.

Can you please briefly explain what the Stapel case is about?

OG: The Stapel case refers to the social psychologist Diederik Stapel, who was later convicted of data fraud. This led to considerable discussion in social psychology and triggered a great deal of controversy. Later, other cases involving scientists such as Dirk Smeesters from the Netherlands and Lawrence Sanna from the USA also came to light. This was my first sensitisation to the fact that there might be irregularities in scientific practice. I was unsettled and considered whether I wanted to stay in science. Then, in 2015, a large research consortium, the Open Science Collaboration, published an article that attempted to replicate findings in the fields of social and cognitive psychology. The authors of the article were only able to successfully replicate about a third to a half of the original results. This definitely caused a shake-up in the field and for me personally and showed what the current situation in our field of research looks like.

And when did you make a clear decision in favour of Open Science?

OG: This happened during my postdoc phase, when I became increasingly aware of the topic of open science at conferences and panel discussions. It became increasingly clear to me that previous research practice in our field was not sufficient to uncover reliable findings. I then decided to implement Open Science methods more strongly in my day-to-day research. During my postdoc phase, however, the scientific environment was not yet very immersed in the subject matter of Open Science. Therefore, it took me until 2017, when I took up a junior professorship at the University of Cologne, to fully integrate Open Science methods into my daily workflow. At the research centre in Cologne – the Social Cognition Center Cologne – many researchers were already practising the principles of Open Science, which made it easier for me to implement them myself.

Looking at your previous writings, I see that you conduct preregistrations and publish Registered Reports. Let’s take a closer look at certain practices, perhaps starting with preregistrations. What has been your experience with this and what have been the benefits? How has the scientific community reacted to this?

OG: Well, I think one advantage of preregistration, even if it sounds trivial, is that you think more intensively about your research in advance. This has improved my research in some ways, as we now think even more intensively about the research design, the expected effect and the analyses before collecting the data. As a result, we sometimes notice things that we wouldn’t have noticed otherwise. This is definitely a practical improvement.

I can’t really judge the feedback from the scientific community, as I haven’t received much feedback on my open science practices so far. What I have noticed, however, is that those who have jumped on the Open Science bandwagon later and have not preregistered their studies have had difficulties publishing their research. In social psychology, it is now difficult to publish research in prestigious journals if it is not preregistered. I would therefore say that it is more of a disadvantage if you don’t preregister your research than an advantage if you preregister your own research.

You have already mentioned that you publish your research data. Do you use online platforms such as Github, RunMyCode, Open Science Framework or similar to share research data and scripts?

OG: I mainly use the Open Science Framework.

Have you noticed any personal benefits after publishing your data?

OG: Yes, actually. I still remember the first time I made data publicly accessible. This process was quite nerve-wracking. But this nervousness led me to check my data even more closely and scrutinise it more critically for errors.

Publishing data also helps me to prepare data and analysis scripts from the outset in such a way that they can be understood and used by others. This also gives you a better overview of the data yourself. If I then receive enquiries about the use of data, whether for a meta-analysis or other purposes, I can simply share a link with all the information. So you can say that if you prepare and share data for the public, you ultimately save work and time. This is not only true for the researchers who provide data, but also for external data users. So it makes work much easier if you can simply go to the Open Science Framework to look at data or perform re-analyses.

Do you carry out replication studies and have you seen any benefits for both yourself and the scientific community?

OG: Yes, we carry out many replication studies, some of them as registered reports. I see many advantages in this for both myself and the scientific community. One of the main benefits is that by conducting replications, I have developed a better understanding of research methods and designs. I have gained a better sense of which methods can be successful and which cannot. This also means that I can better predict which results are replicable and which are not.
At the same time, replications are of great importance for day-to-day research work, as they help us to determine whether a particular result is robust and reliable. This is an aspect of research that should not be neglected. I used to think that conducting multiple studies with small sample sizes was a good idea. I thought that if one study didn’t work out, no resources would be wasted. At the same time, I thought that if there was a positive effect with a small sample, it could be assumed that the effect could also be replicated with a large sample. I have since moved away from this approach. I now prefer to directly conduct a high-powered sample with many subjects in order to obtain the most precise estimate of the effect size. Replications are extremely helpful and informative in this context.

I also believe that replications are of great importance for the entire field of research in order to determine which theories and results are reliable and which are not. We carry out replications both of studies published by other researchers and of our own findings. For example, we recently published an article in which we used a slightly different paradigm to question our own previously published results due to failed replications.

Let’s talk about the topic of science communication. You are one of the editors of the online magazine “In-Mind”, and you have already received several awards for it. It’s not just you who publish your research findings in this magazine, but it’s mainly overview and review articles on various research topics. What is the response to this type of scientific communication? Have you experienced any positive effects on your own work, either from other disciplines or from non-scientific stakeholders?

OG: We typically publish review articles on various research topics at “In-Mind”. Our authors write for a broad audience, especially for laypeople. All magazine articles go through a peer-review process in which experts, and sometimes journalists, ensure that the articles are both scientifically sound and understandable for laypeople.

My work at In-Mind mainly involves coordination and organisational tasks. However, I also write some magazine and blog articles myself. From time to time, I am approached about these articles or receive interview requests from journalists who contact me because of my articles. For example, during the coronavirus pandemic, I wrote an article about the scarcity of consumer goods, which led to me receiving more media enquiries on this topic.

However, the strongest feedback we receive is not for individual articles, but for the work we do as an entire editorial team. We are increasingly receiving positive feedback for our achievements in science communication. However, it was not always the case that our work was appreciated by the scientific community. Especially in our early days, the scientific community sometimes doubted the usefulness and importance of psychological science communication. Fortunately, our work and our contribution to the field is now perceived very positively.

Do you have the impression that your research topics have become more visible through your science communication or through the In Mind magazine? Your research area doesn’t seem to be a niche topic and your work is probably understandable to many, right?

OG: I actually see my area of research as rather niche. This also applies to my research on imitation. In a way, many people are familiar with the phenomenon that people automatically imitate each other. At the same time, many of the studies we conduct in this area are pure basic research that is not necessarily relevant or of interest to most people. In addition, there are not as many researchers working on these issues worldwide, which means that not as many articles are published on the topic compared to other research areas. My research on belief in free will or consumer behaviour is somewhat more mainstream. Nevertheless, even with these research topics, some of my detailed questions are quite specific and niche. That’s why I haven’t published as many in-mind articles on my own research so far.

Would you say that the main aim of the magazine is to raise public awareness of psychological research?

OG: That is exactly our main goal. We want to offer a platform where psychology researchers can make their work understandable to a broad audience. The challenge in psychology in particular is that many people find the subject exciting. At the same time, there are many offers on psychology that are not scientifically sound. If you browse through the media or search for psychological topics online, you will come across many dubious offers and myths that do not stand up to empirical scrutiny. Even well-known magazines have psychology sections in which interesting but not scientifically sound results are often presented. Often these are not even genuine research results, but individual opinions that are then presented as universally valid laws. This gives society a false image of psychology. This has always bothered us, which is why we decided to found the magazine. Our aim is to present reliable research results in a way that is understandable to a wide audience. That is why we also use a peer review process.

As far as the distribution of our articles is concerned, we receive enquiries from a wide variety of sources – whether from publishers of school textbooks or from other media. Typically, we are asked whether our articles can be reused. As we are interested in the dissemination of serious science, all In-Mind articles have a Creative Commons licence that allows anyone to use and redistribute our articles. Some media outlets that reuse our content voluntarily donate to us in return to support our work.

You are very active in the field of Open Science. Do you have any tips for scientists who have no previous experience with Open Science and should pay particular attention when getting started?

OG: That’s a good question. My advice is probably not to be afraid. I often hear from researchers who have not yet had any experience with open science that they have concerns about publishing data or are worried about copyright issues and anonymity. In practice, however, cases where copyright and anonymity concerns stand in the way of publishing data are extremely rare.

I also often hear that the implementation of Open Science measures means a lot of work. My response to these concerns is always that the implementation of Open Science ultimately means less work. This can be nicely illustrated using the example of preregistration: Sooner or later you have to think about your own hypotheses, the number of test subjects and the research design anyway. By carrying out a preregistration, you are likely to think about these aspects even more intensively, which can ultimately lead to shortcomings in the methodology and analysis strategy being recognised before the data are collected. This improves the quality of a study, which ultimately saves resources and time. The same applies in principle to the publication of data and materials. At least I can say that I now organise my documents much better right from the start because I know that I will make everything publicly accessible later. Better organisation saves me time and work. In addition, I can easily refer to the public link when I have questions about data and materials.

Another concern that I often hear is the worry that interesting effects that have not been preregistered will become unpublishable. This is a misconception. As long as a clear distinction is made between hypothesis-driven and exploratory data analysis, non-preregistered analyses and findings can also be published.

In principle, I am of the opinion that preregistrations and the publication of data and materials have no negative effects in the vast majority of cases. In most cases, the advantages outweigh the disadvantages. Finally, I would advise all those who have reservations to simply give it a try. I assume that you will then realise relatively quickly that most of the original concerns do not apply.

In addition to the personal benefits of open science, you ultimately do the entire research community a valuable service by maintaining transparency in science. I have the feeling that through Open Science practices, the quality of research increases and the individual findings become more reliable.

Yes, okay, how do you assess the future significance of Open Science based on your experience?

OG: I believe that the Open Science movement is unstoppable. I actually see little to no real disadvantages in the implementation of Open Science. Ultimately, the implementation of Open Science brings benefits for the entire scientific community as well as for each individual.

There are also studies, particularly from psychology and social psychology, which show that the replication rate of findings has increased since the increased commitment to Open Science. I therefore believe that Open Science will sooner or later play an important role in all disciplines in which the Open Science principles can be implemented.

Do you have concrete plans for further personal involvement in Open Science?

OG: I have already firmly integrated Open Science into my teaching. Each of my courses includes an Open Science block at the beginning of the semester. We first look at the reasons for the replication crisis in social psychology and then read articles that propose solutions for replicable science. Based on the articles we read, we will then work out characteristics of good scientific work. We then use these characteristics later in the course to assess how robust individual study results are. As a result, students are not only informed about the topic of open science, but are also able to realistically assess scientific findings. This approach is received positively and with great interest by students.

I am also currently working on an application for third-party funding in which we are proposing a research project on the replication of classic findings in psychology. The background to this research idea is the current development in the field. In my view, there are definitely negative aspects that have been triggered by the open science movement. For example, the replication crisis in social psychology has led to an enormous increase in the number of test subjects to be surveyed. This increase is entirely justified, as the effects in psychology are often small and can only be detected with adequate sample sizes. At the same time, the increase in sample size has led to research areas based on social interaction and human contact being neglected. It is simply too costly to recruit hundreds of subjects for such time-consuming and complex studies. As a result, this type of research, which in my view is the core of social psychology, is increasingly being lost. This also applies to replications. Many replications of findings that are easy to realise in the laboratory or online are currently being carried out. However, the classic experiments from the 1960s, 1970s and 1980s, which often required social interaction, are hardly ever replicated. This creates a distorted picture of the replicability of social psychological findings. We have now begun to replicate a few of the classics and have found an astonishingly high replication rate. I would like to expand this research approach in the future, also because the classic findings are the basis for textbooks and thus the entire field.

If you were to answer the question “Open Science means for me or Open Science does, with me”, what would this sentence be?

OG: Open Science enables the generation of reliable results and promotes high-quality research.

Thank you very much!

The interview was conducted on 16 January 2024 by Dr Doreen Siegfried.

About Prof Dr Oliver Genschow:

Oliver Genschow is Professor of Cognitive, Social and Economic Psychology at Leuphana University Lüneburg. His current research focuses on imitation behaviour and the effects of the belief in free will. He is also interested in consumer behaviour. Oliver Genschow is committed to open science practices by preregistering his studies and making his data and materials publicly available. Oliver Genschow regularly publishes registered reports and is involved in an often forgotten area of the Open Science movement: Namely, the transparent communication of research findings in language suitable for laypersons. As one of four main editors of In-Mind Magazine, Oliver Genschow supports science communication by psychologists working in science.

Contact: https://www.leuphana.de/institute/imo/personen/oliver-genschow.html

ORCID-ID: https://orcid.org/0000-0001-6322-4392

ResearchGate: https://www.researchgate.net/profile/Oliver-Genschow

Open Science Framework: https://osf.io/u5vf2/

to Open Science Magazine