Open Science takes the stress out of PhD supervision

Professor Susann Fiedler talks about her experience with Open Science

What have been your positive Open Science experiences so far?

SF: Quite clearly: less stress. Both for me and the early career researchers in my team. A lot of feedback and thus many improvements happen before data is collected, i.e. before the moment where I can no longer change the results. We talk much more intensively about the effectivity of the research design, hypotheses, derivations and planned analyses. If you are going to write a preregistration, all these debates must take place before the deadline. The young researchers don’t have to be afraid of not knowing what to do with their data. A lot of uncertainty is removed from the system which is per se defined by uncertainty. I see this as a definite advantage. The preparation and documentation give structure to my own research and supervising that can be copied and adapted easily to other projects. And a paper is halfway finished before the data even arrive. None of my PhD candidates sits before a blank sheet of paper. It has taken a lot of stress out of my own work and out of supervising. It has also given me free space in which to think about stuff, because the rest is “well sorted”.

We know that a large number of business studies graduates do not pursue careers in academia. Can they profit from Open Science, too?

SF: Yes, definitely. The students are very interested in the design of these workflows. They know and understand incentive systems, and therefore can discuss them. Topics from science practice, such as confirmation bias and the forms of sunk cost fallacies, are a fascinating subject in companies, too. If we discuss an example, look critically at the papers, and consider whether it is a result or something generated by the system, it makes a lot of sense to the students as a case study. And along the way they learn how to handle evidence critically. To have understood the behavioural aspects of decisions and organisational structures offers ever bigger comparative advantages to students, because in the end it’s not all about the money, but also about other, social aspects. Students who have mastered this will be at the top of the field when it comes to designing smart organisations.

How optimistic are you that economics and business studies can develop to an Open Science level comparable to that of psychology?

SF: Open Science is not a trend. If we don’t come to an agreement on this we cannot go on. When I look at the path psychology has taken over the last few years, and what debates are going on in economics and business studies right now, I am very optimistic. However, I have to say that economics has engaged awkwardly in the debate about reproducibility with the Evaluating replicability of laboratory experiments in economics science paper published by Camerer, Dreber and others, because they have given an evaluation and communicated that reproducibility rates of 67 per cent are not bad and better than in psychology. In my opinion, social comparisons like that between differently designed studies are not helpful for a rational debate. Because this offers a fallback argument to all those who aren’t keen on replication studies. I think that’s a pity, because the paper could have had much more appeal and a bigger signal effect for economic research.

Why is Open Science so important for the interaction between economic policy and economic research?

SF: My own experience is this: when politicians don’t agree with an argument proposed by researchers, the policy makers draw back and argue that the research is not robust anyway. That’s a form of motivated reasoning, and it’s a problem. If I want to have evidence-based policy consulting it is enormously important to be able to prove that findings are robust and that data are valuable in themselves. Evidence must be more than a rhetorical device.

You are a researcher on decision-making. What would you recommend the Open Science movement to do to convince as many researchers as possible to opt for Open Science?

SF: People who haven’t had much exposure to Open Science until now need simple instructions with clear rules. Every type of reproducible science has its nuances, of course. But researchers must understand from the very first step what the principles of Open Science are. That’s the basis. So it’s about making decisions easy. For instance it’s helpful to explain the benefits and gains in quality from the word “go”. It is also very important to show that Open Science does not require additional efforts, but that it is merely a shifting of efforts within the working process.

When would you start instruction in Open Science?

SF: I would start as early as possible, with students and not just PhD candidates. Students must get used to a good scholarly use of data and where to use documentation. If students learn this from the very beginning, it will soon be easy for them and they won’t question the principle any more. The Open Science movement offers so many opportunities to include good scientific practice in teaching, because we discuss the truly important basics of science in so many different aspects: philosophy of science, scientific practice, ethical and epistemological findings. I think we can not only trust students with Open Science, they themselves demand it. Even people who won’t be working empirically later in life will be able to understand how to deal with evidence. Everybody needs to understand what constitutes good evidence.

What would you suggest to economic researchers who want to become part of the Open Science movement?

SF: The present sitution still is that not every university or economic association has a local Open Science initiative where a researcher can walk in and find information about the topic. We are still in the phase of “do it yourself”. If the topic interests you, if you want to take others with you, go and set up an initiative, organise a workshop, invite someone to give an overview, find out if there could be a demand for it and include it in PhD education. Because PhD candidates in their turn can take it into their tutorials and basic seminars. That is a lot of groundwork laid. And now let me do some advertising on my own behalf. In February 2021 we have founded the German Reproducibility Network. A network of networks. Local Open Science initiatives who are interested in exchange can register there. We want to be a point of contact for university policy, business, publishers and the scholarly associations and to change incentive structures through these bodies. In the long term the idea is to establish Open Science initiatives in all universities, working across and open to all disciplines. A list of active initiatives is available in NOSI – Network of Open Science Initiatives. We try to promote the topic with the German Reproducibility Network. We know that it is difficult for many to find contact persons so we are happy to provide contact info.

If you want to establish Open Science broadly and transform science sustainably, you need to change performance incentives and to reward sharing. Which university can serve as a model here?

SF: At an indivudal level, some universities are pioneers in this, Göttingen University for instance, or LMU Munich. In their job offers there’s an explicit passage stating that Open Science will be an aspect of evaluation and that applicants must provide an Open Science statement. Questions can be: In what way have you contributed to making your research open, transparent and accessible in the past? What are your ideas for the future? With this, universities are giving a signal that there is an evaluation criterion towards which I as a young academic should work. I think that’s great. In the long run it would be nice if transparency and reproducibility were seen as measurable criteria of quality for every researcher. Ideally there would be rewards for Open Science not only at an individual level for individual researchers, but also at the institutional level. Certificates could be used to reward Open Science at different levels. We know about certificates for equal opportunities for men and women (such as the TOTAL E-QUALITY award). Why not hand out certificates for reproducible research? A university could get rightfully decorated for that.

In a presentation you have said that we need more organised skepticism in the science system. How can organised skepticism be organised?

SF: We don’t need a devil’s advocate in our discussions who always finds something to criticise. I can’t be sure that there’s always such a person on the team. Instead we need a standard step in our processes where we critically question our assumptions together, where we question our interpretations. We must see this as a measure of quality assurance and not as criticism or bashing. Unfortunately our system deals badly with mistakes. Mistakes are simply a necessary part of the process and should be valued as such. We need to talk about mistakes and reflect on them. This must be a default setting of the system, so you can improve your work and find better solutions next time. That’s extremely hard for people to get their heads around if they’re only looking to avoid mistakes. That’s why some have such a hard time with transparency, they are afraid their mistakes will be found out. Mistakes are part of the job. Organised skepticism should be constructive.

Thank you!

The interview was conducted on March 29, 2021.

About Professor Susann Fiedler

Susann Fiedler is Professor of Business and Psychology at Vienna University of Economics and Business Studies in the Department of Strategy and Innovation. She is a behavourial scientist who gained a PhD in psychology from Erfurt University in 2013. From 2012 til 2021 she headed the Gielen-Leyendecker research group “Economic Cognition” at the Max Planck Institute for Research on Collective Goods. During the course of her academic career she has been guest researcher at the Edmond J. Safra Center for Ethics (Harvard University) and visiting professor at Hagen University. Her papers on the cognitive and affective bases of evaluation and action processes have been honoured with the Otto Hahn Medal of the Max Planck Society, among others.

Susann Fiedler studies questions of designing decision situations and organisational environments. Her research offers insights into the mental representation of situations and problems, and how such representations (e.g. in the case of biases) can be changed (e.g. to avoid misbehaviour in working surroundings).

Susann Fiedler is an advocate for Open Science and engaged in committee work addressing the reproducibility of research (e.g. within the German Reproducibility Network or the Ombuds committee for psychologists engaged in research).

Contact: https://www.coll.mpg.de/susann-fiedler

ORCID-ID: https://orcid.org/0000-0001-9337-2142

Twitter: https://twitter.com/susafiedler

LinkedIn: https://www.linkedin.com/in/susann-fiedler-89025015a/

to Open Science Magazine