Open Science requires and fosters international networking

Professor Michael Kirchler talks about his experience with Open Science

Professor Kirchler, how do you see the role of Open Science in your field of research?

MK: For nearly 15 years now I have been active in Behavioural Economics, i.e. the study of human behaviour in situations of economic and financial decision-making. From the very beginning, Open Science has been important to me and my research group, even back in the days when the approach was less widespread. We have always added the instructions for the experiments to our papers, which wasn’t nearly as common back then as it is now.

For about four years now we have had a special research unit “Credence goods, incentives and behaviour” at Innsbruck University where we have embedded a metascience approach in research that we initiated in 2014. This means that we look at different questions of science about science. Science, after all, is also a credence good. On one side you have the producers of science, i.e. the authors, and on the other side the consumers, for instance the publishers, but also political decision makers and the general public. The distribution of information is asymmetric because the consumers know little about the development process and the reliability of the studies. Even if a paper has been accepted and has served as basis for political decisions, the decision makers still don’t know if the study is truly reliable. This means that asymmetric information persists to a certain extent even after consumption – and that’s what characterises a credence good.

How did you find an entry to the field of Open Science?

MK: Behavioural Economics is a comparatively new discipline. As early as 2005 we noticed that the questions were exciting, but the sample sizes were often very small and the statistical tests rather simple. When the Open Science wave started in psychology in 2013/14, we in the field of Behavioural Economics quickly found our way to metascience studies and replication studies. I am still highly fascinated by this discipline and I feel that much has changed for the better, with regard to statistical power, but also with regard to topics like pre-registration and pre-analysis plans. That is, researchers describe in advance the experimental protocol and the planned analyses, publish this information and then carry out the study, just as they would in a clinical study. These are very valuable approaches and they have increased dramatically in our field over the last few years. I think this is a very important point, that science can improve on itself.

What have been the reactions to your activities?

MK: At the very beginning, before we started our first big replication study in 2015, there were some critical voices who said this could have negative consequences for our reputation. But we didn’t really care because that is what science is all about: to carry out studies to the best of your knowledge and with the best methods available at the time. After the study was published, the feedback was very positive because the community itself wanted to know about the replication rate of 18 well-known studies and the influencing factors for this replication rate. Our advantage of course was that we replicated several studies, not a single one, because such a large-scale replication study cannot simply be “overlooked”.

So you will persevere with this approach?

MK: Yes, although we have moved away by now from pure replication studies and just started our second big crowd analysis project in empirical finance. For this, we took 652 futures transactions of Deutsche Börse and framed six hypotheses. Scientists from all over the world could register for data analysis. In all, 400 scientists in 230 teams have registered. Now they will deliver analyses for different phases and receive feedback from so-called referees who are also involved in the project. At the end we wil be able to analyse the heterogeneity of the findings for each hypothesis, if they have adjusted over time because of referees’ feedback or not, and which explanatory variables can affect the results for each hypothesis. We have just submitted the pre-analysis plan for this project.

Has your network been changed by these Open Science projects?

MK: The replication studies and the metascience studies have opened many doors for us. They have brought us into contact with research teams with whom we would not have cooperated otherwise. We have learned a lot from these groups, and hopefully they learned something from us. That’s the fascinating thing about Open Science: the structure of the studies itself requires networking, and as a rule it is also very fruitful.

I believe it must be the incentive for all of us that we can do the best possible science. And with our metascience studies and many others we have achieved that reflection about scientific methodology has been intensified and that we do better science than before. I believe this is the most important thing about these metascience studies: we as scientists must realise how important it is that we work correctly and that our findings are truly robust. In the course of the pandemic we have seen the importance of scientific guidance and international networking for the creation of knowledge about the virus and vaccines. And that will be so much easier if we do science correctly.

The interview was conducted on January 15, 2021.

About Professor Michael Kirchler

Michael Kirchler is Professor of Finance and Speaker of the Special Research Group “Credence goods, incentives and behaviour” at Innsbruck University. He received the START prize, Austria’s most prestigious award for early career researchers, in 2012. He heads the newly established research centre “Innsbruck Decision Sciences”.

Contact: https://www.uibk.ac.at/ibf/team/kirchler.html.de

ORCID-ID: https://orcid.org/0000-0002-5416-2545

to Open Science Magazine