Pre-registrations promote theoretical clarity from the outset

Tilman Fries on his experiences with Open Science

Copyright: Private

Was there a moment that sparked your interest in open science – when you particularly realised the importance of open science for yourself?

TF: I belong to a generation in which open science was already a prominent topic at the beginning of my doctoral studies. So I don’t know any different. There was no clear trigger for me. However, certain stages of my career have made its relevance clear. For example, I worked for some time at an institute that conducted economic research. There, I quickly realised how important it is to document your own work carefully, especially when working in a team. It is also noticeable when the work of others is not transparent or difficult to understand. These experiences made it clear to me that I need to document my own research steps in such a way that they remain understandable to me later on and that, in case of doubt, I can also explain them to others in a comprehensible way.

So you’ve had both positive and negative experiences with documentation in teamwork?

TF: It was never really negative, but it wasn’t consistently positive either. The situations that had the greatest impact on me were those in which I had to take over a project and didn’t understand what it was actually about at first. Such experiences show how much a lack of traceability complicates the work process. Another important impetus came from larger replication studies, such as those by Colin Camerer, Anna Dreber and others, published in Nature Human Behaviour in 2018. These studies systematically reviewed several social science experiments. The results appeared early in my doctoral studies and sparked intense discussions among us, just as we were beginning to conduct our first experiments. The central message of this work is not that experiments are fundamentally irre . Rather, it emphasises that certain statistical requirements must be met in order to conduct robust research. It is equally important not only to take individual experiments seriously, but also to pay attention to meta-studies. These debates had a strong influence on my scientific approach in 2017/18.

You deal extensively with incentives and motivation. To what extent do you think open science practices are changing incentive structures in terms of publications, reputation and collaboration? It is often said that questionable research practices are ultimately promoted by incentive systems. Are you observing a counter-trend?

TF: The strongest incentive system remains the publication system, in particular evaluation by anonymous reviewers and journals. What has changed significantly is pre-registration, which is now common practice in many studies. It creates incentives to think through your own research project very thoroughly before collecting data. In experimental research, which I work with a lot, this means that I have to determine and document in advance what data I want to collect, which hypotheses are plausible and what theoretical assumptions underlie them. This approach forces me to also consider scenarios in which the expected results do not materialise and to think about how I can deal with such findings. In the past, there was more leeway to react flexibly after data collection, for example by highlighting alternative correlations in the data material if the expected effect failed to materialise. Such approaches are much more difficult to justify today. I see the biggest change in the fact that publications are more closely linked to the original research plan.

When we talk about incentive structures, can we say that open science practices also address intrinsic motivation? In the sense of providing additional motivation to improve one’s own scientific work and make it more theoretically sound?

TF: I think I have to think much more theoretically about the mechanisms involved in my experiment. I always have to be my own devil’s advocate, because I’m not very flexible ex post. I hope that makes me a sharper theorist.

We just talked about pre-registrations. What other developments do you see in the field of open science that are particularly relevant to economic behavioural research?

TF: Meta-studies are a key development. They have had a major impact on the discourse because they sensitise researchers to the statistical challenges associated with experiments or data analysis. Terms such as “forking paths”, “p-hacking” and “statistical power” illustrate where potential pitfalls lie. Meta-studies show how great the influence of statistical randomness can be and how distortions, such as publication bias, shape the perception of results. Against this backdrop, individual studies lose some of their weight. Instead, systematic evidence and the aggregation of several studies are coming to the fore.

In addition, a separate meta-science complex has developed, reflecting on how science works and what social structures shape science. A example is a meta-experiment in which I myself participated. In it, several research teams worked on the same question. The aim was to highlight the diversity of research designs and to understand how different scientific questions can be approached. Such approaches are typical products of the open science movement: they aim not only to conduct research, but also to critically question its basic assumptions and practices.

In your opinion, what steps are still necessary in economics to make open science more mainstream?

TF: One key point is the review process. The crucial question is how open science practices such as pre-registrations are evaluated in reviews. At present, there is still a great deal of heterogeneity in this area. Some reviewers ignore pre-registration completely, which is problematic because it only gains credibility if it is actually taken into account in the review process. On the other hand, researchers also find that even small deviations from the original pre-registration can be heavily criticised. This makes authors vulnerable and can in turn lead to pre-registrations being perceived as a risk. The challenge, therefore, is to find a balanced approach. Reviewers should take pre-registrations seriously, but also consider them in relation to the results and scientific argumentation in the paper. A study should not be understood merely as a “completed pre-registration,” but should also allow room for scientific flexibility. Journal editors also need to develop an awareness of how to deal with deviations in a meaningful way.

What is your own experience with the question of how strongly pre-registrations constrain researchers and how flexibly deviations are handled?

TF: My main experience is that there is usually a pragmatic approach. Pre-registrations primarily increase traceability, and deviations can usually be justified. Problems only arise in exceptional cases, either when they are ignored in the review process or when they serve as a convenient excuse for criticism. It is important to distinguish between serious violations, such as retroactive data collection to generate significant effects, and comprehensible adjustments. Ultimately, a healthy, case-by-case assessment by reviewers is needed.

It is often said that fierce competition, whether in sport or science, creates incentives for doping or cheating. As a behavioural economist who researches lying, do you see parallels between your research findings and the incentives for questionable research practices?

TF: We know from research that some people lie, but many tell the truth. The situation is similar in science. For the vast majority of researchers, fraud would be unthinkable, simply because they are intrinsically motivated to understand the world. In this pursuit of knowledge, lying makes no sense because we want to get closer to the truth. Nevertheless, there are cases of manipulation, as we know from the past. An interesting insight from research on lying is the correlation between one’s own behaviour and expectations of others: those who do not lie themselves usually assume that others are honest. Those who lie are more likely to assume that others lie too. This could explain why scientific scandals cause such outrage. Many researchers are surprised because they simply cannot imagine such behaviour. It contradicts their fundamental values. There are no immediate concrete institutional consequences. But it illustrates why cases of fraud repeatedly shake the community to its core: they contradict the basic assumptions that most people work conscientiously and assume that others do the same.

There are observations that point to problematic structures in the scientific system. Publications are often strategically driven. The place where research is published is sometimes more important than the findings themselves. Added to this are phenomena such as “paper mills,” which pollute the scientific landscape with fake publications. The system creates enormous pressure. From your perspective as a researcher into lying, where do you see the critical points that may not be immediately obvious?

TF: I agree with you. Publishing is often highly strategic. Sometimes scientific studies operate in grey areas where results are not openly lied about, but rather reported selectively. My colleague Valeria Burdea is researching how people work with vague formulations. Many avoid deliberate lies, but are willing to remain vague if it allows them to create certain impressions. Applied to the publication process, this means that researchers omit tests they could perform, for example, instead of falsifying results. This selective reporting is less noticeable, but can still create a distorted picture. Such practices occur from time to time. It will probably never be possible to prevent them entirely. This is where open science comes in: if data and analyses are accessible, other researchers can verify results and test their robustness and replicability. This creates transparency and provides incentives to limit excessive strategic behaviour. But it is also clear that the pressure to publish remains a structural component of the system.

Under the chairmanship of Professor Klaus Schmidt, the Verein für Socialpolitik (German Economic Association) has put the topic of open science on its agenda. In your opinion, how much power do professional associations have when it comes to winning over an entire discipline to open science practices?

TF: Professional associations have several levers at their disposal. One important point is raising awareness. Professional associations can reach not only researchers who already use open science practices themselves, but also those who consume and evaluate research results. It is crucial for both groups to understand the purpose behind open science, what lies behind individual practices such as pre-registration or data disclosure, and why they are more widespread in some areas, such as experimental research, than in others. The second lever concerns the institutional level. Professional societies publish their own journals and can set specific standards there. One example is registered reports, in which research designs are submitted and reviewed before data collection. Such formats promote transparency and reduce publication bias. Professional societies can not only implement these developments in their journals, but also carry them forward in committees and panels, thus anchoring them within the entire discipline.

How do you perceive the current situation in the economics community? Is open science limited to career stages or sub-disciplines such as behavioural economics?

TF: My impressions come mainly from the experimental community, for example in the Economic Science Association, the umbrella organisation for experimental economists. There has been a significant push towards raising awareness in recent years. In my generation of researchers, it is now common practice to pre-register studies and carefully document codes. At least in this area, I see a positive development and broad acceptance of open science practices.

As a behavioural economist, do you think it is more effective to reward or sanction?

TF: I mainly see the effect of mild positive incentives. Pre-registrations are a good example. Ten to fifteen years ago, pre-registrations were considered unusual and were viewed positively. Standards have since shifted: today, it tends to be viewed negatively if a study has not been pre-registered. So we are seeing a regime shift from an “open science bonus” to a “lack of transparency penalty”. One area where I would like to see more positive incentives is outlets for unsuccessful or unexpected results. So far, there has been no functioning reward system in this area. This could only be achieved through targeted positive incentives.

By “unsuccessful,” do you mean insignificant results?

TF: Exactly, insignificant or unexpected results. This cannot be solved by sanctions, such as forcing researchers to publish every experiment. Positive incentives would be more effective: special outlets where short reports can be published. Something along the lines of: “I conducted this experiment, it didn’t work or it surprised me – here are the data and materials.”

Where have you personally experienced concrete benefits from open science practices – whether in pre-registrations, registered reports, meta-analyses or replication projects?

TF: A formative experience was a replication study at the Institute for Replication. I replicated a paper there using the data and code provided by the journal. In doing so, I realised how important it is to organise your own research in such a way that it is completely reproducible. The ideal workflow is one in which I can trace everything from the raw data to the tables in the paper with a single click. This not only helps others, but also myself. When I return after two months, I immediately know how to reproduce my results and can be sure that they have not been distorted by copy-paste errors or poorly documented code. Two experiences were particularly valuable in this regard: Firstly, I saw how well the authors of the replicated paper had worked and learned a lot from them. Secondly, I took a course with Hans-Martin von Gaudecker in my doctoral programme, which dealt with research organisation, i.e. clean programming, clear folder structures and comprehensible filing. That sounds trivial, but it was crucial for me. I still benefit from it today and am grateful because I can open a project at any time and immediately understand where everything is.

Have you had any other experiences?

TF: Yes, I was part of a many-design meta-experiment with around 100 authors. Working in such a large, international team was particularly exciting. Often, I didn’t know exactly how the others were working on their research questions during the process, so I learned a lot about different approaches. At the same time, such projects create valuable networks. I now meet colleagues at conferences, can make connections and have common points of reference. Especially for researchers at the beginning of their careers, such projects are an important opportunity to become part of the community and get in touch with people.

When you think back to the beginning of your doctoral studies, do you feel more connected to your scientific community today?

TF: Yes, definitely. This is mainly because we now talk much more about the research process itself. During your PhD, you make your first personal connections, but what really shapes you is learning how research is actually done, including from senior researchers. You don’t learn this just by reading papers. They only show the end product, not the path to get there. Open science discussions, on the other hand, reveal how established researchers actually work and what considerations lie behind the published results. This insight was an important learning experience for me during my doctoral studies. And I am very grateful that this discourse exists and has existed.

What advantages do economists who use open science practices have? Assuming you were talking to someone who is just getting started in the field, what would you recommend?

TF: One key tool is pre-registration. Its obvious advantage is transparency. However, another very practical advantage is that it allows you to identify your own mistakes at an early stage. Anyone who creates a pre-registration or a detailed pre-analysis plan has to ask themselves specific questions: What does my research approach actually allow? Which hypotheses can be tested and which cannot? Where are the statistical challenges?

Especially at the beginning, you are often full of enthusiasm and want to collect data very quickly. The danger here is that you skip important considerations. Pre-registrations force you to pause and clarify difficult questions in advance, such as the number of cases or the robustness of the design. I have repeatedly found that we have identified problems when writing a pre-registration and adapted our design accordingly. I would therefore strongly advise all young researchers to make use of this practice: it not only creates transparency, but also protects against avoidable mistakes.

Is there a generational difference in economics when it comes to open science? And how do you experience the interaction between older and younger researchers?

TF: Of course there is heterogeneity. Personally, however, I have had good experiences. Even senior researchers I have worked with were very positive about open science. My supervisor, Agne Kajackaite, for example, really pushed the topic in our collaboration. Basically, open science is often more natural for the younger generation, while more established researchers are sometimes more cautious. This is understandable insofar as they have worked without such practices for a long time and some research designs are so clearly structured that pre-registration does not seem necessary. Both sides can learn from each other: younger researchers should avoid approaching open science too dogmatically or perfectionistically. Overthinking can also be problematic. At the same time, greater transparency is an important goal for the entire discipline, from which everyone benefits.

As a behavioural economist, if you could design a new incentive system for science from scratch, what would it look like?

TF: For me, it would be crucial to incorporate mechanisms that ensure that we evaluate research more on the basis of the quality of the design and less on the basis of the results. One concrete tool for this is registered reports, which I would like to see become more widely accepted. The second problem is the enormous importance of individual publications. This often leads to publishing being done strategically. In economics, this is strongly linked to the dominance of a few top journals. Breaking up this monopoly would be an important step – even if it is much more difficult to implement.

Thank you very much!

*The interview was conducted on 20 August 2025 by Dr Doreen Siegfried.
This text was translated on 30 September 2025 using DeeplPro.

About Dr Tilman Fries:

Dr Tilman Fries is an assistant professor at Ludwig Maximilian University in Munich. He previously worked as a doctoral student at the Berlin Social Science Centre (WZB), where he completed his doctorate in economics at Humboldt University in Berlin in 2024. His research interests lie at the intersection of behavioural economics and experimental economic research, with a focus on expectation formation in strategic interactions and the interplay of expectations and preferences. Fries is a member of the Collaborative Research Centre 190 “Rationality and Competition” and an affiliate of the CESifo Research Network.

Contact: https://tilmanfries.github.io/

to Open Science Magazine