Acquire replication skills during your studies
Interview with Lars Vilhuber about Open Science and reproducibility in teaching
Photo: David Außerhofer
The integration of Open Science into university education is becoming increasingly important in economics. By teaching Open Science principles, methods and tools, teachers can better prepare their students for the requirements of transparent and collaborative research. But which methods work best? What experience already exists for the discipline? And who is the best person to discuss didactic issues with?
We interviewed Lars Vilhuber, who shares his experiences from the Replication Lab at the Labor Dynamics Institute at Cornell University. The economist, who has been working in the USA for many years, is Executive Director of the Labor Dynamics Institute at Cornell University and Data Editor of the American Economic Association.
Mr Vilhuber, you have been campaigning for reproducibility and Open Science in economics for years. How do you see the role of replication courses in student education, especially in economics?
LV: In my opinion, replication courses are an essential element in the education of future scientists and practitioners, especially in economics. Teaching replication methods promotes a deep understanding of how research results are obtained and how robust they are. Students should learn early on how important it is not only to accept research results, but also to critically scrutinise them and check their reliability. Reproducibility is a key prerequisite for scientific integrity. If we manage to firmly anchor these topics in the curricula, we will make a significant contribution to transparent, comprehensible and reliable science.
What content do you cover in your replication courses and how does it help students in their education?
LV: In my courses, we cover a wide range of content, all of which is crucial for sound replication expertise. The fundamental topics include the principles of data management and documentation – in other words, how data can be stored and processed in such a way that it can be understood and reused by others. Another important point is analysis: students learn how to structure analysis scripts and statistical methods in such a way that results are not only obtained once, but can be reproduced at any time. In addition, they learn how to write replication reports so that others can fully reproduce the work. The aim is to prepare students for the typical demands that await them in both academic and applied research.
What methodological approaches do you use in teaching to make replication a practical experience?
LV: Replication work thrives on practical relevance. In the Replication Lab at Cornell University’s Laboratory Dynamics Institute, students – mostly first-year students – work on real research publications and try to reproduce the results published there. This requires a thorough understanding of how data and scripts come together. Because the lab is for undergraduates, it does not go into depth when it comes to statistical methods. Nevertheless, the students develop not only technical expertise, but also a critical eye for methodological subtleties and possible weaknesses in the documentation. Through this practical experience, they learn how replication works in reality, what challenges it entails and how these can be overcome. When they later become active researchers – whether in academia or industry – they know very well what it means to have to explain their own work to others.
What are the biggest challenges for students when dealing with replications, and how do you support them?
LV: One of the biggest challenges is dealing with incompletely documented or difficult to access data and methods. There is often a lack of information on data preparation or there are inconsistencies in the published analysis scripts. Such hurdles are frustrating, but also a valuable learning experience. I support students by encouraging them to be analytical and creative in their approach: What assumptions can be made? Where are alternative paths possible? They also learn the importance of complete and precise documentation – a point that they can later apply to their own work.
The replication crisis is a major topic in many scientific disciplines. How do you see the role of replication courses in this context in economics?
LV: The replication crisis shows how urgently scientists need to have a sound knowledge of replication. Replication competence is an essential future skill, especially in economics. Replication courses make an important contribution here by sensitising students or pre-docs to the topics of replicability and transparency at an early stage. If young researchers understand the importance of clean and reproducible research at the start of their career, they are more inclined to implement these standards in their own work. In the long term, this can promote a cultural change in the scientific community that sustainably strengthens the quality and credibility of research results.
Open science and replication methods are now also a topic at international conferences. Which platforms do you use to share your experiences and methods and to further your education?
LV: The exchange with other researchers is very valuable for me. International conferences such as the American Economic Association (AEA), the European Economic Association (EEA), and many regional and thematic conferences, such as those on labour or macroeconomics, offer excellent opportunities to discuss the latest approaches with researchers. My fellow data editors and I have given talks and workshops at many such conferences in recent years and continue to do so. We also use platforms such as Github to publish best practices together with teaching materials, for example at https://social-science-data-editors.github.io/.
Thank you very much!
The interview was conducted by Dr Doreen Siegfried on 30 October 2024.
About Lars Vilhuber:
Lars Vilhuber, PhD, is Executive Director of the Labor Dynamics Institute at Cornell University and Senior Research Associate in the Department of Economics at Cornell University. He works with the Research and Methodology Directorate of the U.S. Census Bureau on a variety of projects. He also serves on several Data Access Centre Committees (CRDCN, CASD). He has been Data Editor of the American Economic Association since 2018. He is Executive Editor of the Journal of Privacy and Confidentiality. From 2026 to 2021, he was Chair of the Privacy and Confidentiality Committee of the American Statistical Association.
Contact: https://www.vilhuber.com/lars/
GitHub: https://github.com/larsvilhuber
To the Replication Lab: https://www.ilr.cornell.edu/labor-dynamics-institute
Podcast episode with Lars Vilhuber from August 2022: https://podcast.zbw.eu/fos/2022/08/18/fos-21-replikation-und-transparenz/
About his talk at the Open Science Education Coffee Lecture: https://www.youtube.com/watch?v=9pmyJ7Q32e4