Why Open Science can be a future selling point

Three key learnings:

• Students intuitively understand the idea of Open Science. It is difficult to comprehend why Open Science is not yet standard in academic life.
• Once you have experienced the advantages of a code book or a data management plan in a project, you don’t want to miss this in the next project.
• Open Science engagement can increase the visibility within a scholarly community and thus create new opportunities for collaboration or projects.

In your opinion, what is the role of so-called future skills, such as the sorting of information, assessment of evidences, managing of data, in the education of economics students? Especially when bearing in mind that only a small share of the students will pursue a career in research?

DV: It is important to include these skills in teaching. Currently this happens rather implicitly. We talk about studies, we discuss findings, and we teach the students to do their own research. It would be even more important to include these skills explicitly in the curricula. I think these future skills are core competences which all graduates in economics ought to have at the end of their studies. In economic research we often work with data ourselves, and maybe this qualifies us to impart such knowledge to students.

What feedback do you get from students who do not aspire to a career in research after their studies?

DV: When we’re dealing with statistics or evaluations with statistical tools, there are some who ask me what they will need this for eventually. I always try to explain that such data-handling skills are in demand and important now. I also assume that data skills will be expected more and more often in the labour market and the demands on graduates will rise further in the future. We have more and more data available and we want to draw ever more benefit from them. The argument that it is an essential skill for future employment usually convinces them. What I also do: in my courses I use R instead of SPSS which students are often familiar with already. I always extol R with the argument that students won’t be staying in academia but work somewhere else instead, for instance in administrations or non-profit organisations. The chances of graduates coming across SPSS licences there are almost nil, because such organisations usually don’t buy them, certainly not for a single project. An open source tool like R, which you have on hand and can use even for a single project only, is another skill you can take to market. 

What is generally the role of teaching methodological skills in economics studies? Are you in Hamburg an exception with your research seminar that runs for two semesters?

DV: This is a special feature of the master programme “Interdisciplinary Public and Non-Profit Studies”. When it was created – before my time – it was expressly intended as a research-oriented programme. That’s why there is this double semester research seminar and a supplementary methodological course. The other master programmes have similar seminars but they run only for one semester. There was something similar in Potsdam during the time I did my PhD and also taught. There we offered a research seminar for master students as an elective course during one semester. That’s a real challenge, to organise all that into a single semester. If you really want to cover the entire research process and want to ask students to collect their own data and evaluate them, then one semester simply is too short. For one reason, because students have to rely on external participants who fill out questionnaires or take part in experiments and interviews.

What are the big insights you have seen in students regarding this whole area of Open Science?

DV: I have seen two different groups. The first are those who have finished their bachelor studies with a relatively strong focus on research and who therefore have some idea of how research processes work. They are aware that research is a comparatively long process and that at the end of this research process you have the papers. For them it is more surprising to learn that you can have preprints or additional material for the papers. The other group are students who have focused more on content and who are surprised that some things are not standard. That’s a bit like the general public who often wonder that openness and transparency aren’t standard. I often see this in my own private environment, too, where people are astonished that research data and analyses are not reviewed and not published.

Have you seen mental barriers in students where you had to argue a lot that Open Science is a good thing?

DV: Those who are really closely engaged in the research process ask the kind of specific questions you would also ask for a PhD thesis. “Isn’t it rather detrimental to have published an article as a preprint?” “What if there are errors in the code?” Those are rather advanced notions. But there’s no resistance. Students understand Open Science intuitively, for them it is more incomprehensible that it is not the standard.

How important is it to teach Open Science practices to research-oriented students?

DV: Once you have understood the idea of Open Science and know the different opportunitites to practice it, and see the benefits, you reach a point of no return. You can’t go back from there. I notice this in my own research: I started doing it because I thought that this is what research should look like. I think few do this because they see it as a selling point or a career benefit. They do it because they are convinced that it is the right path. Once you embark on it and learn the techniques, you can’t simply drop them in your next project; it would create a certain cognitive dissonance that you cannot live with very well as a researcher. I have had a case where we couldn’t share the project data because they were too sensitive. We considered alternatives and in the end we shared synthetic data so that you could at least understand the code. You can’t go back to a previous level after you have looked more closely at Open Science. Once you have experienced the advantages of a code book or a data management plan in a project, you don’t want to miss this in the next project.

Open Science trainings are usually singular events within the disciplines. Do you have exchanges with other lecturers on this topic? Do you have a network?

DV: At my own level of experience, I have colleagues of course with whom I have such exchanges, who also think it is exciting and who practice it. However, it’s my experience within my discipline that this was and often still is a topic for the younger ones. We who are interested exchange our ideas, but I can’t see yet that the topic is addressed for the entire discipline in a structured manner. As far as I can see, we don’t have a structured approach yet for training doctoral students in Open Science. The way I do it is to organise a colloquium for doctoral students three or four times per semester. The students or we ourselves present projects, as an idea or as finished papers. In this context I try to address Open Science topics. Some of the students have already come into contact with Open Science, have done preregistration or something like that. Generally there are more and more PhD candidates who have already heard about it. When I started on my thesis in 2011, I had never heard of it and I didn’t know anybody who had done it. It happened over time. Now it’s much more likely that doctoral students have come into contact with it already. That’s probably because we have quite a few psychologists among us who already know this from courses.

Where have you seen aha-moments among your students?

DV: If researchers share data and codes, other researchers or students can learn from it, copy things and test them. Many found this truly exciting. That was definitely an aha-moment. And many have learned about the option of making a preregistration for a student research project. I think that has been a thing that wouldn’t have occurred to students otherwise. We had to explain why we were doing that and why it is not only for the “big” researchers.

How important is a healthy skepticism towards other people’s research findings?

DV: Skepticism is very important and I try to include this firmly in my lectures. Not because you cannot trust others, but because science always is about critical questioning, i.e. approaching research in general critically. One example: at the beginning of my research seminar I asked students to select a paper, present it and discuss it critically. I always asked them how they would grade the paper and the students were always generous. They thought the papers were brilliant. But once you started discussing them and pointing out individual aspects, their skepticism grew. This also matters to me, because it is a competence that will be useful later in many respects. First of all professionally, of course, either in academia or in some other professional activity: you should be able to question data and results critically. But for me personally the societal component is even more important which has acquired new relevance with Covid-19 and has become evident. In my opinion it is extremely important that non-scientists are also able to weigh evidences against each other. Teaching things that go beyond the mere curriculum, and may even have a direct use in daily life, lies close to my heart. Evidence is something you meet with everywhere in life, even if you deal critically with science only in your private life.

What kind of feedback do you get for such research seminars?

DV: Normally, bachelor students don’t really come into contact with the problem of how the stuff they are reading is actually produced. Such a research seminar is therefore a great opportunity to understand, to recreate, and to critically question the research process and the challenges and obstacles connected with it. At the end there’s feedback from some who say that the new perspectives and the knowledge acquired have enriched them, even if they won’t use it later. Many students think it’s exciting and interesting to have a view of the entire research process.

Do today’s doctoral students constitute a new generation of researchers?

DV: It’s difficult to answer this because I don’t know what kind of a generation of researchers my predecessors have been. I just think the ideas of Open Science are more easily available and the likelihood of getting into contact with them is greater. I can imagine that someone who wrote his PhD thesis twenty or thirty years ago would have found many ideas interesting and attractive back then, if they had been around.

Do you have an idea where doctoral students today come into contact with it?

DV: I believe some of them have picked up on it during their bachelor studies. And if they haven’t, then it’s very likely they come across the topic at conferences or in papers. When I attended conferences as a PhD candidate, there were no preregistrations and no Open Data. If I go to a conference today there’s a good chance that I will come across a preregistered study or a paper with an Open Science badge on it. This option simply didn’t exist before.

When did you first come into contact with the entire Open Science ecosystem?

DV: There was a professor (a political scientist) during my bachelor studies who published data and codes on his website. I found that exciting. But I think I came to Open Science through Open Access, when I published a paper for the very first time. In those days you had to sign the copyright agreements by hand and send them by mail or fax. In that moment I thought: okay, I have signed away all rights, really all. Everything now belongs to the publisher, it’s just my name left on top. That was quite strange. I think after that I came into contact with the Open Access topic again and again, and the next step was preregistration.That was interesting because I thought this is how it should be done. Research should not be about fishing in your own data until you find significant evidence and then furnishing it with a theory. My wife is a psychologist and brought information about the replication crisis from her studies and that’s why I was aware of it. I actively started practising Open Science through exchange on social media. I’m rather active on Twitter and came across many interesting new approaches here. I have also tried out new things I learned about from others, such as Open Science Framework. That was towards the end of my doctoral studies, the beginning of my junior professorship, around five years ago, but then nothing more happened for a while. At some point Open Data and Open Materials showed up everywhere and I decided that I want to do this and to include the topic in my teaching.

Have you seen benefits for yourself from Open Science?

DV: Here I can give two examples: for one thing I have systematically uploaded postprints once the embargo had ended and advertised the articles on Twitter. I also wrote a blogpost about what you may do and not do with your postprint. I got lots of positive feedback on that from the community. The other thing is my most important paper so far, where my co-author and I used preregistration and Open Data excessively. We could publish this in a prestigious journal probably because we mentioned the aspects of openness as a selling point in the paper. The paper was accepted in 2019 when you could still use this as a good selling point. The novelty has probably worn off since then. Now, one or two years later it is still noticed favourably, but putting a paragraph praising yourself in a paper would be difficult.

Where have you published your postprints?

DV: At the very begining, when I was still at Potsdam University, they were uploaded into the repository of the university library after the applicable embargo. Nowadays many journals allow you to post the paper on your own website immediately after publication and to upload it into one of the repositories after 18 or 24 months. That’s why I post things first on my website and later, when the embargo has run out, my research can be found on SocArXiv.

Do you see Open Access citation benefits? That papers who are available in Open Access and easily accessible are cited more often than other articles?

DV: That’s difficult to assess in my case because I systematically use Open Access and therefore everything is available sooner or later.There’s no benchmark to measure it against. Articles written for practitioners have higher click rates, possibly, which makes sense because these are often read by people who don’t have access to academic libraries. I published my dissertation also as a monograph in Open Access because even back then I had this idea that it would be more successful if the work is freely available. Our university offered the opportunity to publish free of charge with the university press if the grades were good enough. I believe my dissertation has been disseminated much more widely in this way than if I had published it with a publishing house in a dissertation series.

Where do you publish research data?

DV: This has also changed over time. We published some of them on Dataverse, because the journals suggested it. Recently I have uploaded the data to Zenodo because it’s non-profit and supported by the EU. But I have to say that neither is nice to handle, I would like to have a much more intuitive handling there.

Have you also seen more visibility for yourself? Have you had requests for reuse or co-authoring?

DV: There are a few people in the community who practise Open Science forcefully. I think some people know my name as being part of this group. So people come up and talk to me or ask questions. There’s also a new journal in our discipline, created expressly to be Open Access and to bundle as many Open Science aspects as possible. That is Journal of Behavioral Public Administration. Because I actively practise Open Science I got involved and now I am responsible for communication, together with a colleague. It does mean a kind of added prestige for my career. All in all, the journal is exactly what I have in mind.  

Do you preregister yourself?

DV: Not for all projects, but whenever we test concrete hypotheses and have strong theoretical premises. We do that in Open Science Framework.

Do you also publish insignificant results, so-called null results?

DV: Yes, definitely. I published a study with a Danish colleague which is based on null results. But it was probably uncontroversial because it basically confirmed what most had probably hoped for. It addressed Non Response Bias, i.e. whether in studies about leadership we run into problems because some managers respond and others don’t. We combined our own data with those of others and didn’t find such effects in the end.

What are the three reasons for doing Open Science you would give to peers and doctoral or master students?

DV: My first argument would likely always be the idealistic argument. We scientists are interested in uncovering real connections and looking behind the surface. Many Open Science tools contribute to doing exactly this without deceiving yourself. Especially preregistration is a strong argument here. When talking to younger colleagues I would mention career as a second argument. It is my hope that Open Science aspects are becoming ever more important and that journals will require these practices more forcefully. The top journals in our discipline have taken steps in this direction lately, so that nowadays you must state if you share data. If yes, you have to say where they are available, and if not you have to give a reason. This is verified and I think the other journals will follow. So you have a competitive advantage if you are already familiar with this. I also believe that this will play a much larger role in appointment procedures in the years to come. You’re well advised to familiarise yourself with Open Science now and to open up your research.

Will the group of idealists who practise Open Science from intrinsic motives grow in future?

DV: I believe a priori that the group won’t grow larger. But those on the other side of the fence may change sides because the self-benefits will grow over time. If I start with a junior professorship today it constitutes a rather strong statement to say you know which criteria will appear in an appointment procedure five years from now. Will Open Science be among them? I wouldn’t take an oath on that. As we are seeing now, Open Science is almost always mentioned when applications for vacancies are invited. The first and the most important item is the publications list, there’s no way around that. But then there comes a point where several candidates are neck-to-neck, and where you can score points with additional competences. Good scientific practice and knowledge transfer will certainly matter then.

You are very much engaged with Open Science, you do science communication, you have this teaching concept. Where do you see your engagement in the future?

DV: I have had my mind on the reproducibility of findings for some time now. I’ve got two or three ideas how you could do this in the field. I’d like to look at the data myself on this. And then, in a longer timeframe, I’d like to exert more influence on journals and propel the recent, positive development. In our field we don’t have a journal offering Registered Reports, i.e. a two-stage review process. I have initiated first debates, but I would like to engage myself more.

Thank you!

The questions were asked by Dr Doreen Siegfried.

The interview was conducted on November 9, 2021.

About Professor Dominik Vogel

Dominik Vogel has been Junior Professor of Public Management at Hamburg University since 2016. His research focuses on leadership in the public sector, motivation and attitudes of employees, Human Resources Management, Performance Management and the interaction of citizens and administration. He regularly publishes his findings in international journals such as Journalof Public Administration Research and Theory or Public Administration Review.

Before he joined Hamburg University, Dominik Vogel worked as a PostDoc at the Chair of Public & Nonprofit Management of Potsdam University. Dominik Vogel wrote his dissertation on Leadership in the public sector. In 2020, he acquired his habilitation with “Essays on Motivation and Leadership in the Public Sector”.

Since 2019, Dominik Vogel has been editor of Journal of Behavioral Public Administration and a member of the Editorial Board at Public Administration Review and Review of Public Personnel Management.

Contact: https://www.wiso.uni-hamburg.de/fachbereich-sozoek/professuren/vogel-dominik/team/vogel-dominik.html

ORCID-ID: 0000-0002-0145-7956

Twitter: https://twitter.com/drdominikvogel

LinkedIn: https://www.linkedin.com/in/dominik-vogel/?originalSubdomain=de

ResearchGate: https://www.researchgate.net/profile/Dominik-Vogel