Assessing researchers with regard to Open Science
PRACTICAL GUIDE 12
Science controls itself
The scientific quality of scientific work is currently evaluated in two ways: on the one hand peers review papers before they are published, and on the other hand peers respond to papers after they are published, i.e. the research findings enter the scholarly discourse. This kind of inspection is reaching its limits because only citation frequency (e.g. the impact factor) is used for evaluation.
The desire to modify the evaluation process rests on the San Francisco Declaration of Research Assessment (DORA) of 2013 and its first recommendation: the use of bibliometrics should be avoided because it mostly employs journal impact factors for assessing researchers.
The new standards for assessing researchers, designed according to Open Science principles, must:
- promote the unimpeded dissemination of scientific output,
- consider all aspects of research activity.
And finally these new assessment standards must take Open Science into account by promoting the following new practices:
- the Open Access dissemination of results even if they are insignificant (see Worksheet 10)
- the publication of papers that reproduce experiments,
- the publication of a research notebook describing the development stages of a scientific approach,
- the online availability of a database with textual or iconographic sources,
- the submission of research protocols for reviewing experiments to render the research process transparent etc.
- science communication to scientists from other disciplines or non-scholarly stakeholders
Worth knowing – what is altmetrics?
Altmetric (plural: altmetrics) is composed of the words alternative and metric. Altmetrics in scholarly publishing are those indicators that try to identify and to quantify a range of online reactions to a scholarly publication as broadly as possible. They measure those actions that use (access and download) a document on the web, review and link it, discuss and like it on websites of scientific institutions or news portals, or reference it in web-based literature management systems such as Mendeley.
Worth knowing – what is an impact factor?
The impact factor provides information about how often articles in a journal are cited by other publications. For this, the reference lists of publications indexed in the Web of Science database are analysed. The number of citations for the publications in a journal within a year are counted. This number is divided by the number of articles published in the journal. The impact factor assumes that good papers are cited more often than inferior ones. The impact factor is calculated by the American company ISI (Institute for Scientific Information) and published by the Journal Citation Reports (JCR) database. Impact factors are calculated only for those journals indexed in the Web of Science database.
Good luck with your research!
Date: March 2021
Questions, comments and notes are welcome at open-science@zbw.eu