METM19 presentation
The peer review process is the gateway to scientific publication. It is a long-standing process that aims to provide a fair decision-making mechanism and improve the quality of manuscripts submitted to journals. Peer reviewers, pivotal actors in this process, are asked to write a report evaluating a manuscript and provide recommendations on how to improve, if needed, its quality. However, these reports are often of poor quality.
My research team and I are currently developing and validating a new tool to measure peer review report quality in biomedical research. Such a tool could be regularly used by journal editors to evaluate reviewers’ work. So far, we have identified the quality components of peer review reports by conducting a systematic review of the existing tools used in biomedical research. In addition, we surveyed international editors and authors to better understand their perspectives towards the quality of peer review reports and identified further quality components.
In the presentation, I will show a preliminary version of our own item-based tool and provide real examples from peer review reports with good or poor compliance with each item. I will also outline the steps planned to evaluate the psychometric properties of the tool.
Journal editors-in-chief will eventually be able to use the tool to train reviewers, assess their training programmes or evaluate and reward their best reviewers. Moreover, authors and authors’ editors or translators who attend this presentation can gain insight into behind-the-scenes decision-making processes and better understand how to work together throughout the peer review process.
About the presenter
Cecilia Superchi is a research fellow pursuing her PhD within the Methods in Research on Research (MiRoR) project, an innovative doctoral training programme in the field of clinical research funded by Marie Skłodowska-Curie Actions. She obtained a BSc degree in Biological Sciences at the University of Parma (Italy) and an MSc degree in Epidemiology and Public Health at Wageningen University (Netherlands). Before starting her PhD, she worked at the Iberoamerican Cochrane Centre in Spain. Her PhD focuses on how to measure the quality of peer review reports in health research.
Measuring the quality of peer review reports in biomedical research
Cecilia Superchi, Barcelona, SpainThe peer review process is the gateway to scientific publication. It is a long-standing process that aims to provide a fair decision-making mechanism and improve the quality of manuscripts submitted to journals. Peer reviewers, pivotal actors in this process, are asked to write a report evaluating a manuscript and provide recommendations on how to improve, if needed, its quality. However, these reports are often of poor quality.
My research team and I are currently developing and validating a new tool to measure peer review report quality in biomedical research. Such a tool could be regularly used by journal editors to evaluate reviewers’ work. So far, we have identified the quality components of peer review reports by conducting a systematic review of the existing tools used in biomedical research. In addition, we surveyed international editors and authors to better understand their perspectives towards the quality of peer review reports and identified further quality components.
In the presentation, I will show a preliminary version of our own item-based tool and provide real examples from peer review reports with good or poor compliance with each item. I will also outline the steps planned to evaluate the psychometric properties of the tool.
Journal editors-in-chief will eventually be able to use the tool to train reviewers, assess their training programmes or evaluate and reward their best reviewers. Moreover, authors and authors’ editors or translators who attend this presentation can gain insight into behind-the-scenes decision-making processes and better understand how to work together throughout the peer review process.
About the presenter
Cecilia Superchi is a research fellow pursuing her PhD within the Methods in Research on Research (MiRoR) project, an innovative doctoral training programme in the field of clinical research funded by Marie Skłodowska-Curie Actions. She obtained a BSc degree in Biological Sciences at the University of Parma (Italy) and an MSc degree in Epidemiology and Public Health at Wageningen University (Netherlands). Before starting her PhD, she worked at the Iberoamerican Cochrane Centre in Spain. Her PhD focuses on how to measure the quality of peer review reports in health research.
