Think about the last time you purchased something like a new tablet or the latest phone. Did you notice the little symbols on the packaging? These symbols are there to indicate that the item has met certain standards and you can use it with confidence. Peer review aims to perform the same function for published academic research so that anyone reading it can be confident that the claims made are sound.
Typically this review is carried out by experts in the field who will independently verify the research underpinning the publication including looking at areas such as methodology and results. This process helps to ensure academic integrity, both for the journal publishing the research and the wider field of research.
Peer review is not a linear process and may involve several cycles of activity. An author will submit their work to a journal for consideration. The editor will assess whether the work is suitable for publication in their title and forward it to one or more experts for review. These peer reviewers will look at the work, check the science and how it reads as a piece of communication and then return their verdict to the editor who will make a decision. If the reviewers have asked for major or minor revisions of a work the author can choose to complete these and then resubmit. This process may happen several times before a journal agrees to publish the final output.
This guide will highlight some of the fundamentals of peer review and show you how to get involved in the process as either a reviewer or a reviewee.
There are many different models of peer review in operation. Exactly which model a journal operates will depend on the publisher.
Both the editor and the reviewer know the identity of the author but the author does not know the identity of the reviewer. This allows the reviewer to build on their knowledge of the author's previous work whilst giving them freedom to be honest in their comments. However, knowing the identity of the author may lead to conscious or unconscious bias.
Neither the author or the reviewer know the identity of the other but the editor knows the identity of both. This helps to protect against any potential reviewer bias whilst still giving them the anonymity to make their comments. However, there may be some identifying elements in the paper itself which the reviewer can pick up on. It also does nothing to address potential bias from the journal editor.
The editor, author and reviewer all have their identity hidden from the others and an editorial office takes responsibility for facilitating the process. This aims to ensure that the editor with decision making power over publication is not influenced by any potential bias. This model is quite complex to administer and it may still be possible to identify any parties through their work.
Under the open model all identities are known to all parties. This allows for a dialogue between all involved rather than a one way judgement. The transparent model builds on this by publishing the review alongside the final publication, adding to its developmental history. Whilst this approach can incentivise reviewers, some are concerned that the lack anonymity will prevent honest comment.
The current system of peer review is far from perfect and there are several issues which need to be addressed. The current system dates back to the 1950s and many argue that it is no longer suitable for a modern research environment. The video below outlines some of the concerns around peer review.
Peer reviewers are a vital part of the scholarly research ecosystem but what are you actually expected to do? The aim of peer review is to assess the suitability of an output for publication but there are many roles for the reviewer in this process.
Any journal title reviewers work for should provide guidance on what they expect including the length of the review, what to include and which parts of the review are visible to authors. Some titles may offer a form with multiple questions whilst others will provide a single box. Reviewers should always check the guidance from the title they are compiling the review for to avoid problems. It is also a good idea to check the scope of the review - are they reviewing an article in isolation or are they also commenting on accompanying material such as datasets?
Once reviewers understand the scope of the review the best thing to do is to read the article through at least once first. This will allow them to get a feel for the research and the writing and to check that they have all the materials they need to review.
Although each title is different, some common areas to include in a review are highlighted below.
The paper should clearly state its research question and aims so that readers understand what the investigations are working towards. Check that any objectives have been outlined and crucially, addressed in the discussion and conclusion. It is important to ensure that the question and objectives are made clear as much of the paper is built around them.
The paper should outline the methods used including justifying why these were selected for this piece of research. Consider whether these methods are actually appropriate for the question(s) that the research is trying to address.
Reporting the results of the research is likely to take up a substantial amount of the paper. Results should be clearly presented, understandable and address the research question(s). Authors may use charts, graphs or tables to communicate results. These should enhance understanding of the material, be clearly labelled and necessary to the report.
It is important to consider the paper as a both a report of research and a piece of writing. The paper title should give an indication of its topic. Humorous titles may be acceptable but this depends on both the journal title and the conventions of the discipline. The abstract should clearly summarise the topic, main findings and conclusions of the research.
Reviewers should also look at the narrative of the paper. Does it flow and is it readable as a piece of writing? It is also important to consider the quality of the language used to ensure that it meets the standard for publication. Whilst it is not the job of the reviewer to edit the piece, they should comment on the clarity of the language used and whether it is an appropriate level for the audience of the journal. Remember that authors may not be writing in their first language. Reviewers need to look beyond this to the quality of the underlying research itself.
Reviews may be asked to comment on the breadth of existing literature used to support the paper, for example in a literature review. This includes areas such as whether key materials have been acknowledged, a range of viewpoints are represented and the currency of the literature referenced. Remember that researchers in some institutions may not have easy access to the latest work in their field. The paper should contain links between the theory discussed in the literature and the outcomes of the current project.
Reviews should also check that any references have been formatted using the correct style. It is not the job of the reviewer to correct these if they are wrong but if there are consistent errors in referencing then they should note that corrections will be needed.
Reviewers should record their overall impressions of the paper, including reflections on both the standard of the underlying research and the quality of the writing. Is the research original and something the readership is likely to be interested in? Does it bring a fresh perspective to existing issues? Be aware of cultural issues here and remember that different countries will be at different stages with their implementation of initiatives. Reviewers should assess whether the paper contributes to the wider discussion of an issue and adds depth to the scholarly conversation.
It is important to provide feedback to the author in a review so that they can improve both this submission and their general standard of writing. Reviewers should offer constructive suggestions which the authors can use to amend their paper. It is also important to comment on any areas that were done particularly well. This helps to produce a balanced review and encourages authors to continue productive behaviours.
Depending on the format of the review submission, this may be the only part of the review that authors have access to. Reviewers should present a balanced and constructive review which summarises all of the elements listed above.
The final element of the review should be a recommendation to the editor on whether they should proceed with publication. Typically this recommendation will be one of four options:
When compiling your review it is important to remember that your role is to provide constructive criticism. This is feedback which is designed to improve something. It offers comments which are clear, actionable and beneficial to the recipient rather than just a list of things they have done wrong.
Always remember that there is a person at the other end of the review and avoid making personal comments. If you find a mistake or disagree with something say so respectfully. Using statements such as "I think..." and "I feel..." helps to take the focus away from the author. Any comment you make should be developmental. Offer suggestions as to how the author might make improvements to their work. Instead of making general statements, focus on specific areas where authors need to improve. Rather than "the results section needs work" detail exactly how you think this should be done. Remember: this is about helping the author improve their work, not having them rewrite the work in the manner you would have written it. And don't forget to include positive comments. It is important to let the author know if they have done something well or if you particularly liked their approach. This will help to encourage the continuation of useful behaviours.
Peer review can be a very rewarding task and a chance to help develop both yourself and your colleagues. If you are new to reviewing and want to get involved, try some of the strategies below:
Good peer reviews are valuable intellectual outputs in their own right. Increasingly this previously hidden labour is facing calls for recognition and a number of schemes are starting to appear. Open and transparent peer review models will allow you to publish your name alongside your review but this is not a model that many traditional journals operate. If you are undertaking any type of anonymised review you can sign up for a service such as Clarivate (formally Publons) which allows you to record your verified reviews on your profile without sharing their content. Alternatively you can add a list of journal titles you review for to your existing online presence.
Reading the information above, do you think the following statements are TRUE or FALSE. Click on the statement to reveal the answer
You have to be a senior researcher to undertake peer review.
FALSE. Anyone with the knowledge, time and commitment to review is able to. It is important that the review is honest with their editor about their level of knowledge and expertise but career stage is not a barrier to reviewing
A big problem with the current review system is the lack of consistency.
TRUE. Human judgement means that comments and decisions from reviewers can be inconsistent. This can further delay publication whilst a consensus is reached
Reviewers are expected to copyedit the work they are reviewing
FALSE. Reviewers are there to look at the science behind the paper and check that any claims made are supported by the evidence. Reviewers may find that they need to make a comment on the general standard of the writing but editing should be left to the copyeditors.
Undertaking peer review is a great way to increase the citations of reviewers or their colleagues
FALSE. Reviewers should never encourage authors to cite the work of themselves or their colleagues unless it is clearly relevant to the research. Peer review is not a chance to artificially inflate citations and reviewers should behave ethically at all times.
It is easier than ever before to get recognition for review work
TRUE. No matter which model of review a publisher uses, the reviewer can get recognition for their work. This could be through signing up to a relevant scheme or simply by including a list of titles they review for on their online presence.
Peer review can seem complex but it is also a very rewarding activity to get involved in. It allows you to read a wide range of research and helps to improve your own scientific communication. There are many resources available to help you, some of which are detailed below.
Want to Know Moore? Check out the slides and notes from our session on Peer Review to learn all you need to know.