Research metrics
Welcome to this module about the ways in which research can be measured.
Data, Metrics, Key Performance Indicators... these terms are everywhere these days, as we increasingly seek hard data to monitor and improve the quality of many of the things we do. Research metrics can be very useful, but they also come with important caveats, so we need to be responsible in how we use these tools.
In this module, you will learn:
Types of metrics
Let’s start with the most commonly used research metrics. This video by Clair Castle at the Chemistry Library explains what they are and how they can be used.
To put this into practice, choose one article relevant to your work and follow these steps. Answers to the questions can be found at the bottom of this page. If you would like a breakdown of the steps for this activity, here is a technical guidance document
There are many more types of research metrics, beyond the common ones discussed so far. If you are interested to know more Taylor and Francis have produced this quick and informative one-page guide.
If you want to dig deeper into metrics, the Metrics Toolkit has pages about a variety of metrics, with detailed explanations of how they are calculated and the appropriate ways of using them.
Uses and limitations of metrics
Metrics can be used to make comparisons at different levels.
In her blog post clarifying conversations about metrics, Lizzie Gadd identifies six possible uses for metrics.
Measure to understand. “Science of science” activities that study publication patterns and trends for the sole purpose of understanding them better.
Measure to show off. “Pick me!” activities. The use of metrics to market an individual, group or university on promotional materials or grant applications.
Measure to monitor. Plotting progress against an objective whether internally or externally set. This may include some comparison activity as outlined below.
Measure to compare. The use of indicators to compare one entity with another. University rankings are an example of this.
Measure to incentivise. The use of indicators to incentivise certain behaviours (recognising that once you start to measure anything it can act as an incentive by default). The measurement of open access content submitted to REF is an example of this.
Measure to reward. Any activity that results in some kind of reward for the entity being measured, be this a job, promotion, grant, prize or award of any description.
However, metrics have significant limitations. This video by Thomson-Reuters highlights some of the ways in which citation numbers can be misleading.
The risks associated with using metrics irresponsibly increase as one moves from large to small scale and from understanding to incentivising or rewarding. For instance, it is probably acceptable to use a simple metric to investigate how patterns of research vary between countries, but not acceptable to use a single metric to determine the hiring of individual researchers.
So what is a better alternative?
Responsible metrics
Using a single, or a few measures to assess research quality would be unfair and create skewed incentives that could perversely decrease the quality of the research. However, metrics still have value when understood and used appropriately. This video by Claire Sewell at the Moore Library will give you an overview of what we mean by the term ‘responsible metrics’. If you prefer, you could find the same information in this handy guide.
If you are interested in knowing more about the potential impact of irresponsible metrics and the recommendations made by the Independent Review of the Role of Metrics in Research Assessment and Management, you can read the Metric Tide report
In 2019 the University of Cambridge signed the San Francisco Declaration On Research Assessment (DORA), committing to a responsible use of metrics in assessing research. Find out more about what this means through the slides below and on the DORA website.
Answers
Types of metrics
You researched metrics for an article, journal and author. Here are the answers to the questions.