Welcome to this module about the ways in which research can be measured.
Data, Metrics, Key Performance Indicators... these terms are everywhere these days, as we increasingly seek hard data to monitor and improve the quality of many of the things we do. Research metrics can be very useful, but they also come with important caveats, so we need to be responsible in how we use these tools.
In this module, you will learn:
Let’s start with the most commonly used research metrics. This video by Clair Castle at the Chemistry Library explains what they are and how they can be used.
There are many more types of research metrics, beyond the common ones discussed so far. If you are interested to know more Taylor and Francis have produced this quick and informative one-page guide.
If you want to dig deeper into metrics, the Metrics Toolkit has pages about a variety of metrics, with detailed explanations of how they are calculated and the appropriate ways of using them.
Metrics can be used to make comparisons at different levels. The diagram below shows some of these uses:
In her blog post clarifying conversations about metrics, Lizzie Gadd further explores some possible uses for metrics:
(Taken from The Blind and the Elephant: Bringing Clarity to our Conversations about Responsible Metrics by Lizzie Gadd)
However, metrics have been shown to have significant limitations.
The risks associated with using metrics irresponsibly increase as one moves from large to small scale and from understanding to incentivising or rewarding. For instance, it is probably acceptable to use a simple metric to investigate how patterns of research vary between countries, but not acceptable to use a single metric to determine the hiring of individual researchers.
So what is a better alternative?
Using a single, or a few measures to assess research quality would be unfair and create skewed incentives that could perversely decrease the quality of the research. However, metrics still have value when understood and used appropriately. This video by Claire Sewell at the Moore Library will give you an overview of what we mean by the term ‘responsible metrics’. If you prefer, you could find the same information in this handy guide.
If you are interested in knowing more about the potential impact of irresponsible metrics and the recommendations made by the Independent Review of the Role of Metrics in Research Assessment and Management, you can read the Metric Tide report
In 2019 the University of Cambridge signed the San Francisco Declaration On Research Assessment (DORA), committing to a responsible use of metrics in assessing research. Find out more about what this means through the slides below and on the DORA website.
There is much to explore in the wonderful world of metrics. We have collected some of the most informative links below if you want to read further on any aspect of bibliometics, Altmetrics or responsible metrics.
The Metrics Toolkit contains lots of useful information on how traditional bibliometrics are calculated and how they can be used appropriately.
Learn more about Altmetrics (alternative metrics) including how they are calculated and how to see the results on the Altmetric website. You can also see the metrics for your own research using the University of Cambridge login option.
The University of Cambridge has committed to the principles of DORA - the San Francisco Declaration on Research Assessment. You can read the document for yourself on the DORA website.