Website displaying the latest Altmetric scores and outlining how these are calculated
This resource is licenced under a CC-BY-NC-SA 4.0 licence by Claire Sewell, the Office of Scholarly Communication, Cambridge University Libraries.
There are many types of metric which measure everything from an individual researcher to the wider academic system and new methods are being developed all the time. Below are outlines of some of the most common bibliometrics and how they are calculated together with an introduction to Alt or alternative metrics. Over time, librarians will develop a working knowledge of the measures which are popular in their discipline and with the researchers that they support.
Bibliometrics is the area of metrics that most librarians are likely to be familiar with. Although they were originally developed to help library staff assess the publications they wanted to subscribe to, they have evolved into a much wider method of assessment which is used at every level of the research process. From grant funders to hiring committees, bibliometrics are used to assess performance and in some circumstances to make quite important life decisions!
Bibliometrics are quantitative measures of impact which are used to rank people, institutions and their outputs. For example the higher the citations of a particular researcher or department the better it is perceived to be. Relying only on numbers to assess impact in this way can be dangerous (something which will be discussed later in the unit) but as part of a wider picture they do have value. As these metrics measure things in numbers they often rely on formulae which need to be calculated. When looking at these metrics it is important to keep in mind that they were developed in a scholarly world which relied heavily on printed outputs such as books and journals. Although some have been adapted to take this into account there are others which may seem a bit dated.
It is important for library staff to maintain an awareness of bibiliometrics as their research community are likely to rely heavily on them as a traditional method of assessment. The most common types of bibliometric and how they are calculated are outlined below:
Perhaps the bibliometric most librarians will have heard of, the Journal Impact Factor (JIF) calculates the number of citations received by a journal title over the last two years and uses an average to calculate its impact. It is easy to calculate (and therefore easy to verify) and is relatively simple to understand. However it is calculated using databases which are heavily science focused which may limit its use to other disciplines.
Devised as a way to solve the problems of some of the most popular metrics, the Eigenfactor is another form of citation counting. This count attempts to take into account the quality of research by giving these citations a weighting depending on which titles they are from with the more prestigious being given a higher score. However it is still relatively new as a measure and may not be readily accepted by some assessments.
Citation analysis is a simple count of the number of times an output, researcher or wider institution has been cited by other works. Its effectiveness is based on the assumption that the more citations something receives the better it is but there are many acknowledged problems with this (which will be discussed later in the unit).
Measuring the impact of an individual researcher the H-Index compares the number of outputs from a researcher with the number of times these have been cited. The outputs are ranked in order from highest to lowest number of citation and the point where an output has more citations than its ranking is its score. This can be quite complex to calculate and is often accused of being biased against newer researchers who haven't yet had time to build up a citable body of work.
Looking at journal titles, this metric is similar to the Eigenfactor as it looks at the perceived prestige of citation sources. This helps to give some wider context for the mention being counted, allegedly helping to build a more rounded picture of impact. However the calculations are based on a proprietary database which makes it harder to verify.
Although these are the main bibliometrics in use there are many more that you may come across. As with much else in research support, a great deal depend on the discipline and its standards and expectations.
Bibliometrics were designed to measure impact in a world where books and journals were the main means of disseminating research. In the 21st century methods of sharing have changed and metrics need to respond to this and develop new methods of measurement. Altmetrics (alternative metrics) were introduced in 2010 in response to these changes and are designed to help measure these new methods of sharing research. Like bibliometrics they measure the attention research has received, how widely it has been disseminated and its influence but they look at a wider range of information sources.
The sources used in Altmetrics cover everything from the official to the popular and each source carries a different weighting which goes towards making up the final Altmetric score. Publicly available news (both in print, online and TV) is one of the highest rated sources as it exposes research to a wider audience who may not otherwise have gone looking for it. Social media sources such as Twitter and Facebook are a large part of Altmetrics and most of the popular platforms are monitored which allows researchers to follow the impact of research as it happens. Peer review and reference managers are usually closed processes were the material looked at is only available to certain people. Including open peer review as a source allows these citations to be collected. If researchers can demonstrate that their work has influenced policy in an organisation or government this is also included and is weighted highly.
Each different source has a different weighting and these are used towards calculating the final Altmetric score. Represented by the dounghnut shape seen on the left, each colour represents a different source of information. The more colours, the more sources and the higher the overall score. Altmetrics are particularly useful for monitoring immediate impact as the results are collected quickly rather than having to wait for traditional citations to build up via the traditional publication process. Altmetrics are also useful for showing the tone of the reaction to a research output. Whereas traditional metrics just offer numbers, Altmetrics show you the content of the reactions so researchers can instantly find out whether people are responding positively.
