Skip to content

Altmetrics: the pros and the cons

nmat3485-f1

Following this week’s post on how to analyse online data we publish this more technical post by Nick Scott discusses altmetrics on the LSE Impact of Social Sciences blog. Almetrics are a new form of measuring research impact by adding on a wider set of metrics to traditional bibliographic rankings based on academic journal citation analysis. Aside from measuring citations, it also includes tweets, Facebook shares and saves on Delicious and Mendeley.

However, despite my support for Altmetrics, I’m a little concerned that the nature of debate is not helpful. Messaging is too fixated on whether altmetrics are better than traditional forms of assessing impact – citation analysis and the like. To me, this is to miss the point completely. We shouldn’t be arguing that altmetrics is a better way to measure reach, that makes it seem like we have an alternative. No, we should be arguing that altmetrics are the central way to measure the more varied forms of scholarly communication of the digital age.

While altmetrics are a very useful initiative in measuring impact beyond traditional scholarly output, they still have to show progress in certain areas. For instance, it is difficult to identify different types of outputs. Two main types are used, Digital Object Identifiers and ORCiD, but these cannot be assigned to items other than academic journals. Using URLs is also complicated because most items do not have just one webpage address, and there isn’t a clear system of compiling sets of URLs across multiple sites. Counting hits also seem limited to journals.

Another problem arises from ‘dark social’, which is sharing in places that analytics can’t reach, like emails, bookmarks and offline sharing. An M&E log, like the one used at ODI where researchers can forward examples of their work being used, can be helpful. Integrating atlmetrics with a media monitoring services can also track impact in the media. Campaigns that complement academic papers with outputs in other formats can be measured with altmetrics to get a full sense of its reach, and compare the different types of communication. And finally, measuring real impact is hard to do with altmetrics, which measures indicators of success and not of real impact.

Again, integration of an M&E log dataset of some sort, where people could add some of the more qualitative information of success – or even just a commenting facility to detail how work is used, received and leads to change – would be a great addition to altmetrics tools. Researchers do often hear about impact of their work, so tracking this somehow within altmetrics systems would be wonderful.

Nick has published a guide on monitoring and evaluating research communications using digital tools, which lists an array of principles on measuring impact as well as a list of digital tools. He also explains ODI’s award – winning online strategy.

About these ads
2 Comments Post a comment
  1. It’s very good to read a nuanced opinion on pros and cons. I’d point out that digital object identifiers can be assigned to scholarly records that are not papers in academic journals. (Content uploaded to Figshare, for example, gets a doi). Moreover, preprints and other scholarly records uploaded to repositories like arXiv or Pubmed can also be tracked by their arXiv and Pubmed id’s.

    I’ve been writing a series called Fieldwork for Altmetric. You and your readers might be interested.

    January 16, 2013
  2. Nick Scott #

    Hi Ernesto

    Thanks for the comment!

    Yes, I’ve been looking into DOIs for non-journal items at the Overseas Development Institute and I realise it is possible. However there are so many problems with every different potential identifier!

    For DOIs, the cost is a factor, especially for our partners who are in developing countries. arXiv and Pubmed both seem to not have any interest in social sciences at all, which is where most of our work takes place (and the work for readers of this site).

    Do you know of any plans to overhaul DOIs to make them free (and able to have parent-child relationships for different versions of the same paper, which is another seeming limitation)? Or are there any other URI initiatives on the horizon I should be looking out for?

    Thanks
    Nick

    January 18, 2013

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 5,070 other followers

%d bloggers like this: