Altmetric measurements derived from the social web are increasingly advocated and used as early indicators of article impact and usefulness. Nevertheless, there is a lack of systematic scientific evidence that altmetrics are valid proxies of either impact or utility although a few case studies have reported medium correlations between specific altmetrics and citation rates for individual journals or fields. To fill this gap, this study compares 11 altmetrics with Web of Science citations for 76 to 208,739 PubMed articles with at least one altmetric mention in each case and up to 1,891 journals per metric. It also introduces a simple sign test to overcome biases caused by different citation and usage windows. Statistically significant associations were found between higher metric scores and higher citations for articles with positive altmetric scores in all cases with sufficient evidence (Twitter, Facebook wall posts, research highlights, blogs, mainstream media and forums) except perhaps for Google+ posts. Evidence was insufficient for LinkedIn, Pinterest, question and answer sites, and Reddit, and no conclusions should be drawn about articles with zero altmetric scores or the strength of any correlation between altmetrics and citations. Nevertheless, comparisons between citations and metric values for articles published at different times, even within the same year, can remove or reverse this association and so publishers and scientometricians should consider the effect of time when using altmetrics to rank articles. Finally, the coverage of all the altmetrics except for Twitter seems to be low and so it is not clear if they are prevalent enough to be useful in practice.
ER - TY - CONF AU - Bar-Ilan, Judit AU - Haustein, Stefanie AU - Peters, Isabella AU - Priem, Jason AU - Shema, Hadas AU - Terliesner, Jens A2 - Archambault, Éric A2 - Gingras, Yves A2 - Larivière, Vincent T1 - Beyond citations: Scholars' visibility on the social Web T2 - Proceedings of 17th International Conference on Science and Technology Indicators, Montréal: Science-Metrix and OST PB - CY - PY - 2012/ M2 - VL - 1 IS - SP - 98 EP - 109 UR - http://2012.sticonference.org/Proceedings/vol1/Bar-Ilan_Beyond_98.pdf M3 - KW - authors KW - citations KW - correlation KW - social KW - visibility KW - web L1 - SN - N1 - Beyond citations: Scholars' visibility on the social Web N1 - AB - Traditionally, scholarly impact and visibility have been measured by countingpublications and citations in the scholarly literature. However, increasingly
scholars are also visible on the Web, establishing presences in a growing
variety of social ecosystems. But how wide and established is this presence,
and how do measures of social Web impact relate to their more traditional
counterparts? To answer this, we sampled 57 presenters from the 2010 Leiden STI
Conference, gathering publication and citations counts as well as data from the
presenters' Web "footprints." We found Web presence widespread and diverse: 84%
of scholars had homepages, 70% were on LinkedIn, 23% had public Google Scholar
profiles, and 16% were on Twitter. For sampled scholars' publications, social
reference manager bookmarks were compared to Scopus and Web of Science
citations; we found that Mendeley covers more than 80% of sampled articles, and
that Mendeley bookmarks are significantly correlated (r=.45) to Scopus citation
counts. ER - TY - JOUR AU - Bar-Ilan, Judit AU - Haustein, Stefanie AU - Peters, Isabella AU - Priem, Jason AU - Shema, Hadas AU - Terliesner, Jens T1 - Beyond citations: Scholars' visibility on the social Web JO - PY - 2012/ VL - IS - SP - EP - UR - http://dblp.uni-trier.de/db/journals/corr/corr1205.html#abs-1205-5611 M3 - KW - citations KW - scholars KW - social KW - visibility KW - web L1 - SN - N1 - N1 - AB - Traditionally, scholarly impact and visibility have been measured by counting
publications and citations in the scholarly literature. However, increasingly
scholars are also visible on the Web, establishing presences in a growing
variety of social ecosystems. But how wide and established is this presence,
and how do measures of social Web impact relate to their more traditional
counterparts? To answer this, we sampled 57 presenters from the 2010 Leiden STI
Conference, gathering publication and citations counts as well as data from the
presenters' Web "footprints." We found Web presence widespread and diverse: 84%
of scholars had homepages, 70% were on LinkedIn, 23% had public Google Scholar
profiles, and 16% were on Twitter. For sampled scholars' publications, social
reference manager bookmarks were compared to Scopus and Web of Science
citations; we found that Mendeley covers more than 80% of sampled articles, and
that Mendeley bookmarks are significantly correlated (r=.45) to Scopus citation
counts. ER - TY - GEN AU - Priem, Jason AU - Piwowar, Heather A. AU - Hemminger, Bradley M. A2 - T1 - Altmetrics in the wild: Using social media to explore scholarly impact JO - PB - AD - PY - 2012/ VL - IS - SP - EP - UR - http://arxiv.org/abs/1203.4745 M3 - KW - altmetrics KW - bookmarking KW - citations KW - correlation KW - coverage KW - impact KW - wild L1 - N1 - [1203.4745v1] Altmetrics in the wild: Using social media to explore scholarly impact N1 - AB - In growing numbers, scholars are integrating social media tools like blogs,
Twitter, and Mendeley into their professional communications. The online,
public nature of these tools exposes and reifies scholarly processes once
hidden and ephemeral. Metrics based on this activities could inform broader,
faster measures of impact, complementing traditional citation metrics. This
study explores the properties of these social media-based metrics or
"altmetrics", sampling 24,331 articles published by the Public Library of
Science.
We find that that different indicators vary greatly in activity. Around 5% of
sampled articles are cited in Wikipedia, while close to 80% have been included
in at least one Mendeley library. There is, however, an encouraging diversity;
a quarter of articles have nonzero data from five or more different sources.
Correlation and factor analysis suggest citation and altmetrics indicators
track related but distinct impacts, with neither able to describe the complete
picture of scholarly use alone. There are moderate correlations between
Mendeley and Web of Science citation, but many altmetric indicators seem to
measure impact mostly orthogonal to citation. Articles cluster in ways that
suggest five different impact "flavors", capturing impacts of different types
on different audiences; for instance, some articles may be heavily read and
saved by scholars but seldom cited. Together, these findings encourage more
research into altmetrics as complements to traditional citation measures. ER - TY - JOUR AU - Haustein, Stefanie AU - Siebenlist, Tobias T1 - Applying social bookmarking data to evaluate journal usage JO - Journal of Informetrics PY - 2011/ VL - 5 IS - 3 SP - 446 EP - 457 UR - http://www.sciencedirect.com/science/article/pii/S1751157711000393 M3 - http://dx.doi.org/10.1016/j.joi.2011.04.002 KW - bibsonomy KW - bookmarking KW - citations KW - scientometrics KW - social KW - tagging L1 - SN - N1 - Applying social bookmarking data to evaluate journal usage N1 - AB - Web 2.0 technologies are finding their way into academics: specialized social bookmarking services allow researchers to store and share scientific literature online. By bookmarking and tagging articles, academic prosumers generate new information about resources, i.e. usage statistics and content description of scientific journals. Given the lack of global download statistics, the authors propose the application of social bookmarking data to journal evaluation. For a set of 45 physics journals all 13,608 bookmarks from CiteULike, Connotea and BibSonomy to documents published between 2004 and 2008 were analyzed. This article explores bookmarking data in STM and examines in how far it can be used to describe the perception of periodicals by the readership. Four basic indicators are defined, which analyze different aspects of usage: Usage Ratio, Usage Diffusion, Article Usage Intensity and Journal Usage Intensity. Tags are analyzed to describe a reader-specific view on journal content. ER - TY - JOUR AU - Aksnes, Dag W. T1 - A macro study of self-citation JO - Scientometrics PY - 2003/ VL - 56 IS - 2 SP - 235 EP - 246 UR - http://dx.doi.org/10.1023/A%3A1021919228368 M3 - 10.1023/A:1021919228368 KW - citationRate KW - citations KW - culture KW - scientometrics KW - selfCitation L1 - SN - N1 - A macro study of self-citation - Springer N1 - AB - This study investigates the role of self-citation in the scientific production of Norway (1981-1996). More than 45,000 publications have been analysed. Using a three-year citation window we find that 36% of all citations represent author self-citations. However, this percentage is decreasing when citations are traced for longer periods. We find the highest share of self-citation among the least cited papers. There is a strong positive correlation between the number of self-citations and the number of authors of the publications. Still, only a minor part of the overall increase in citation rates that can be found for multi-authored papers is due to self-citations. Also, the share of self-citation shows significant variations among different scientific disciplines. The results are relevant for the discussion concerning use of citation indicators in research assessments. ER -