Tweets and Mendeley readers Two different types of article level metrics firstname.lastname@example.org Stefanie Haustein @stefhaustein
Overview • Altmetrics • increasing use • meaning? • Aim of the studies • Data sets and methods • Results • documents • correlations • disciplines • Conclusions & outlook
Altmetrics: increasing use • social media activity around scholarly articles growing by 5% to 10% per month (Adie & Roe, 2013) • Mendeley and Twitter largest altmetrics sources • Mendeley • 521 mil ion bookmarks • 2.7 mil ion users • 32% increase of users from 09/2012 to 09/2013 • Twitter • 500 mil ion tweets per day • 230 mil ion active users • 39% increase of users from 09/2012 to 09/2013 Adie, E. & Roe, W. (2013). Altmetric: Enriching Scholarly Content with Article-level Discussion and Metrics. Learned Publishing, 26(1), 11-17. Mendeley statistics based on monthly user counts from 10/2010 to 01/2014 on the Mendeley website accessed through the Internet Archive Twitter statistics: https://business.twitter.com/whos-twitter and http://www.sec.gov/Archives/edgar/data/1418091/000119312513400028/d564001ds1a.htm
Altmetrics: meaning? • ultimate goals • similar to but more timely than citations Ø predicting scientific impact
• different, broader impact than captured by citations Ø measuring societal impact
• impact of various outputs Ø “value al research products”
Piwowar (2013) Piwowar, H. (2013). Value al research products. Nature, 493(7431), 159.
Altmetrics: meaning? • Altmetrics are “representing very different things”
(Lin & Fenner, 2013) • unclear what exactly they measure: • scientific impact • social impact • “buzz” Lin, J. & Fenner, M. (2013). Altmetrics in evolution: Defining and redefining the ontology of article-level metrics. Information Standards Quarterly, 25(2), 20-26.
Altmetrics: meaning? ad-hoc classifications need to be supported by research
Altmetrics: meaning? scientist on Twitter tweeting scientific paper in non-scholarly manner: • scientific impact? • social impact? • buzz?
Aim of the studies • providing empirical evidence of Mendeley reader counts and tweets of scholarly documents for a large data set • generate knowledge about factors influencing popularity of scholarly documents on Mendeley and Twitter • analyzing the fol owing research questions: • What is the relationship between social-media and citation counts? • How do social-media metrics differ? • Which papers are highly tweeted or highly bookmarked? • How do these aspects differ across scientific disciplines? Haustein, S., Peters, I., Sugimoto, C.R., Thelwall, M., & Larivière, V. (2014). Tweeting Biomedicine: An Analysis of Tweets and Citations in the Biomedical Literature. Journal of the Association for Information Sciences and Technology. Haustein, S., Larivière, V., Thelwall, M., Amyot, D., & Peters, I. (submitted). Tweets vs. Mendeley readers: How do these two social media metrics differ? IT-Information Technology.
Aim of the studies • large-scale analysis of tweets and Mendeley readers of biomedical papers • Twitter and Mendeley coverage • Twitter and Mendeley user rates • correlation with citations • discovering differences between: • documents • disciplines & specialties Ø providing an empirical framework to compare coverage, correlations and user rates
Data sets & methods • 1.4 mil ion PubMed papers covered by WoS • publication years: 2010-2012 • document types: articles & reviews • matching of WoS and PubMed • tweet counts col ected by Altmetric.com • col ection based on PMID, DOI, URL • matching WoS via PMID • Mendeley readership data col ected via API • matching title and author names • journal-based matching of NSF classification
Data sets & methods: framework
Data sets & methods: age biases Current biases influencing correlation coefficients Ø compare documents of similar age Ø normalize for age differences
Results: documents • Twitter coverage is quite low but increasing • correlation between tweets and citations is very low Publication Twitter Papers Spearman's ρ Mean Median Maximum year coverage (T≥1) T2010 2.1 1 237 2.4% 13,763 .104** C2010 18.3 7 3,922 T2011 2.8 1 963 10.9% 63,801 .183** C2011 5.7 2 2,300 T2012 2.3 1 477 20.4% 57,365 .110** C2012 1.3 0 234 T2010-2012 2.5 1 963 9.4% 134,929 .114** C2010-2012 5.1 1 3,922
Results: documents Top 10 tweeted documents: catastrophe & topical / web & social media / curious story
scientific discovery / health implication / scholarly community Article Journal C T Hess et al. (2011). Gain of chromosome band 7q11 in papil ary thyroid carcinomas of young patients PNAS 9 963 is associated with exposure to low-dose irradiation Yasunari et al. (2011). Cesium-137 deposition and contamination of Japanese soils due to the PNAS 30 639 Fukushima nuclear accident Sparrow et al. (2011). Google Effects on Memory: Cognitive Consequences of Having Information at Science 11 558 Our Fingertips Journal of Physical Onuma et al. (2011). Rebirth of a Dead Belousov–Zhabotinsky Oscil ator -- 549 Chemistry A Silverberg (2012). Whey protein precipitating moderate to severe acne flares in 5 teenaged athletes Cutis -- 477 Wen et al. (2011). Minimum amount of physical activity for reduced mortality and extended life Lancet 51 419 expectancy: a prospective cohort study Journal of Sexual Kramer (2011). Penile Fracture Seems More Likely During Sex Under Stressful Situations -- 392 Medicine New England Newman & Feldman (2011). Copyright and Open Access at the Bedside 3 332 Journal of Medicine Reaves et al. (2012). Absence of Detectable Arsenate in DNA from Arsenate-Grown GFAJ-1 Cel s Science 5 323 Bravo et al. (2011). Ingestion of Lactobacil us strain regulates emotional behavior and central GABA PNAS 31 297 receptor expression in a mouse via the vagus nerve
Results: correlations PubMed papers covered by Web of Science (PY=2011) Spearman correlations between citations (C), Mendeley readers (R) and tweets (T) for al papers published in 2011 (A, n=586,600), for papers with respectively at least one citation (B, n=410,722), one Mendeley reader (C, n=390,190) or one tweet (D, n=63,800), one Mendeley reader and one tweet (E, n=45,229) and one citation, one Mendeley reader and one tweet (F, n=36,068). All results are significant at the 0.01 level (two-tailed).
Results: disciplines PubMed papers covered by Web of Science 2010-2012
Altmetrics: disciplinary biases x-axis: coverage of specialty on platform y-axis: correlation between social media counts and citations bubble size: intensity of use based on mean social media count rate
Results: disciplines General Biomedical Research papers 2011 Scatterplot of number of citations and number of tweets (A, ρ=0.181**) and Mendeley readers (B, ρ=0.677**), bubble size represents number of Mendeley readers (A) and tweets (B). The respective three most tweeted (A) and read (B) papers are labeled showing the first author.
Results: disciplines Public Health papers 2011 Scatterplot of number of citations and number of tweets (A, ρ=0.074**) and Mendeley readers (B, ρ=0.351**) for papers published in Public Health in 2011. The respective three most tweeted (A) and read (B) papers are labeled showing the first author.
Conclusions & outlook • uptake, usage intensity and correlations differ between disciplines and research fields Ø social media counts of papers from different fields are not directly comparable • citations, Mendeley readers and tweets reflect different kind of impact on different social groups • Mendeley seems to mirror use of a broader but stil academic audience, largely students and postdocs • Twitter seems to reflect the popularity among a general public and represents a mix of societal impact, scientific discussion and buzz Ø the number of Mendeley readers and tweets are two distinct social media metrics
Conclusions & outlook • before applying social media counts in information retrieval and research evaluation further research is needed: Ø identifying different factors influencing popularity of scholarly documents on social media Ø analyzing uptake and usage intensity in various disciplines Ø differentiating between audiences and engagements
Haustein, S., Peters, I., Sugimoto, C.R., Thelwal , M., & Larivière, V. (in press). Tweeting biomedicine: an analysis of tweets and citations in the biomedical literature. Journal of the Association for Information Sciences and Technology.
Haustein, S., Larivière, V., Thelwall, M., Amyot, D., & Peters, I. (submitted). Tweets vs. Mendeley readers: How do these two social media metrics differ? IT-Information Technology. Thank you for your attention! Questions? Stefanie Haustein email@example.com @stefhaustein