144 - Beyond Bibliometrics: Altmetric Analysis as an Early Signal of the Impact of the National Center for Advancing Translational Sciences (NCATS) Clinical and Translational Science Awards (CTSA) Program

Abstract Information: The NCATS CTSA Program supports 60+ medical institutions throughout the U.S. The Program strives to improve health by speeding up research translation to clinical practice via supporting institutions’ research infrastructure and developing the translational research enterprise and workforce.(1) Understanding the CTSA Program’s impact is challenging due to the numerous, complex activities supported under multiple funding mechanisms, plus the difficulties of collecting and linking data in ways that can demonstrate impact. One traditional method of understanding research program impacts is bibliometrics. While bibliometrics impact measures, such as impact factor, h-index, and quantity of citations by article or author, can be useful, one major limitation is the time lag between research publication date and accrual of citations. This lag means that bibliometric impact analyses should not be conducted on recently published research papers. Identifying earlier indicators of potential research impact could greatly assist research funders in supporting the most promising future research. Some studies (2,3) have found a positive association between research articles’ Altmetric Attention Score and its component sources of attention -- which identify journal article mentions in other public domains (i.e., news, social media, Wikipedia, policy documents, peer review, and other online sources (4)) – and the articles’ later journal citations. This suggests Altmetric scores may represent early signals of future research impacts. Therefore, to identify future potentially high impact research generated by CTSA Program, we evaluated the Almetric scores for recently published scientific articles associated with the Program. We used the NIH SPIRES database (5) to identify scientific papers published between July 1, 2023 and December 31, 2023 that cited support from a CTSA Program UL1 or UM1 award (the primary grant to the program institution). SPIRES yielded 2,440 publications whose PMIDS or DOIs were used to construct an Altmetric Explorer report. We identified journal articles with the highest total and subcomponent Altmetric scores, assessing their distribution across topic areas and grantees. We found both topic and grantee diversity among the highest-scoring publications. However, many of the highest scoring papers were focused on COVID-19 or SARS-CoV-2, reflecting the continued high level of public interest in this pandemic-related research. These preliminary results suggest that Altmetric measures may help elucidate the potential impact of the CTSA Program. Future work will assess the relationship between Altmetric scores and citations to validate this methodology for the CTSA Program research portfolio. 1. U.S. Department of Health and Human Services (HHS). (n.d.). Clinical and translational science awards (CTSA) program. NCATS. https://ncats.nih.gov/research/research-activities/ctsa 2. Akhil Pandey Akella, Hamed Alhoori, Pavan Ravikanth Kondamudi, Cole Freeman, Haiming Zhou, Early indicators of scientific impact: Predicting citations with altmetrics, Journal of Informetrics, Volume 15, Issue 2, 2021, https://doi.org/10.1016/j.joi.2020.101128. 3. Ouchi, A., Saberi, M.K., Ansari, N., Hashempour, L. and Isfandyari-Moghaddam, A. (2019), "Do altmetrics correlate with citations? A study based on the 1,000 most-cited articles", Information Discovery and Delivery, Vol. 47 No. 4, pp. 192-202. https://doi.org/10.1108/IDD-07-2019-0050 4. Our sources. Altmetric. (2023, December 11). https://www.altmetric.com/about-us/our-data/our-sources/ 5. U.S. HHS (n.d.). Reporting and analytics. National Institutes of Health. https://www.era.nih.gov/about-era/services-for-agency-staff/reporting-analytics