We want to hear from you. You're viewing the newest version of the Library's website. Please send us your feedback!
We want to hear from you! Fill out the Library's User Survey and enter to win.
Citation-tracking databases, such as Scopus, Web of Science, and Google Scholar, are used extensively to collect and report a range of bibliometric measures. Citation-tracking databases are susceptible to limitations based on factors such as:
This content is informed by section 3.1.1 "Limitations of citation-tracking databases" of the White Paper, "Measuring Research Output Through Bibliometrics".
1 Federation for the Humanities and Social Sciences. (2014). The Impacts of Humanities and Social Science Research: Working Paper. Retrieved from http://www.ideas-idees.ca/sites/default/files/2014-10-03-impact-project-draft-report-english-version-final2.pdf
2 Archambault, E., & Lariviere, V. (2010). The limits of bibliometrics for the analysis of the social sciences and humanities literature. World Social Science Report : Knowledge Divides. (pp. 251-254). Paris: UNESCO Publishing and International Social Science Council.
To be reliable, a large sample size is required in a number of measures, and percentiles may be a more suitable approach with small sample sizes depending on the context.
1 Vieira, E. S., & Gomes, J. A. N. F. (2010). A research impact indicator for institutions. Journal of Informetrics, 4(4), 581-590. doi:10.1016/j.joi.2010.06.006
2 Abramo, G., D’Angelo, C. A., & Viel, F. (2013). The suitability of h and g indexes for measuring the research performance of institutions. Scientometrics, 97(3), 555-570. doi:10.1007/s11192-013-1026-4
3 Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). Leiden Manifesto for research metrics. Nature, 520, 429-431.
Citation-tracking databases calculate bibliometric data based on the items they index, and each database has differing coverage and a unique data universe. This means:
A limitation of citation-tracking databases is the different ways authorship is attributed. Problems can stem from data errors, name ambiguity, and how multi-authored articles are attributed.
Citation-tracking databases are also susceptible to gender bias, which limits the reliability and utility of citation-based measures. Evidence shows that:
1 Larivière, Ni, Gingras, Cronin & Sugimoto, 2013
2 Ferber & Brun, 2011
3 Maliniak, Powers, and Walter, 2013
4 Sarsons, 2015
Time affects both a researcher's impact and how their impact is understood.
1 Council of Canadian Academies. Expert Panel on Science Performance and Research Funding (Ed.). (2012). Informing research choices: Indicators and judgement. Ottawa: Council of Canadian Academies.
2 Wang, J. (2013). Citation time window choice for research impact evaluation. Scientometrics, 94(3), 851-872. doi:10.1007/ s11192-012-0775-9