Bibliometrics & Measuring Research Output: Institutional data

The University's White Paper, "Measuring Research Output Through Bibliometrics", identifies how bibliometric data can be used to support a bibliometric analysis at the institutional level, and offers context for:

• Possible objectives of the analysis
• Appropriate uses of the data, including possible measures and considerations
• Inappropriate uses of the data

Possible objectives of an institutional-level bibliometric analysis include:

1. Understand the institution’s research activity in terms of production of publication outputs and associated citations.
    
2. Understand the institution's performance over time, or compared with a selected peer.
    
3. Validate university-level data that external agencies have published. For example, the data generated by a university rankings organization.

It is also important to be mindful of how the level of detail influences an institutional-level bibliometric analysis. For example, there are too many disparate elements integrated at this level of analysis to obtain a detailed understanding.

Open image in larger format

Appropriate uses of an institutional-level bibliometric analysis include:

• To better understand performance measurement and reporting analysis at the institutional level.
   
• Internal reporting to examine the validity of data that other institutions or agencies have published.
   
• Performance measurement relative to key peers, only with acknowledgement and consideration of differing contexts that would impact comparison, including: different missions, program mixes, institution age, regional or international foci, dominant language for publishing and administrative or funding environment.
   
• For example, a comparison of a relatively new research-intensive institution with a liberal arts primarily undergraduate university must consider and acknowledge the institutional mission, the faculties and program mix, the number of researchers at the institution by discipline, and the institution age.

Possible Measures

• Publication counts in Web of Science based on selected journal classifications where faculty members are most active.
   
• Normalized citation impact.
   
• Top percentiles.
   
• For more information, see Measures.

Considerations

• Measures or analysis provided with contextual information, e.g., number of researchers, areas of specialty, career stage.
   
• Measures or analysis provided as part of a data package along with other measures, e.g., research funding, awards and honours.

Inappropriate uses of a institutional-level bibliometric analysis include:

• Comparing publication and citation data that has not been appropriately normalized.
   
• Comparing across different institutions based on h-index data without appropriate normalization. 
   
• Comparing performance relative to other institutions where the universities or institutions are substantively different, for example, have different missions (liberal arts compared to a technical institute; regionally focused institution compared to a national-focused institution) or publish in predominantly different languages.