Optometry and Vision Science Research Guide: Literature review practice with Covidence and Zotero: Transitional justice

Send yourself an invitation and register for a Covidence account

Please make sure to use your uWaterloo email account when registering for Covidence; this will ensure that you can access the University's institutional licence, which access to unlimited reviews. Covidence reviews can also be shared with colleagues outside of the University Waterloo community. 

What is Covidence?

Covidence is a web-based platform, and no additional software installation is required.There is no desktop version of Covidence, it is web-based only. Covidence acts as a digital tool that researchers can use to streamline the screening and data collection of comprehensive literature review. And because reviews are often undertaken in teams, not only to minimize bias but also just to share the volume of work, Covidence excels at being a collaborative hub for research teams. Teams can ensure consistency for their roles in the review, as well as for data collection and extraction.

Covidence is very specific to the workflow of a review article, and it is currently for screening and reviewing journals articles only. Covidence does not take the place of other important tools that researchers may need during the compilation of a review, such as a reference manager (Zotero or Endnote) or the statistical software necessary for meta-analysis such as RevMan Web (Cochrane). Covidence works in conjunction with these tools and researchers can import and export data across tools. The Cochrane Library recently recognized Cochrane as the primary screening and data extraction tool for Cochrane authors conducting standard intervention reviews. The quality assessment template in Covidence 2.0 and 1.0 is based on The Cochrane Library's Risk of Bias (RoB) template. 

Practice literature review: Vision screening programs in Canada for preschoolers.

In this workshop, we will be working with a sample review using Covidence, and Zotero. Covidence and Zotero are fantastic tools for streamlining your structured review and the University of Waterloo has licensed, institutional access to both programs. 

The sample search and topic is, transitional justice and truth and reconciliation commissions.

For this libguide a yellow flags will indicate action items. There will be no marking for this workshop and you are welcome to explore this Libguide at your own pace or to work with your own examples. If you would like to follow along with the example topic and search utilized by the librarian for this session, please follow these steps: 

1. Register for Covidence using your UWaterloo email; no download required. 
2. Register for Zotero using your UWaterloo email; free download. 
3. Download the practice files to your computer; these are the practice files that we will be using for working with Covidence and Zotero.

Database: Sociological abstracts (Proquest)

Search strategy (example):

"transitional justice" AND (truth* OR reconcil* NEAR/3 (commission* OR inquir* OR meeting* OR reception*))

Limits:

• 2019 - present
• Peer-reviewed, scholarly journals 

If you would like to follow along with the librarian, please download the practice files for each database that we will be briefly looking at today: Pubmed and Scopus.

One file is a text file exported from Pubmed (Pubmed export), and the other file is a RIS file exported from Scopus. These are the data files that you will be able to import into Covidence and Zotero during the session. In (Step 1), we will be quickly running through a literature search in Pubmed and Scopus. Literature searching takes time and planning - be sure to give yourself enough time for this critical stage and to record your methods. If you would like to learn more, please visit Syntheses: systematic, scoping and literature reviews. The search strategies utilized for this library session will be narrow in scope to account for smaller practice data files. 

Send out and manage invitations

You can return to this screen at any time and issue or cancel invitations. The University of Waterloo Institutional CRKN License allows for unlimited reviewers. With this license you are also able to invite colleagues to your review from outside the University of Waterloo.

Covidence support

Creating a new review and inviting people, video

Learn more about the Institutional CRKN License for the University of Waterloo, article.

Team Settings

In the Team Settings screen, researchers can check in on the teams' progress, as well as set the screening and review rules for the team. There are currently no team settings' options for Data Extraction. 

The default option is Everyone can do everything this means there are no rules as to who can screen, review, extract data and decide consensus.  When working on very large reviews, or perhaps multi-site reviews this can be a very useful screen for assigning a group to screen for example and another groups solely focused on extraction. The data extraction stage can take quite awhile and therefore if you have references that meet all you inclusion criteria for their abstract, and then for the full-text, having roles assigned to review members can be helpful for managing time spent on the review. 

Select Manage rules in order to determine who on the team does what during the review process. 

The All studies must be screened by option lets you define who must screen each study in the stage. You could use this functionality to ensure that an experienced reviewer(s) will at the very least screen all studies.

The Conflicts can be resolved by option lets you control who can resolve conflicts during the screening process.  Any reviewer who is a member of this group will have the permission to resolve conflicts.  If no one is assigned to the group then everyone will be able to resolve conflicts. 

Covidence support

How to oversee a review and setup who does what, video and article.

Study Tags

Customized tags offer a convenient way to quickly add info and keep track of studies. The default tags for studies ongoing, and studies awaiting classification can not be deleted as these tags are counted for the PRISMA chart. 

Covidence support

How to create and manage study tags, article.

Eligibility criteria 

Covidence reworked this section for a major software update in July 2023. The PICO structure is now explicit and you can use it to organize your inclusion and exclusion criteria. However, if you aren't using a PICO structure or aren't sure where to enter a term, you can always use the Other Inclusion and Exclusion field. To learn more about PICO, visit the page for Asking and Answering Clinical Questions.

Highlights

Covidence allows you to input certain keywords that might indicate inclusion or exclusion for reviewers. Inclusion key words or phrases will be highlighted in green, and exclusion key words or phrases will be highlighted in red. This section is incredibly useful for larger teams as all of the key words and phrases can be customized for your review. Note that the criteria for highlights is one key word or phrase per line, and that Covidence only finds EXACT matches; spelling variations will not be captured.

• Let's ADD+ some highlights for our practice review:

Full-text exclusion reasons

These exclusion criteria are going to be more specific as they will be used for the full-text review. Covidence pre-populates the list of exclusion reasons with popular options, such as wrong does, paediatric population, wrong setting, wrong study design, wrong intervention etc. 

If you like, you can add your own exclusion reasons as well as delete those that there by default. 

Inclusion reasons are not used in full-text review; you have already indicated in title and abstract screening that you are including the article for further review of the full-text, so the full-text review is just going to be a matter of excluding the article from the final data extraction.

Always SAVE before leaving the settings page!

Covidence Support

Creating and managing eligibility criteria, article and video

Importing to Zotero 

We are going to great a new shared group library in Zotero. In Zotero we can organize our references, attach full-text, attach annotations and also share our libraries with a group. 

1. Open the desktop version of Zotero, and minimize to the background
2. All shared group libraries must be created online first. Go to zotero online and sign-in with your registered login. 
3. In Zotero online:
   1. In the top menu, choose GROUPS, and then select Create a New Group.
   2. Name the group: practice_review_yourname, and choose the membership value. Let's do Public closed membership, meaning that you have to be invited to participate in the group, but zotero users can also find this group and request access. 
   3. For Group Settings, let's confirm our settings. Note that open, public Groups can't share stored files (pdfs) due to copyright, so this is why it is important to have a closed or private group. Save Settings
4. In Zotero desktop:
   1. Maximize your desktop version of Zotero, and click sync (top right, green arrow). 
   2. You should now see your shared, Practice_Review group library in the Group Libraries of your Zotero desktop. If you see your new group library, you can close the browser window for Zotero online.
   3. In your Zotero desktop, File > Import our two practice files - the exact same files you used to import into Covidence. Make sure that your desktop is open to the Group Library, Practice_Review.
   4. Once both files are imported, select all and right-click to bring up a menu -- select Fine available PDFs. Zotero will use open access links as well as the proxy with the University of Waterloo to try and match each reference to the full-text.
5. Minimize and return to Covidence for the time-being.

A "Maybe" vote is treated the same way as a "Yes" vote, meaning that it will move that citation forward in your review. 

If in dual reviewer mode, you can filter those citations that you have cast one "Maybe" vote on by using the filter function at the top of your 'Awaiting other reviewer' list. If in dual reviewer mode and a citation receives two "Yes"/"Maybe"votes it will move forward in your review and you'll be able to see those votes cast by clicking on "History" underneath each citation.

If the second vote is a "No" then that citation will move to your 'Resolve conflicts' list. Here you won't be able to see those "Maybe" votes unless they were cast by you, as seeing your co-reviewer's previous votes can lend itself to biasing the adjudicating vote. 

You won't be able to generate your list of citations that received a "Maybe" vote during title and abstract screening. All voting is blinded meaning that if you are in dual reviewer mode, your co-reviewer won't be able to see which citations you voted "maybe" on until they have cast their vote, and if this vote is also a "Yes" or a "Maybe".  

Data Extraction template

The data extraction templates is pre-populated with generic options for data extraction, such as study ID, author details, methods, baseline population characteristics, interventions and comparisons, outcomes, etc. These fields are going to be specific to your team's review, so this template can be fully customized. 

This is also where you can publish instructions for reviewers, so that everyone is on the same page. You must publish the template in order to utilize your extraction fields. 

Covidence support

• How to create, publish and edit a data extraction template, video and article.
• How to export data to excel using extraction 2.0, video and article
• What does extraction 2.0 data look like when exported to Excel? video and article
• How to use multi-select checkboxes, article

Quality Assessment template 

The default QA template is Cochrane’s Risk of Bias. If you want to use a different tool, select I want to start from scratch in the Editor to delete the RoB domains and start with an empty template.

If you don’t want to do quality assessment in your review, don’t publish your QA template and QA will not appear for the extractors of your review. 

Covidence support

How to start a quality assessment, video and articles

Exporting references

Data can be exported from Covidence in a variety of formats. References or bibliographic records can be exported at any point from Covidence in CSV format, Cochrane Registry format, or formatted for reference managers like Zotero, Endnote and Mendeley. 

Inter-rater reliability data for screening and full-text review can also be exported in CSV format. Inter-rater reliability refers to the degree to which different raters or observers produce similar or consistent results when evaluating the same thing. 

The extracted data, compiled by reviewers using the inclusion and exclusion criteria, can be exported in CSV format. The extracted data can be concsenus-only data for quality assessment or individual data. This is a critical component of Covidence 1.0 extraction for interventional studies. 

Covidence support, articles and videos

• How to export study lists to your reference manager, video and article
• How to export extracted data to Excel using Extraction 2, video and article
• What is inter-rater reliability? article
• How to export inter-rater reliability data, video and article

As of January 2023, the University of Waterloo now has institutional access to Zotero. When you are signing up for a new Zotero account, please ensure to use your Uwaterloo email so that you can access the institutional storage. 

Zotero instantly creates references and bibliographies for any text editor, and directly inside Word and Google Docs. With support for over 10,000 citation styles, you can format your work to match any style guide or publication.

1. Open your zotero desktop, and minimize
2. Open Microsoft Word and create a new blank document. 
3. Zotero automatically installs the extension for Microsoft word - check your top toolbar for the Zotero menu. 
4. Now let's work with a few of our references (workshop demo).

Zotero Support

• Plugins for Zotero. Zotero
• Zotero for mobile, and the barcode scanner. Zotero