Systematic and Scoping Review Workshop Series: Workshop 3: Streamlining your review with Covidence and Zotero

Covidence is a web-based platform, and no additional software installation is required.There is no desktop version of Covidence, it is web-based only.

For this practice review, we will be using Covidence to manage the workflow of our review, as well as extract data for our final analysis.

Covidence is very specific to the workflow of a review article, and it is currently for screening and reviewing journals articles only. Covidence does not take the place of other important tools that researchers may need during the compilation of a review, such as a reference manager (Zotero or Endnote) or the statistical software necessary for meta-analysis such as RevMan Web (Cochrane). 

Login to Covidence online; prior registration and sign-in required.  Registering with your uWaterloo email will ensure that you will be able to access our institutional license, which allows for unlimited reviews. 

1. Select Create a new Review and provide a working title for your review, example: Communication technologies in long-term care homes and their impact on social isolation and loneliness for older residents
2. Select your Review Type: Scoping review
3. Question type: Therapy/intervention
4. Select your Area of Research: Medical and health sciences

Don't stress when selecting any of the above settings - all of these can be easily updated and changed once the review is created. The most important step on this screen is going to be, Which account would you like to use? The default will be your personal registration, however you must select the UWaterloo institutional account in order to access the full license. 

Covidence Support

How to create a Covidence review, video

Review settings

The Review Settings screen lets you change your review’s name and type (like scoping, umbrella, or narrative review). You can update details such as your search strategy, which is helpful for team projects.

By default, Covidence requires two reviewers to screen each study to minimize bias. However, with Extraction 2.0, you can change this to just one reviewer, which can be useful for solo projects. Extraction 1.0 (mainly used for classic systematic reviews of interventional studies and Cochrane reviews) requires two reviewers.

It’s best practice to have at least two reviewers for structured reviews, including scoping and literature reviews. This is essential for minimizing bias and you should ideally be working on a review with a team.

For our practice review let's add our search strategy details:

• Date of Last Search
• Strategy: 

(((("communication technolog*") OR ("Information Technology"[Mesh] OR "Digital Technology"[Mesh])) AND (("Aged"[Mesh]) OR ((elderly[Title/Abstract] OR older[Title/Abstract] OR senior[Title/Abstract]) AND citizen*[Title/Abstract] OR person*[Title/Abstract] OR people[Title/Abstract]))) AND (("nursing home"[Title/Abstract] OR "care home"[Title/Abstract] OR "retirement home"[Title/Abstract] OR "assisted living"[Title/Abstract] OR "long*term care"[Title/Abstract]) OR (("Long-Term Care"[Mesh]) OR "Residential Facilities"[Mesh]))) AND (("Social Isolation"[Mesh]) OR ("social isolat*"[Title/Abstract] OR "social loneliness"[Title/Abstract] OR lonel*[Title/Abstract]))

• Now for the purposes of working through the references of our Practice Review, let's change all of the Reviewers Required to 1.
• Ensure that extraction 2 is selected and SAVE.

Covidence support

How to configure settings, article and video

Learn more about using Covidence for a Cochrane Review, article and video

Team Settings

On the Team Settings screen, you can view your team’s progress and set rules for screening and reviewing. Data extraction currently has no team settings.

The default is “Everyone can do everything,” meaning all members can review, screen, extract data, and resolve consensus. For large or multi-site reviews, you can assign different groups to specific tasks, like screening or extraction, which helps manage workflow as data extraction can take a long time.

Click “Manage rules” to decide which team members do each step of the review. The “All studies must be screened by” option lets you specify who screens each study, helpful to ensure experienced reviewers look at every study. The “Conflicts can be resolved by” option controls who can resolve disagreements; assigned members will have this permission, otherwise anyone on the team can do it.

Covidence support

How to oversee a review and setup who does what, video and article.

Eligibility criteria 

Covidence reworked this section for a major software update in July 2023. The PICO structure is now explicit and you can use it to organize your inclusion and exclusion criteria. However, if you aren't using a PICO structure or aren't sure where to enter a term, you can always use the Other Inclusion and Exclusion field. To learn more about PICO, visit the page for Asking and Answering Clinical Questions.

For this review, we are going to be using the PCC (Population, Concept, Context) framework as oppose to the PICO (Population, Intervention, Comparison, Outcome) framework, which is traditionally used for systematic reviews. PCC is designed to structure broader, exploratory questions where explicit interventions and outcomes may not be defined. In contrast, PICO supports focused clinical questions by specifying interventions and outcomes to facilitate targeted evidence synthesis and analysis. PCC is better suited for research questions that seek to map the extent and nature of literature, particularly outside strictly clinical areas, whereas PICO is structured for assessing effectiveness or impacts of specific interventions in clinical context. Learn more about PCC from the JBI Manual for Evidence Synthesis. 

Highlights

Covidence allows you to input certain keywords that might indicate inclusion or exclusion for reviewers. Inclusion key words or phrases will be highlighted in green, and exclusion key words or phrases will be highlighted in red. This section is incredibly useful for larger teams as all of the key words and phrases can be customized for your review. 

• Let's ADD+ some highlights for our practice review:

Full-text exclusion reasons

These exclusion criteria are going to be more specific as they will be used for the full-text review. Covidence pre-populates the list of exclusion reasons with popular options, such as wrong does, paediatric population, wrong setting, wrong study design, wrong intervention etc. If you like, you can add your own exclusion reasons as well as delete those that there by default. 

Inclusion reasons are not used in full-text review; you have already indicated in the Title and Abstract screening that you are including the article for further review of the full-text, so the full-text review is just going to be a matter of excluding the article from the final data extraction.

Always SAVE before leaving the settings page!

Covidence Support

Creating and managing eligibility criteria, article and video

Study Tags

In Covidence, Study Tags are an effective way to identify and organize important studies, such as references that provide excellent background for your introduction or discussion, even if they don’t meet inclusion criteria for your results. Custom tags help you quickly add notes, track the status of articles, or flag items for specific use by the review team.

Some tags, such as “Ongoing Study” and “Awaiting Classification,” are built into Covidence by default—these cannot be deleted, as they are automatically counted in your PRISMA chart for tracking review flow. Custom tags (e.g., “pending review,” “full-text on order,” “Background/context information,” or “Covidence support”) can be added and filtered to manage your references efficiently.

To create or manage tags, go to the ‘Study tags’ page in Covidence settings where you can add new tags or delete custom tags. Tags are visible to all reviewers and are included when exporting lists of references to Excel/CSV, making it easier to collaborate and track study status throughout your project.

Covidence support

How to create and manage study tags, article.

Database: Pubmed

Search strategy example using the PCC (Population, Concept, Context) framework :

copy/paste in search

(((("communication technolog*") OR ("Information Technology"[Mesh] OR "Digital Technology"[Mesh])) AND (("Aged"[Mesh]) OR ((elderly[Title/Abstract] OR older[Title/Abstract] OR senior[Title/Abstract]) AND citizen*[Title/Abstract] OR person*[Title/Abstract] OR people[Title/Abstract]))) AND (("nursing home"[Title/Abstract] OR "care home"[Title/Abstract] OR "retirement home"[Title/Abstract] OR "assisted living"[Title/Abstract] OR "long*term care"[Title/Abstract]) OR (("Long-Term Care"[Mesh]) OR "Residential Facilities"[Mesh]))) AND (("Social Isolation"[Mesh]) OR ("social isolat*"[Title/Abstract] OR "social loneliness"[Title/Abstract] OR lonel*[Title/Abstract]))

This search is as example of balancing indexing lanaguage (MeSH) with natural language. The scope for this term is deliberately narrow and this example is for practice only.

Database: Web of Science (WoS)

Search strategy example using the PCC (Population, Concept, Context) framework :

copy/paste into the advanced search query field

TS=(digital OR communicat* OR information* NEAR/3 (technolog* OR device*) ) AND TS=(aged OR senior OR older OR elderly NEAR/3(person* OR people* OR citizen* OR person*)) AND TS=("long?term?care" OR "nursing home" OR "retirement home") AND TS=( loneliness OR isolation)

A "Maybe" vote is treated the same way as a "Yes" vote, meaning that it will move that citation forward in your review. 

If in dual reviewer mode, you can filter those citations that you have cast one "Maybe" vote on by using the filter function at the top of your 'Awaiting other reviewer' list. If in dual reviewer mode and a citation receives two "Yes"/"Maybe"votes it will move forward in your review and you'll be able to see those votes cast by clicking on "History" underneath each citation.

If the second vote is a "No" then that citation will move to your 'Resolve conflicts' list. Here you won't be able to see those "Maybe" votes unless they were cast by you, as seeing your co-reviewer's previous votes can lend itself to biasing the adjudicating vote. 

You won't be able to generate your list of citations that received a "Maybe" vote during title and abstract screening. All voting is blinded meaning that if you are in dual reviewer mode, your co-reviewer won't be able to see which citations you voted "maybe" on until they have cast their vote, and if this vote is also a "Yes" or a "Maybe".  

Batch XML uploads of full-text

During the full-text review stage in Covidence, you can streamline access to articles and maintain thorough review documentation by integrating with Zotero. Here’s how the process works:

• Export studies missing full text from Covidence as an EndNote XML or RIS file. This can be done via the “Bulk upload PDFs” option in the full-text review section, ensuring that you have a precise list of which articles still require full-text retrieval.
• Import this file into Zotero using the desktop app. Once imported, use Zotero’s “Find Available PDFs” feature to automatically locate and attach as many full-text PDFs as possible. You can also manually attach files and use Zotero add-ons to check for retractions before proceeding.
• After attaching PDFs to the correct references, select all records with PDFs and export them from Zotero as an EndNote XML file, making sure to include both “Export Notes” and “Export Files” so that PDFs are packaged with the citation data.
• Back in Covidence, use the bulk upload tool to import the updated XML from Zotero. Covidence will automatically match PDFs to their corresponding records in the review and make the documents available to all team members for assessment.

This workflow lets you take advantage of Zotero’s full-text retrieval and reference management, ensures you are using the most accurate records, and provides a reliable way to track full-text acquisition and keep your review materials organized

The data extraction template is pre-populated with common data options for interventional, randomized controlled trials which is designed for a systematic review. However, we are going top publish a customised template to account for our scoping review protocol. Every data extraction template can be fully customised to meet the needs of your team. 

This is also where you can publish instructions for reviewers, so that everyone is on the same page. You must publish the template in order to utilize your extraction fields. 

Sample data extraction template, for set up in Covidence:

Covidence support

• How to create, publish and edit a data extraction template, video and article.
• How to export data to excel using extraction 2.0, video and article
• What does extraction 2.0 data look like when exported to Excel? video and article
• How to use multi-select checkboxes, article

The Quality Assessment template, and why we're not using this for our scoping review

Covidence’s quality assessment template is designed to capture risk of bias (ROB) directly within your review workflow. When you reach the quality assessment stage, Covidence offers a built-in template based on the Cochrane Risk of Bias (RoB) tool, which is widely recognized for evaluating randomized controlled trials. 

Risk of bias assessments are not typically performed in scoping reviews because the purpose of a scoping review is to provide an overview of the breadth of available evidence, without critically appraising or judging the quality or validity of individual studies. Scoping reviews aim to map the literature broadly, including studies regardless of methodological rigor or risk of bias.

In contrast, risk of bias assessment is a key component of systematic reviews, where the goal is to synthesize high-quality evidence and make clinical or policy recommendations. Systematic reviews must evaluate the internal validity of each included study to identify potential sources of bias that could affect the results and conclusions; this step is essential to ensure transparency and reliability in the final recommendations

Covidence support

How to start a quality assessment, video and articles

Data can be exported from Covidence in a variety of formats. References or bibliographic records can be exported at any point from Covidence in CSV format, or formatted for reference managers like Zotero, Endnote and Mendeley.

Key exports from Covidence:

• Study lists: Export references at any stage (e.g., all, included, excluded, screening, missing full text) in RIS, EndNote XML, or CSV formats for use in reference managers like EndNote, Zotero, Mendeley, etc. 
• PRISMA flow diagram: Export PRISMA data tracking how references were screened, included, or excluded, for reporting 
• Data extraction: Export extracted data (summary data, consensus, or individual reviewer data) as CSV or Excel files using Extraction 1 or 2
• Quality assessments: Export risk of bias/quality assessment forms as CSV 
• Inter-rater reliability data: Export reviewer agreement statistics as CSV 
• RevMan export: Export extracted data for direct use in RevMan Web/5 for Cochrane reviews 

Covidence support, articles and videos

• How to export study lists to your reference manager, video and article
• How to export extracted data to Excel using Extraction 2, video and article
• What is inter-rater reliability? article
• How to export inter-rater reliability data, video and article

Zotero can be used to insert in-text citations directly into your document while you write. By using the Zotero plugin for Word, Google Docs, or LibreOffice, you can quickly add citations from your Zotero library in any major citation style. Zotero automatically builds your bibliography as you go, ensuring in-text citations and the reference list stay perfectly matched and formatted