Rapid evaluations, assessments and reviews are becoming more and more relevant, as the need increases for quick and reliable evidence to be available when it is needed. In this quest, the balance between timeliness, cost, and rigour is essential.
When the word “rapid” is used in the context of evaluation, care should be taken to find out exactly what is required. The range of terminology incorporating the word “rapid”, is extensive. Semantics are important, and each of the terms below have their own nuances. We will take a closer look at Rapid Evaluation, Rapid Assessment and Rapid Review.
Rapid evaluation, assessment, and appraisal
Methods of rapid evaluation, assessment and appraisal, are mostly qualitative and emanate from ethnography. Various approaches 1 can be used in rapid evaluation.
What is it? An approach that employs intensive, team-based fieldwork, multiple methods for data collection, iterative processes of data collection and analysis, simultaneous data analysis, and community participation.
What is it rooted in? It comes from the tradition of cultural anthropology and ethnography.
What can it be used for? It is suitable where limited time or other resources are available, and where issues in the question are not yet clearly articulated. It can quickly generate a holistic understanding of a programme from multiple perspectives, including programme “insiders” and “outsiders”.
What are the primary data collection methods? Data collection methods are mostly qualitative: interviews, direct observations, focus group discussions, mapping. Quantitative techniques, e.g. surveys can also be used.
How long does it take? 4 to 6 weeks.
What are the advantages? It is fast, fast, cost-effective, produces accurate data, provide “insider perspectives” on complicated problems, works well for investigating emerging problems or specific issues.
What are the disadvantages? Less precise than more structured evaluations Limited scope and depth.
Take note: “Rapid” does not mean “rushed”. Although rapid evaluations are quicker to execute than traditional evaluations, extensive up-front preparations are required, coupled with meticulous planning and skillful scheduling to ensure strategic use of methods, synchronisation of data collection and analysis. Requires a team of trained evaluators Evaluation team leader must be highly trained in qualitative research methods.
Source: I -Tech Technical Implementation Guide #6\.2008. Rapid Evaluation2.
What is it? Rapid reviews use components of the systematic review for knowledge synthesis, in a simplified process3, and can be described as “a form of evidence synthesis that may provide more timely information for decision making compared with standard systematic reviews”.4
What is it rooted in? Rapid review is rooted in Systematic Review methodology. Systematic Reviews use “systematic and explicit methods to identify, select, critically appraise, and extract and analyze data from relevant research”.5
Source: Virginia commonwealth university. Research Guides. Rapid Review Protocol. What is a rapid review? 6
What can it be used for? Rapid reviews can be used for exploring new or emerging research topics, or critical topics. It can be used to produce updates of previous reviews, to assess what knowledge already exists about a policy or practice.
How is it done? Methods vary, and depends on factors such as the type of review, resources available, and quality of literature, as well as the experience of reviewers.
How long does it take? Different sources give different estimates, varying from less than 5 weeks, to one to 12 weeks, and up to 6 months.
What are the advantages? It speeds up the systematic review process because it omits stages of the process. It makes it possible to produce a review at short notice when doing a systematic review is not practical.
What are the limitations? The process is less rigorous, and the search is not as comprehensive as in a systematic review. It necessitates “cutting corners”, and researchers must be mindful that this may lead to bias. Findings have to be interpreted cautiously, and may have limitations.
Take note: Content experts and those experienced with systematic reviews have to be included. Rapid reviews are “ill defined” – no universal definition exists.8
Written by Fia van Rensburg
References McNall, M., Foster-Fisherman, P.G. 2007. Methods of Rapid Evaluation, Assessment and Appraisal. American Journal of Evaluation. Volume: 28 issue: 2, page(s): 151-168. Issue published: June 1, 2007 Available on: http://aje.sagepub.com  http://www.pointk.org/resources/files/TIG_6_Rapid_Eval.pdf  Temple University Libraries. https://guides.temple.edu/c.php?g=78618&p=4156608  Tricco, A.C., Jesmin, A, Straus, S.E. n.d. Systematic reviews vs. Rapid reviews: What’s the didfference? CADTH Rapid Review Summit. University of Toronto  Higgins, J.P.T, and Green, S. 2011. Cochrane Handbook for Systematic Reviews of Interventions V5.1.0. Available on: https://handbook-5-1.cochrane.org/  Other sources suggest different timeframes.  https://guides.library.vcu.edu/rapidreview  Cochrane: Rapid Reviews-An Introduction. Available on: https://methods.cochrane.org/rapidreviews/publications