Support To Rural India's Public Education System (STRIPES2) and impact on numeracy and literacy scores: A cluster randomized trial in rural villages of Madhya Pradesh, India

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • The study randomized entire villages (clusters), satisfying the requirement for class-level or stronger randomization.
    • In this Madhya Pradesh cluster-randomized controlled trial, 196 villages (clusters) were randomized to receive either a health ... or education ... intervention.
    • Relevant Quotes: 1) "In this Madhya Pradesh cluster-randomized controlled trial, 196 villages (clusters) were randomized to receive either a health (CHAMPION2: community health promotion and medical provision and impact on neonates) or education (STRIPES2) intervention." (p. 1) 2) "Randomization of clusters was performed by the trial statistician based in London in June 2019 using a random number generator, with stratification by village size..." (p. 9) 3) "In STRIPES2, the target group was children ... resident in the trial villages." (p. 4) Detailed Analysis: The study explicitly describes itself as a cluster-randomized controlled trial where the unit of randomization was the "village" (cluster). All eligible children within the village were assigned to the condition of their village. This level of randomization (village) encompasses the schools and classes within that geographic unit, effectively operating at a level equal to or higher than the class or school. The ERCT standard accepts stronger school-level (or in this case, village-level) randomization as meeting the class-level criterion, as it prevents contamination between students in the same environment. Criterion C is met because randomization was performed at the village level, which is a cluster exceeding the class-level requirement.
  • E

    Exam-based Assessment

    • The study used EGRA and EGMA, which are widely recognized standardized assessments.
    • The primary outcome was the arithmetic mean of the child's scores on EGRA (Early Grade Reading Assessment) and EGMA (Early Grade Mathematics Assessment) tests [26,27].
    • Relevant Quotes: 1) "The primary outcome was the arithmetic mean of the child's scores on EGRA (Early Grade Reading Assessment) and EGMA (Early Grade Mathematics Assessment) tests [26,27]." (p. 8) 2) "We chose to use EGRA and EGMA because of our positive experiences applying them in similar contexts ... and because they are widely used tests for assessing early grade literacy and numeracy." (p. 8) 3) "The EGRA and EGMA protocols were adapted to the local context by the National Foundation for Educational Research (NFER, [28]) and a board of local language and curriculum experts..." (p. 10) Detailed Analysis: The study used the Early Grade Reading Assessment (EGRA) and Early Grade Mathematics Assessment (EGMA). These are widely recognized, international standardized assessments for measuring foundational literacy and numeracy in early grades. The text confirms they were adapted to the local context by NFER (a recognized testing body), ensuring validity while maintaining the standardized framework. Criterion E is met because the study utilized widely recognized standardized exams (EGRA and EGMA) for assessment.
  • T

    Term Duration

    • The study duration spanned multiple years, significantly exceeding the one-term requirement.
    • In some intervention arm villages, classes ran for up to 23 months (October 2019 to March 2020, January 2021 to April 2021 and July 2021 to July 2022 (inclusive))...
    • Relevant Quotes: 1) "In November 2019, Pratham started the 3-month preparatory phase for students' readiness." (p. 6) 2) "Tests were conducted about one month after the intervention teams had stopped the main activities (between 24 July and 19 September 2022)." (p. 10) 3) "In some intervention arm villages, classes ran for up to 23 months (October 2019 to March 2020, January 2021 to April 2021 and July 2021 to July 2022 (inclusive))..." (p. 13) Detailed Analysis: The intervention spanned a significant period, beginning in late 2019 and concluding with testing in mid-2022. Even accounting for COVID-19 interruptions, the active periods (e.g., July 2021 to July 2022) far exceeded the minimum requirement of one academic term (3-4 months). The outcomes were measured at the end of this multi-year process. Criterion T is met because the interval between intervention start and measurement covered multiple years, far exceeding one academic term.
  • D

    Documented Control Group

    • The control group is well-documented, including demographics and confirmation that they received no educational intervention.
    • The villages receiving the CHAMPION2 intervention acted as controls for the STRIPES2 trial... even though they were not receiving any educational intervention.
    • Relevant Quotes: 1) "Villages receiving the health intervention were controls for the education intervention and vice versa." (p. 1) 2) "The villages receiving the CHAMPION2 intervention acted as controls for the STRIPES2 trial... encouraging STRIPES2 control children to give us data and attend the midline and endline tests even though they were not receiving any educational intervention." (p. 4) 3) "Table 2. Baseline village characteristics... Table 3. Baseline child characteristics." (pp. 13-15) 4) "In the control arm 3684 Children... 3667 Children (after duplicate removal)." (p. 13) Detailed Analysis: The paper provides detailed documentation of the control group. It explains the specific nature of the control group (they received a health intervention, CHAMPION2, but no educational intervention). Tables 2 and 3 explicitly list the demographic characteristics, sample sizes, and baseline data for the control arm alongside the intervention arm. The study clearly states the control group did not receive the educational program. Criterion D is met because the control group's characteristics, size, and treatment (health intervention only) are clearly documented.

• Level 2 Criteria

  • S

    School-level RCT

    • Randomization occurred at the village level, which serves as the implementation unit, satisfying the school-level RCT criterion.
    • In this Madhya Pradesh cluster-randomized controlled trial, 196 villages (clusters) were randomized to receive either a health ... or education ... intervention.
    • Relevant Quotes: 1) "In this Madhya Pradesh cluster-randomized controlled trial, 196 villages (clusters) were randomized..." (p. 1) 2) "Randomization of clusters was performed... with stratification by village size..." (p. 9) Detailed Analysis: The study randomized at the "village" level. In the context of this rural education intervention, the village unit encompasses the local schools and the eligible child population. Randomizing villages is a higher level of aggregation than randomizing individual schools within a city or classes within a school, and it effectively prevents contamination at the school level. The intervention was community-based (before/after school classes in the village), making the village the appropriate "site" or "unit" of implementation. Criterion S is met because the randomization was conducted at the village level, which satisfies the school-level requirement.
  • I

    Independent Conduct

    • Data collection was conducted by an independent organization (GHTC) with blinded administrators, ensuring independent conduct.
    • All data collection and related research activities for the STRIPES2 trial were organized by independent teams recruited, trained and monitored by GH Training and Consultancy (GHTC), India.
    • Relevant Quotes: 1) "All data collection and related research activities for the STRIPES2 trial were organized by independent teams recruited, trained and monitored by GH Training and Consultancy (GHTC), India." (p. 4) 2) "The intervention was designed and conducted by Pratham Foundation..." (p. 2) 3) "GHTC recruited an independent team of test administrators for the Endline test. They were otherwise not involved in the trial and were unaware of the randomization." (p. 10) 4) "Formal analysis: Nicholas Magill, Suzanne Keddie, Peter Boone, Chris Frost." (p. 30) Detailed Analysis: The intervention was designed by the Pratham Foundation. The study's data collection was conducted by GHTC, an independent agency, using test administrators who were unaware of randomization. The analysis was performed by researchers from LSHTM and Effective Intervention (the funder), who are distinct from the Pratham implementation team, although they are co-authors. The criterion requires independent conduct to avoid bias in data collection/analysis. The use of an external agency (GHTC) for data collection and blinded test administrators explicitly meets the requirement for independent oversight. Criterion I is met because data collection was managed by an independent agency (GHTC) with blinded administrators.
  • Y

    Year Duration

    • The study assessed outcomes approximately 30 months after the intervention began, satisfying the year duration criterion.
    • The period of the trial was extended with the primary outcome ... assessed around 30 months after classes started.
    • Relevant Quotes: 1) "The period of the trial was extended with the primary outcome ... assessed around 30 months after classes started." (p. 1-2) 2) "In some intervention arm villages, classes ran for up to 23 months (October 2019 to March 2020, January 2021 to April 2021 and July 2021 to July 2022 (inclusive))..." (p. 13) 3) "Tests were conducted ... between 24 July and 19 September 2022." (p. 10) Detailed Analysis: The intervention start date was late 2019, and the final measurement was in mid-2022. The active implementation periods, even when separated by COVID-19 lockdowns, cumulatively and chronologically cover significantly more than one academic year. The follow-up period from start to finish was approximately 30 months. Criterion Y is met because the study tracked outcomes over a period of roughly 30 months, exceeding the one-year requirement.
  • B

    Balanced Control Group

    • The study explicitly tested the impact of the additional resources (para-instructor intervention) as the treatment variable.
    • The STRIPES2 trial was designed to assess whether a similar intervention ... would have similar effects ... and be cost-effective.
    • Relevant Quotes: 1) "The STRIPES2 trial was designed to assess whether a similar intervention ... would have similar effects ... and be cost-effective." (p. 1) 2) "For children newly enrolled in primary school, the STRIPES2 intervention comprised before/after-school classes (2 hours per day, 6 days a week) given by trained para-instructors..." (p. 1) 3) "Villages receiving the health intervention were controls for the education intervention... encouraging STRIPES2 control children to give us data... even though they were not receiving any educational intervention." (p. 4) Detailed Analysis: The intervention group received significant additional resources: para-instructors, daily classes, and learning materials. The control group received a health intervention but no educational resources ("not receiving any educational intervention"). To evaluate this against Criterion B, we apply the standard's logic: Is the additional resource (para-instructors/classes) the explicit treatment variable being tested? Yes. The study's stated goal is to "assess whether a similar intervention [the para-instructor model] ... would have similar effects." The additional resources are integral to the design and are the specific variable under investigation. Therefore, the control group represents the "business as usual" baseline required to measure the net effect of adding these resources. While there is an imbalance in educational inputs, it is intentional and integral to the research question. Criterion B is met because the additional resources (para-instructors and classes) were the explicit treatment variable being tested against a business-as-usual educational baseline.

• Level 3 Criteria

  • R

    Reproduced

    • The intervention methodology has been replicated by independent teams (e.g., J-PAL/Banerjee et al.) as cited in the paper.
    • Supplementary teaching with para-instructors and volunteers have shown large effects... India... [11,17]
    • Relevant Quotes: 1) "In the Support To Rural India's Public Education System (STRIPES) trial, a para-instructor intervention improved numeracy and literacy levels in Telangana, India (2008-10). The STRIPES2 trial was designed to assess whether a similar intervention ... would have similar effects..." (p. 1) 2) "Supplementary teaching with para-instructors and volunteers have shown large effects... India... [11,17...]" (p. 26) 3) Reference 11: "Banerjee A, Banerji R... Mainstreaming an effective intervention: evidence from randomized evaluations of 'teaching at the right level' in India. National Bureau of Economic Research; 2016" (p. 31) Detailed Analysis: STRIPES2 is a reproduction of the original STRIPES trial. While the STRIPES2 team includes original researchers, the paper explicitly links the intervention ("Teaching at the Right Level" / para-instructors) to other independent studies, specifically citing Banerjee et al. (2016) [cite_start][cite: 11] and Banerjee et al. (2007) [cite_start][cite: 17]. These citations refer to studies by the J-PAL team, which is a different research team from the Effective Intervention team leading STRIPES2, though they collaborate with the same implementation partner (Pratham). The intervention concept (TaRL/para-instructors) has been independently replicated in peer-reviewed journals. Criterion R is met because the intervention (Teaching at the Right Level/para-instructors) has been independently replicated and cited (e.g., Banerjee et al., 2016).
  • A

    All-subject Exams

    • The study assessed outcomes in Reading and Mathematics, covering the main subjects for the target grade levels.
    • The primary outcome was the arithmetic mean of the child's scores on EGRA (Early Grade Reading Assessment) and EGMA (Early Grade Mathematics Assessment) tests...
    • Relevant Quotes: 1) "The primary outcome was the arithmetic mean of the child's scores on EGRA (Early Grade Reading Assessment) and EGMA (Early Grade Mathematics Assessment) tests..." (p. 8) 2) "The STRIPES2 intervention aimed at building foundational literacy and numeracy skills for Grade 1 and 2 children..." (p. 4) Detailed Analysis: The study assessed Literacy (Reading) and Numeracy (Mathematics). For the target demographic (Grade 1 and 2 students in rural India), these represent the core "main subjects" of the curriculum. There is no evidence of other main subjects (like Science or Social Studies) being standard or neglected at this specific educational stage in this context. Therefore, measuring both Reading and Math covers the main subjects. Criterion A is met because the study assessed both literacy and numeracy, which are the main subjects for Grade 1-2 students.
  • G

    Graduation Tracking

    • The study explicitly states that no further follow-up was conducted after the intervention ended, failing the graduation tracking requirement.
    • We do not know if the effect of the intervention persisted after the intervention was completed as our study did not continue to conduct further follow-up.
    • Relevant Quotes: 1) "We do not know if the effect of the intervention persisted after the intervention was completed as our study did not continue to conduct further follow-up." (p. 27) 2) "Tests were conducted about one month after the intervention teams had stopped the main activities..." (p. 10) Detailed Analysis: The criterion requires tracking participants until graduation. The authors explicitly state that they did *not* conduct further follow-up after the endline tests, which occurred shortly after the intervention ended. The children were in primary school grades and were not tracked through to the end of primary school graduation. An external search for follow-up papers yielded no results for this specific STRIPES2 cohort as of the review date. Criterion G is not met because the authors explicitly state that no follow-up was conducted after the intervention ended.
  • P

    Pre-Registered

    • The paper cites a protocol published after the trial start but does not provide a specific pre-registration registry link or date in the text.
    • Relevant Quotes: 1) "Full details of the STRIPES2 trial design and planned statistical analysis can be found in the CHAMPION2/STRIPES2 protocol paper [20], protocol update [21] and statistical analysis plan [22]." (p. 3-4) 2) "The trial's protocol [20] and its amendment [21] were approved by the Ethics Committees..." (p. 4) Detailed Analysis: The paper cites a published protocol paper [20] (Agarwal et al., 2020) and an analysis plan. However, it does not provide a specific pre-registration registry ID (e.g., ISRCTN or ClinicalTrials.gov) or registration date in the text that definitively proves registration occurred before the initial data collection (enumeration) began in 2017 or 2019. While the trial likely has a registry entry (e.g., ISRCTN80596324), the paper does not confirm the timing relative to data collection start within the text provided, and strict adherence to the standard requires quoted evidence of pre-registration. Criterion P is not met because the paper does not provide a registry link or date in the text confirming pre-registration before data collection.