Results from a Phase 3 definitive trial of reciprocal reading in English high schools

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomisation was student-level, but the intervention is targeted small- group instruction outside normal English lessons, fitting the tutoring exception.
    • Individual student participants were randomised to condition.
    • Relevant Quotes: 1) "Up to 800 students were randomised to condition at the individual student level, recruited from 20 schools in England." (p. 4) 2) "Individual student participants were randomised to condition." (p. 6) 3) "The participating students aged 11 to 12 from sixth grade classes, engaged in Reciprocal Reading instruction in small groups of between 4-8, for 20-30 minutes once per week over two terms (approximately six months) during academic year 2021-2022." (p. 5) 4) "It was recommended that treatment group students (up to 20 students selected per school) receive the intervention outside of English instruction time, and the school timetable the intervention prior to the start of the implementation period." (p. 5) Detailed Analysis: Criterion C requires class-level (or stronger) randomisation to reduce contamination, but the ERCT standard provides an exception when the intervention is "personal teaching like tutoring". Here, students are pulled into small groups (4-8) for targeted sessions, recommended to be scheduled outside English instruction time. This is closer to a tutoring- style delivery model than a whole-class instructional change, making student-level randomisation acceptable under the exception. Final sentence explaining if criterion C is met because the study uses targeted small-group instruction outside normal English lessons, so the tutoring exception applies to student-level randomisation.
  • E

    Exam-based Assessment

    • Outcomes were measured using recognised standardised digital reading tests (NGRT and ART).
    • Primary outcome measure: New Group Reading Test (NGRT digital version) from GL-Assessment.
    • Relevant Quotes: 1) "Primary outcome measure: New Group Reading Test (NGRT digital version) from GL-Assessment." (p. 7) 2) "Secondary outcome measure: Access Reading Test (ART digital version) from RS Assessment, Hodder Education." (p. 7) Detailed Analysis: Both NGRT and ART are named, commercially available standardised reading assessments, and the paper describes them as adaptive tests with high reliability. The tests are not custom-built for this study, satisfying the ERCT requirement for standardised exam-based assessment. Final sentence explaining if criterion E is met because the study used recognised standardised assessments (NGRT and ART) rather than custom measures.
  • T

    Term Duration

    • The outcome measurement occurred after approximately two terms (about six months), exceeding the one-term minimum.
    • After approximately two terms of intervention delivery, students then completed both reading tests as a post-test at the end of the intervention.
    • Relevant Quotes: 1) "One 30-minute session is delivered per week over approximately 6 months (two terms)." (p. 4) 2) "After approximately two terms of intervention delivery, students then completed both reading tests as a post-test at the end of the intervention." (p. 7) Detailed Analysis: The intervention and the time from start to post-test are described as approximately two school terms (about six months). This is longer than the ERCT minimum of one academic term (typically 3-4 months) from intervention start to outcome measurement. Final sentence explaining if criterion T is met because outcomes were measured after about two terms (around six months) from intervention delivery.
  • D

    Documented Control Group

    • The control group and its business-as-usual condition were described, and the paper reports baseline equivalence plus detailed counterfactual information about interventions used with control students.
    • During implementation, the control group students (up to 20 per school) continued with business as usual, comprising of their normal literacy and specific comprehension instruction practices for students aged 11 to 12 years.
    • Relevant Quotes: 1) "During implementation, the control group students (up to 20 per school) continued with business as usual, comprising of their normal literacy and specific comprehension instruction practices for students aged 11 to 12 years." (p. 5) 2) "Staff from 11 schools reported specific intervention programmes used with control group students during the study." (p. 10) 3) "No significant differences were found between control and treatment groups at pre-test on all scales of the primary and secondary outcome measures:" (p. 7) Detailed Analysis: The paper describes the control condition as business as usual and gives a concrete counterfactual description by reporting survey-based information about literacy programmes used with control students. It also reports baseline equivalence at pre-test across primary and secondary outcome scales. Although demographic breakdowns are reported for the full analysed sample rather than by arm, the control condition, baseline status, and concurrent instruction are documented sufficiently for ERCT criterion D. Final sentence explaining if criterion D is met because the control condition is described as business as usual and is further documented via reported counterfactual programme use and baseline equivalence.

• Level 2 Criteria

  • S

    School-level RCT

    • Randomisation was at the individual student level, not the school level.
    • Individual student participants were randomised to condition.
    • Relevant Quotes: 1) "Up to 800 students were randomised to condition at the individual student level, recruited from 20 schools in England." (p. 4) 2) "Individual student participants were randomised to condition." (p. 6) Detailed Analysis: Although 20 schools participated, the unit of randomisation was the student within each school. Criterion S requires schools (not students) to be randomly assigned to conditions. Final sentence explaining if criterion S is not met because assignment was student-level rather than school-level.
  • I

    Independent Conduct

    • The intervention is attributed to Fischer Family Trust Literacy (FFTL), while the trial is authored by university researchers and funded by an independent foundation.
    • This approach was adapted into a targeted intervention by Fischer Family Trust Literacy (FFTL).
    • Relevant Quotes: 1) "This approach was adapted into a targeted intervention by Fischer Family Trust Literacy (FFTL)." (p. 1) 2) "Materials: A teacher training program delivered by the program developer, which includes external school training (for teachers and support staff) interspersed with follow up school support/training." (p. 4) 3) "The study was funded by a grant from the Nuffield Foundation [Grant reference number EDO/FR-000000363]." (p. 15) Detailed Analysis: The paper distinguishes the programme developer (FFTL) from the research team, and describes training delivered by the developer. The authors are affiliated with universities, and funding is from the Nuffield Foundation, supporting independence from the developer in evaluation and reporting. Final sentence explaining if criterion I is met because the intervention is developed by FFTL while the evaluation is conducted and funded independently of the developer.
  • Y

    Year Duration

    • Outcomes were measured after approximately two terms (about six months), which is shorter than a full academic year.
    • One 30-minute session is delivered per week over approximately 6 months (two terms).
    • Relevant Quotes: 1) "One 30-minute session is delivered per week over approximately 6 months (two terms)." (p. 4) 2) "After approximately two terms of intervention delivery, students then completed both reading tests as a post-test at the end of the intervention." (p. 7) Detailed Analysis: The tracking window from intervention delivery to post-test is explicitly described as about six months (two terms). This does not meet the ERCT requirement of at least one full academic year of tracking. Final sentence explaining if criterion Y is not met because measurement occurred after about two terms rather than a full academic year.
  • B

    Balanced Control Group

    • The intervention adds time, training, and materials, but these resources are integral to the intervention being tested against business as usual.
    • The program design includes both staff training and instruction delivery to students.
    • Relevant Quotes: 1) "The program design includes both staff training and instruction delivery to students." (p. 4) 2) "Staff teams received two training days focused on program delivery, with two additional support sessions (online) during implementation." (p. 4) 3) "One 30-minute session is delivered per week over approximately 6 months (two terms)." (p. 4) 4) "During implementation, the control group students (up to 20 per school) continued with business as usual, comprising of their normal literacy and specific comprehension instruction practices for students aged 11 to 12 years." (p. 5) Detailed Analysis: Extra resources are clearly present: (a) additional staff training and support sessions, and (b) additional weekly small-group instructional sessions over roughly six months. The control group is business as usual and is not described as receiving matched time or training. Under the ERCT criterion B decision rule, this can still be "met" if the extra time/training is integral to what the study explicitly tests. The quotes show the intervention is defined as a package that includes training plus targeted instruction delivery. Therefore, the added resources are not incidental, they are part of the intervention definition, and the study is testing the effect of adding this package compared to standard practice. Final sentence explaining if criterion B is met because added time and training are integral components of the intervention package being tested against business as usual.

• Level 3 Criteria

  • R

    Reproduced

    • I found no peer-reviewed independent replication of this specific high- school FFTL Reciprocal Reading evaluation by a different research team.
    • The same intervention was further adapted by the program developer for use with high school students and tested with positive results in a small Phase 2 RCT study involving 14 English high schools (Thurston et al., 2020).
    • Relevant Quotes: 1) "It was tested in 98 elementary schools with positive outcomes using randomised controlled trial (RCT) methodology (O'Hare et al., 2019)." (p. 1) 2) "The same intervention was further adapted by the program developer for use with high school students and tested with positive results in a small Phase 2 RCT study involving 14 English high schools (Thurston et al., 2020)." (p. 1-2) Detailed Analysis: The paper cites prior trials in primary schools and a Phase 2 RCT in high schools. These are not presented as independent replications by a separate research team of the Phase 3 evaluation reported here. After searching external sources, I did not find a peer-reviewed journal article by an independent team explicitly replicating this specific high- school FFTL Reciprocal Reading study and reporting comparable results. (I did find indications of new trials in progress, but not peer-reviewed replication results.) Final sentence explaining if criterion R is not met because no independent peer-reviewed replication of this specific evaluation was found.
  • A

    All-subject Exams

    • Only reading outcomes were assessed; impacts on all main subjects were not measured.
    • Primary outcome measure: New Group Reading Test (NGRT digital version) from GL-Assessment.
    • Relevant Quotes: 1) "Primary outcome measure: New Group Reading Test (NGRT digital version) from GL-Assessment." (p. 7) 2) "Secondary outcome measure: Access Reading Test (ART digital version) from RS Assessment, Hodder Education." (p. 7) Detailed Analysis: All reported outcome measures are reading assessments (NGRT and ART). There is no reporting of standardised outcomes in other main subjects (for example mathematics, science, or other curriculum areas). Criterion E is met, but criterion A requires coverage across main subjects. Final sentence explaining if criterion A is not met because the study measures only reading rather than all main subjects.
  • G

    Graduation Tracking

    • The study reports post-test at the end of the intervention and provides no evidence of tracking participants to graduation; also, criterion Y is not met so G cannot be met under the ERCT rules.
    • After approximately two terms of intervention delivery, students then completed both reading tests as a post-test at the end of the intervention.
    • Relevant Quotes: 1) "After approximately two terms of intervention delivery, students then completed both reading tests as a post-test at the end of the intervention." (p. 7) Detailed Analysis: The paper describes outcome measurement at post-test immediately at the end of the intervention window and does not describe any later follow-up to the end of schooling or graduation. I also searched for subsequent publications about this cohort and did not find evidence of graduation- level follow-up reporting. Additionally, the ERCT rules state that if criterion Y (Year Duration) is not met, then criterion G cannot be met. Since Y is not met here, G is not met regardless of whether later work might exist. Final sentence explaining if criterion G is not met because there is no reported tracking to graduation and the prerequisite year-duration criterion is not met.
  • P

    Pre-Registered

    • The trial was registered after enrolment had begun, so it does not meet the requirement for a prospectively pre-registered protocol.
    • Registration date 12/08/2021
    • Relevant Quotes: 1) "A research protocol was produced outlining the proposed design for this trial in accordance with the SPIRIT (2015) guidelines and was published prior to research completion in the International Journal of Educational Research (Cockerill et al., 2021)." (p. 3) Other Source Quotes (trial registry): 2) "Registration date 12/08/2021" (ISRCTN record, ISRCTN12090112, p. 1) 3) "Date of first enrolment 01/04/2021" (ISRCTN record, ISRCTN12090112, p. 4) Detailed Analysis: The paper reports that a protocol was produced and published, but ERCT criterion P requires that the study protocol be pre-registered before the study begins (prospectively), with registration timing prior to data collection/enrolment. The ISRCTN registry record for this trial reports a registration date of 12/08/2021, while the same record reports the date of first enrolment as 01/04/2021. Because registration occurs after enrolment began, the study is not prospectively pre-registered under the ERCT definition. Final sentence explaining if criterion P is not met because the registry shows registration after first enrolment, so the protocol was not prospectively pre-registered.