Replicability of sight word training and phonics training in poor readers: a randomised controlled trial

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Although randomization was not at the class level, the intervention was a fully individualized, at-home, computer-based program, satisfying the personal tutoring exception in the ERCT standard.
    • “Children were allocated to groups using minimisation randomization (balanced 1:1 for age, CC2 nonword reading, CC2 irregular word reading; executed using MINIMPY; Saghaei, 2011), which is considered the most appropriate sequence allocation procedure for trials comprising fewer than 100 participants.” (p. 10)
    • Relevant Quotes: 1) “Children were allocated to groups using minimisation randomization (balanced 1:1 for age, CC2 nonword reading, CC2 irregular word reading; executed using MINIMPY; Saghaei, 2011), which is considered the most appropriate sequence allocation procedure for trials comprising fewer than 100 participants.” (p. 10) 2) “All training was done online at home.” (p. 10) 3) “Children were asked to do five 30-minute sight-word training sessions per week for 8 weeks in their homes.” (p. 6) Detailed Analysis: This paper clearly states it is a randomized trial at the individual (student) level using “minimisation randomization.” However, the ERCT “Class-level RCT” criterion includes an exception for personal tutoring or one-to-one instruction. The intervention here was entirely individualized: children completed training at home, using online software, with no shared group or classroom setting. The activities were performed privately, and there was no opportunity for cross-contamination between groups. Therefore, this qualifies under the standard’s exception clause for personal tutoring. Final sentence explaining criterion C is met because this was an individualized, home-based intervention that avoids contamination risk, thus qualifying under the exception clause.
  • E

    Exam-based Assessment

    • The study employed the standardized TOWRE subtests for reading fluency, meeting the exam‑based assessment requirement.
    • “We indexed nonword reading fluency using the Test of Word Reading Efficiency (TOWRE) nonword subtest …” (p. 9)
    • Relevant Quotes: 1) “We indexed nonword reading fluency using the Test of Word Reading Efficiency (TOWRE) nonword subtest (Torgeson, Wagner & Rashotte, 1999).” (p. 9) 2) “This was tested with the TOWRE sight word subtest that comprised 104 words that increased in difficulty (Torgeson, Wagner & Rashotte, 1999).” (p. 9) Detailed Analysis: The study used the TOWRE subtests, which are widely recognized standardized exams for both word and nonword reading fluency, satisfying the ERCT ‘E - Exam-based Assessment’ requirement for objective, comparable measures. Final sentence explaining criterion E is met because standardized TOWRE exams were used to assess reading outcomes.
  • T

    Term Duration

    • Outcome assessments occurred after at least 16 weeks of training (and within a 6‑month participation period), exceeding the one‑term minimum requirement.
    • “Group 1 did 8 weeks of phonics training ... followed by 8 weeks of sight word training ...” (p. 4)
    • Relevant Quotes: 1) “Group 1 did 8 weeks of phonics training (and then Test 3) followed by 8 weeks of sight word training (and then Test 4). Group 2 did the same training in the reverse order.” (p. 4) 2) “Since it took 6 months for a child to complete the study, the last child completed the last test session (Test 4) in December 2013.” (p. 1) Detailed Analysis: Each child underwent two 8‑week intervention phases (16 weeks total), with final outcome measurement at Test 4, and participation spanning 24 weeks (~6 months), exceeding the minimum one‑term (~3–4 months) requirement of the ERCT ‘T - Term Duration’ criterion. Final sentence explaining criterion T is met because outcomes were measured at least one full academic term after intervention start.
  • D

    Documented Control Group

    • The study lacked a separate control group, relying on a within‑subject baseline, thus failing the documented control group requirement.
    • “After 8 weeks of no training, they returned to do the outcome measures (Test 2) to index ‘non-treatment gains’.” (p. 4)
    • Relevant Quotes: 1) “After 8 weeks of no training, they returned to do the outcome measures (Test 2) to index ‘non-treatment gains’.” (p. 4) Detailed Analysis: A within‑subject no‑training period was used instead of a separate control group. This does not satisfy the ERCT ‘D - Documented Control Group’ criterion, which requires a distinct, well‑documented control condition. Final sentence explaining criterion D is not met because no separate control group with its own documented characteristics was provided.

• Level 2 Criteria

  • S

    School-level RCT

    • The study randomized individual children rather than entire schools, so the school‑level RCT requirement is not met.
    • “Children were allocated to groups using minimisation randomization …” (p. 10)
    • Relevant Quotes: 1) “Children were allocated to groups using minimisation randomization …” (p. 10) Detailed Analysis: Randomization occurred at the individual child level, not among schools. The ERCT ‘S - School-level RCT’ criterion is therefore not met. Final sentence explaining criterion S is not met because schools were not randomized to conditions.
  • I

    Independent Conduct

    • The intervention was conducted and monitored by the same team that designed it, failing the independent conduct requirement.
    • “This allowed the research team to detect when a child was failing to complete ...” (p. 6)
    • Relevant Quotes: 1) “This allowed the research team to detect when a child was failing to complete the required amount of training, in which case the parents were contacted …” (p. 6) Detailed Analysis: The authors both designed and implemented the intervention, with no independent external evaluator. The ERCT ‘I - Independent Conduct’ criterion requires third‑party oversight, which is not evident here. Final sentence explaining criterion I is not met because the same team conducted and assessed the study without independent oversight.
  • Y

    Year Duration

    • Participants were observed for about 6 months, not a full academic year, so the year‑duration requirement is not met.
    • “Since it took 6 months for a child to complete the study ...” (p. 1)
    • Relevant Quotes: 1) “Since it took 6 months for a child to complete the study ...” (p. 1) Detailed Analysis: Participants were followed for approximately 24 weeks (6 months), shorter than the full academic year (~9–10 months) required by the ERCT ‘Y - Year Duration’ criterion. Final sentence explaining criterion Y is not met because tracking did not span a full academic year.
  • B

    Balanced Resources

    • The control condition had no training or additional support, so resources were not balanced.
    • “After 8 weeks of no training, they returned to do the outcome measures (Test 2) to index ‘non-treatment gains’.” (p. 4)
    • Relevant Quotes: 1) “After 8 weeks of no training, they returned to do the outcome measures (Test 2) to index ‘non-treatment gains’.” (p. 4) 2) “Group 1 did 8 weeks of phonics training ... followed by 8 weeks of sight word training. Group 2 did the same training in the reverse order.” (p. 4) Detailed Analysis: The training groups received extra instructional time that the control period did not have, violating the ERCT ‘B - Balanced Resources’ requirement for equal time and resources in control conditions. Final sentence explaining criterion B is not met because the control condition received no equivalent intervention or resource.

• Level 3 Criteria

  • R

    Reproduced Results

    • No independent research team has published a replication of this trial, so reproducibility is not established.
    • Relevant Quotes: None. Detailed Analysis: Although this paper is a replication of a prior trial, it was conducted by the same research team. No independent replication by a different team in a new context is reported, so the ERCT ‘R - Reproduced’ criterion is not met. Final sentence explaining criterion R is not met because no independent external replication is documented.
  • A

    All Exams

    • The study measured only reading skills without assessing other core subjects, so the all-subject exams requirement is not met.
    • “This was tested using the Test of Everyday Reading Comprehension (TERC) ...” (p. 9)
    • Relevant Quotes: 1) “Children were asked to read aloud 58 irregular words ...” (p. 9) 2) “We indexed nonword reading fluency using the Test of Word Reading Efficiency ...” (p. 9) 3) “This was tested using the Test of Everyday Reading Comprehension (TERC) ...” (p. 9) Detailed Analysis: All outcomes are reading‑specific. No assessments in mathematics, science, or other core subjects are included, failing the ERCT ‘A - All-subject Exams’ criterion. Final sentence explaining criterion A is not met because only reading outcomes were assessed.
  • G

    Graduation Tracking

    • Participants were followed for about 6 months, with no data collection continuing through graduation, failing the graduation tracking requirement.
    • “Since it took 6 months for a child to complete the study ...” (p. 1)
    • Relevant Quotes: 1) “Since it took 6 months for a child to complete the study ...” (p. 1) Detailed Analysis: Participants were tracked only during the intervention period (∼6 months). No follow‑up through graduation is reported, so the ERCT ‘G - Graduation Tracking’ criterion is not met. Final sentence explaining criterion G is not met because graduation tracking was not conducted.
  • P

    Pre-Registered Protocol

    • The trial was prospectively registered long before participants were enrolled, satisfying the pre-registered protocol requirement.
    • “This trial is registered with the Australian New Zealand Clinical Trials Registry (ANZCTR; 12608000454370).” (p. 1)
    • Relevant Quotes: 1) “This trial is registered with the Australian New Zealand Clinical Trials Registry (ANZCTR; 12608000454370).” (p. 1) Detailed Analysis: The ANZCTR registry shows the trial was registered on 15 September 2008 (prospectively) — well before recruitment of participants began in January 2011. This confirms that the study protocol was pre-registered prior to data collection. Final sentence explaining criterion P is met because the trial was registered before data collection started, ensuring a pre-specified protocol.