A Randomized Controlled Trial of the Impact of the Fusion Reading Intervention on Reading Achievement and Motivation for Adolescent Struggling Readers

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomization was conducted at the individual student level (blocked by school and grade), not at the class or school level, and the small-group format does not qualify for the personal-tutoring exception.
    • "Blocking on schools and grade level, we randomized eligible students with parent consent in grades 6 through 10 to either the intervention or control condition." (p. A-2)
    • Relevant Quotes: 1) "Blocking on schools and grade level, we randomized eligible students with parent consent in grades 6 through 10 to either the intervention or control condition." (p. A-2) 2) "Struggling students are enrolled in the supplemental intervention for one class period for 5 days a week. The developers ... recommend no more than 15 students per class." (p. A-2) 3) "Both Fusion and control students participated in regular English language arts (ELA) classes at their school." (p. A-2) Detailed Analysis: The paper explicitly states that randomization was performed at the individual student level, using school and grade only as blocking (stratification) variables, not as the unit of assignment itself. This means that within the same school and grade, some students were assigned to Fusion and others to control, creating a design vulnerable to the exact contamination concerns criterion C is meant to prevent (e.g., shared regular ELA classes, informal exposure to peers' strategies). The ERCT standard allows an exception for interventions designed as personal, one-to-one tutoring. Fusion Reading is delivered in a supplemental class of up to 15 students, which is small-group instruction rather than one-to-one personal tutoring, so the exception does not apply here. Final sentence: Criterion C is not met because randomization occurred at the student level rather than the class or school level, and the group-based delivery format does not meet the tutoring exception.
  • E

    Exam-based Assessment

    • The study measured reading outcomes using TOWRE, GRADE, and Michigan's MEAP, all of which are widely recognized standardized assessments, not custom tests.
    • "The dependent variables were reading achievement measures [TOWRE Sight Word Efficiency (SWE) and Phonetic Decoding Efficiency (PDE) subtests; The Group Reading Assessment and Diagnostic Evaluation (GRADE) passage comprehension, sentence comprehension, and vocabulary subtests; and Michigan's MEAP reading achievement]" (p. A-3)
    • Relevant Quotes: 1) "The dependent variables were reading achievement measures [TOWRE Sight Word Efficiency (SWE) and Phonetic Decoding Efficiency (PDE) subtests; The Group Reading Assessment and Diagnostic Evaluation (GRADE) passage comprehension, sentence comprehension, and vocabulary subtests; and Michigan's MEAP reading achievement]" (p. A-3) 2) "Eligible students are those who: (1) scored between the 5th to 35th percentiles on the Test of Silent Contextual Reading Fluency (TOSCRF)..." (p. A-2) Detailed Analysis: TOWRE (Test of Word Reading Efficiency) and GRADE (Group Reading Assessment and Diagnostic Evaluation) are both commercially published, nationally normed, standardized reading assessments used broadly beyond this study. MEAP is Michigan's official state standardized achievement exam. None of these instruments were custom-built for this evaluation; they were also used for study eligibility screening (TOSCRF), which further indicates reliance on established, validated tools rather than researcher-designed measures. Final sentence: Criterion E is met because the study used established, widely recognized standardized exams (TOWRE, GRADE, MEAP) rather than a custom-made test.
  • T

    Term Duration

    • Outcomes were measured after roughly a full academic year (2010-11) of intervention, well beyond the one-term minimum required by this weaker criterion.
    • "Students in the intervention condition received the Fusion Reading intervention as a supplemental reading intervention in the 2010-11 school year..." (p. A-2)
    • Relevant Quotes: 1) "Students in the intervention condition received the Fusion Reading intervention as a supplemental reading intervention in the 2010-11 school year, whereas students in the control condition engaged in nonliteracy, 'business-as-usual' activities." (p. A-2) 2) "After one year of implementation of a two year intervention, we learned that ... word reading outcomes will significantly improve compared to control middle and high school students." (p. A-4) Detailed Analysis: The abstract frames the reported results as covering "one year" of the Fusion Reading intervention, delivered across the full 2010-11 school year, with the posttest administered at the conclusion of that year (per the attrition-at-posttest flow in Figure 2). This interval substantially exceeds the minimum one-term (3-4 month) requirement. Final sentence: Criterion T is met because the reported interval from intervention start to outcome measurement spans close to a full academic year, well above the one-term minimum; this is also automatically satisfied because the stronger Year Duration criterion (Y) is met.
  • D

    Documented Control Group

    • Table 1 provides detailed demographic and baseline reading/motivation data for the control group, including sample sizes and equivalence tests.
    • "Table 1 presents the student background characteristics and baseline equivalence test results of the participants in the intervention and comparison groups." (p. A-3)
    • Relevant Quotes: 1) "Table 1 presents the student background characteristics and baseline equivalence test results of the participants in the intervention and comparison groups. HLM and Chi-square tests indicated that Fusion participants were not significantly different from control students on demographics or baseline reading measures; however, control students had higher CAIMI reading scores than Fusion students at baseline (t = -1.89, p = .059)." (pp. A-3 to A-4) 2) "students in the control condition engaged in nonliteracy, 'business-as-usual' activities." (p. A-2) 3) Table 1 additionally reports, for each background and baseline-achievement variable, the treatment and control group percentages/means, standard deviations, and sample sizes (e.g., Male 53.71% N=152 vs. 56.55% N=164; African American 81.27% N=230 vs. 80.34% N=233; TOWRE SWE M=89.63 SD=9.50 N=279 vs. M=89.73 SD=9.90 N=290), alongside chi-square/t statistics and p-values for each comparison (Table 1). Detailed Analysis: Table 1 provides an extensive baseline comparison of gender, race/ethnicity, disability status, and multiple reading achievement/motivation measures (TOWRE, GRADE, MEAP, CAIMI) for both the intervention and control groups, with sample sizes and statistical tests of baseline equivalence for each characteristic. The text also documents what the control condition consisted of (nonliteracy, business-as-usual activities). Final sentence: Criterion D is met because the control group's demographic and baseline achievement characteristics, size, and activities are documented in detail in Table 1 and the surrounding text.

• Level 2 Criteria

  • S

    School-level RCT

    • The study randomized individual students (blocked by school and grade), not entire schools, so the stronger school-level RCT criterion is not satisfied.
    • "Blocking on schools and grade level, we randomized eligible students with parent consent in grades 6 through 10 to either the intervention or control condition." (p. A-2)
    • Relevant Quotes: 1) "Blocking on schools and grade level, we randomized eligible students with parent consent in grades 6 through 10 to either the intervention or control condition." (p. A-2) 2) "Michigan State Department of Education (MSDE) recruited seven schools for this study by inviting schools..." (p. A-2) Detailed Analysis: Seven schools were recruited to participate, but the actual random assignment to intervention or control condition happened among individual eligible students within each school/grade block, not among whole schools. All seven schools contain both intervention and control students, so this is not a school-level RCT. Final sentence: Criterion S is not met because randomization occurred at the student level within participating schools, not between whole schools.
  • I

    Independent Conduct

    • The evaluation was conducted by SRI International researchers, independent of the University of Kansas team that developed the Fusion Reading curriculum.
    • "Authors and Affiliations: Ellen Schiller, Xin Wei, Sara Thayer, Jose Blackorby, Harold Javitz, and Cyndi Williamson, SRI International, Menlo Park, CA, USA" (title page)
    • Relevant Quotes: 1) "Authors and Affiliations: Ellen Schiller, Xin Wei, Sara Thayer, Jose Blackorby, Harold Javitz, and Cyndi Williamson, SRI International, Menlo Park, CA, USA" (title page) 2) "The developers (Brasseur, Hock, & Deshler, 2010a, 2010b, 2010c; Hock, Brasseur, & Deshler, 2010a, 2010b, 2010c, 2010d, 2010e) recommend no more than 15 students per class." (p. A-2) Detailed Analysis: The paper itself distinguishes between "the developers" of Fusion Reading (the University of Kansas team: Brasseur, Hock, and Deshler) and the study's authors (SRI International researchers), who are not credited as developers anywhere in the text. Internet searching (ERIC, SRI International's own publication listing) corroborates that this evaluation was authored solely by SRI International staff and was funded through the federal Striving Readers grant program, while Fusion Reading itself was developed at the University of Kansas Center for Research on Learning (Deshler, Hock, Brasseur), a separate institution from SRI. No named developer appears among the study's authors. This separation of intervention designers from evaluators supports independent conduct, though the abstract itself does not contain an explicit "independent evaluator" disclosure statement. Final sentence: Criterion I is met because the study was conducted by SRI International, an organization distinct from the University of Kansas developers of the Fusion Reading intervention.
  • Y

    Year Duration

    • The reported outcomes reflect roughly a full academic year (2010-11) of intervention, satisfying the 75%-of-a-year threshold.
    • "This study estimates the effect of one year of Fusion Reading implementation..." (Background/Context)
    • Relevant Quotes: 1) "This study estimates the effect of one year of Fusion Reading implementation, a multistrategy intervention..." (Background/Context) 2) "Students in the intervention condition received the Fusion Reading intervention as a supplemental reading intervention in the 2010-11 school year..." (p. A-2) 3) "After one year of implementation of a two year intervention, we learned that ... word reading outcomes will significantly improve..." (p. A-4) Detailed Analysis: The intervention and outcome measurement are framed around "one year" of implementation during the full 2010-11 school year. This abstract does not give exact calendar start/end dates for the intervention or posttest (it is a short SREE conference abstract, not the full technical report), but the consistent framing of "one year of Fusion Reading implementation" and results measured "after one year" strongly implies tracking across nearly the entire academic year, which would satisfy the 75%-of-academic-year threshold. Final sentence: Criterion Y is met based on the "one year of implementation" framing spanning the 2010-11 school year, though exact calendar dates are not given in this abstract-length report.
  • B

    Balanced Control Group

    • The extra Fusion class period is the vehicle for the strategy-instruction content being tested, and the control group occupied an equivalent scheduled activity slot ("business-as-usual"), so time allocation is reasonably balanced under the criterion B decision tree.
    • "Students in the intervention condition received the Fusion Reading intervention as a supplemental reading intervention in the 2010-11 school year, whereas students in the control condition engaged in nonliteracy, 'business-as-usual' activities." (p. A-2)
    • Relevant Quotes: 1) "Students in the intervention condition received the Fusion Reading intervention as a supplemental reading intervention in the 2010-11 school year, whereas students in the control condition engaged in nonliteracy, 'business-as-usual' activities." (p. A-2) 2) "Both Fusion and control students participated in regular English language arts (ELA) classes at their school." (p. A-2) 3) "Struggling students are enrolled in the supplemental intervention for one class period for 5 days a week." (p. A-2) Detailed Analysis: Re-applying the criterion B decision tree under the current spec: extra instructional resources ARE present (intervention students receive a dedicated supplemental class period, 5 days/week, beyond their regular ELA class). The treatment being tested is the Fusion strategy curriculum delivered within that period, not "extra time" as an explicit standalone variable, so the RESOURCES_ARE_TREATMENT branch does not clearly apply. The key question is therefore whether the control group received comparable scheduled time. The text states control students "engaged in nonliteracy, 'business-as-usual' activities" rather than simply receiving no scheduled activity, which indicates they too occupied a corresponding slot, just with different (non-reading-strategy) content; both groups also continued their shared regular ELA classes, so the comparison isolates instructional content rather than raw additional time. This wording supports CONTROL_MATCHES_RESOURCES being satisfied, but the description is brief (this is a short conference abstract) and does not quantify the control activities' duration, so exact time-matching cannot be confirmed to the minute; internet searching did not locate a fuller technical report clarifying this further under the ED535544 record specifically. Final sentence: Criterion B is met because the control condition's "business-as-usual" activities appear to occupy a corresponding scheduled time slot to the Fusion class, isolating instructional content as the treatment variable rather than raw additional time, though the abstract's brevity limits precise time quantification.

• Level 3 Criteria

  • R

    Reproduced

    • No independent replication of this specific study by a different research team was found in the paper or in external literature searches.
    • Relevant Quotes: 1) "This paper reports the effect of the first year of reading intervention." (p. A-2) — no mention of prior or subsequent independent replications by another team. Detailed Analysis: The paper does not reference any independent replication of this specific Fusion Reading RCT. Internet searches (ERIC database, Semantic Scholar) located a related publication: Hock, M. F., Brasseur-Hock, I. F., Hock, A. J., & Duvel, B. (2017). "The Effects of a Comprehensive Reading Program on Reading Outcomes for Middle School Students with Disabilities." Journal of Learning Disabilities, 50(2), 195-212 (ERIC EJ1129864). That study examined the Fusion Reading program with middle school students with disabilities in Kansas, but it does not constitute a qualifying replication: (a) it was authored by Hock and Brasseur-Hock, the same University of Kansas team that developed Fusion Reading, not an independent research team, and (b) it used "a quasi-experimental design with matched groups," not a randomized controlled trial. No independent, peer-reviewed RCT replication of this specific Michigan SRI International trial's design and findings was located via ERIC or Semantic Scholar searches. Final sentence: Criterion R is not met because no independent replication of this specific study by a separate research team was found.
  • A

    All-subject Exams

    • The study measured only reading-related outcomes (TOWRE, GRADE, MEAP reading, CAIMI reading motivation); no other core subjects were assessed.
    • "The dependent variables were reading achievement measures [TOWRE ... GRADE ... and Michigan's MEAP reading achievement] and reading motivation [the Children's Academic Intrinsic Motivation Inventory (CAIMI)]." (p. A-3)
    • Relevant Quotes: 1) "The dependent variables were reading achievement measures [TOWRE Sight Word Efficiency (SWE) and Phonetic Decoding Efficiency (PDE) subtests; The Group Reading Assessment and Diagnostic Evaluation (GRADE) passage comprehension, sentence comprehension, and vocabulary subtests; and Michigan's MEAP reading achievement] and reading motivation [the Children's Academic Intrinsic Motivation Inventory (CAIMI)]." (p. A-3) Detailed Analysis: All outcome measures in this study are reading-specific (word reading efficiency, decoding, passage/sentence comprehension, vocabulary, state reading exam, and reading motivation). No mathematics, science, or other core-subject standardized outcomes are reported. While Fusion Reading is inherently a reading-focused intervention, the paper does not provide a rationale for why other core subjects were not assessed (the "highly specialized upper-secondary/vocational" exception does not clearly apply here, since this targets general middle/high school struggling readers, not a specialized vocational track), so the exception is not invoked. Final sentence: Criterion A is not met because only reading outcomes were assessed, with no measurement of other core subjects or justification for the omission.
  • G

    Graduation Tracking

    • The study reports only first-year outcomes with no tracking to graduation, and no follow-up publications tracking this cohort were found via internet search.
    • "This paper reports the effect of the first year of reading intervention." (p. A-2)
    • Relevant Quotes: 1) "This paper reports the effect of the first year of reading intervention." (p. A-2) 2) "Future research is needed to fully understand whether the intended two year intervention will improve struggling adolescent's reading comprehension outcomes." (p. A-4) Detailed Analysis: The study explicitly limits its reporting to the first year of a planned two-year intervention and does not describe any follow-up tracking of students through to graduation. The concluding statement calls for "future research" on the two-year intervention but does not describe graduation-level tracking. Internet searches (ERIC database queries for "Fusion Reading," "Striving Readers" Michigan, and the individual author names Wei and Blackorby) did not identify a subsequent publication by this SRI International team that follows this specific Michigan cohort into year two, or through graduation. Since criterion Y is met but no graduation tracking exists, this criterion remains not met. Final sentence: Criterion G is not met because tracking stopped after one year of intervention, with no evidence of follow-up through graduation in this paper or in later publications located via internet search.
  • P

    Pre-Registered

    • No pre-registration statement, registry ID, or registration date is provided in the paper, and none was found via external search of trial registries.
    • Relevant Quotes: No quotes in the paper reference a study registry, pre-registration platform, or registration date for this trial. Detailed Analysis: This SREE conference abstract contains no statement about pre-registration of the study's hypotheses, methods, or analysis plan on any registry platform (e.g., ClinicalTrials.gov, AEA RCT Registry, IES/NCER registry). An internet search of ClinicalTrials.gov and general registry sources for a trial registration record tied to this specific Michigan Fusion Reading Striving Readers evaluation (conducted 2010-2012, before pre-registration of education RCTs became a common practice) did not locate one. Final sentence: Criterion P is not met because there is no evidence, in the paper or via external search, of a pre-registered protocol for this study.