Accelerating Opportunity: The Effects of Instructionally Supported Detracking

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomization was primarily at the student level (not whole classes or schools), so the class-level randomization requirement is not satisfied.
    • "Students at three of the four campuses were individually randomized between the A1 Initiative and the business-as-usual condition."
    • Relevant Quotes: 1) "In its inaugural year, eligible ninth grade students were randomized into either the control condition (i.e., their conventional assignment to a remedial prealgebra class or to algebra, based on prior achievement) or to the treatment condition." 2) "Students at three of the four campuses were individually randomized between the A1 Initiative and the business-as-usual condition." 3) "At the only campus that did not randomize at the student level, an administrator used a two-stage process." Detailed Analysis: Criterion C requires that randomization occur at the class level (or stronger, school level) to reduce within-class contamination, unless the intervention is explicitly one-to-one tutoring (which is not the case here). The paper states that on three campuses students were "individually randomized" between treatment and business-as-usual. While one campus used a section-level approach, the study is not described as a class-randomized trial overall. Final summary sentence: Criterion C is not met because the study primarily uses student-level random assignment rather than class-level (or school-level) randomization.
  • E

    Exam-based Assessment

    • The primary achievement outcome uses California’s SBAC state math assessment, a standardized exam.
    • "Of these, we privileged the SBAC assessment administered under the California Assessment of Student Performance and Progress system in spring 2021, during our pilot cohort’s eleventh grade year."
    • Relevant Quotes: 1) "Math proficiency, our focal outcome, was measured using multiple sources of assessment data. Of these, we privileged the SBAC assessment administered under the California Assessment of Student Performance and Progress system in spring 2021, during our pilot cohort’s eleventh grade year." 2) "Scale scores were standardized using the published statewide mean and standard deviation (California Department of Eduction, 2023)." Detailed Analysis: Criterion E requires standardized, widely recognized exam-based assessments rather than researcher-created tests closely aligned to the intervention. The paper identifies the Smarter Balanced Assessment Consortium (SBAC) mathematics assessment used statewide in California as the privileged/focal measure of math proficiency (administered as part of the California Assessment of Student Performance and Progress system). The use of this statewide standardized exam satisfies the ERCT exam-based assessment requirement. Final summary sentence: Criterion E is met because the focal academic outcome is measured with the statewide SBAC math exam.
  • T

    Term Duration

    • Outcomes were measured long after the intervention started (e.g., grade 11 SBAC after starting in grade 9), exceeding a term-long follow-up.
    • "We observe enrollment, transcripts, test scores, and attendance for students in the pilot cohort from ninth grade (academic year [AY] 2019–20) through twelfth grade (AY 2022–23)."
    • Relevant Quotes: 1) "We observe enrollment, transcripts, test scores, and attendance for students in the pilot cohort from ninth grade (academic year [AY] 2019–20) through twelfth grade (AY 2022–23)." 2) "Of these, we privileged the SBAC assessment administered under the California Assessment of Student Performance and Progress system in spring 2021, during our pilot cohort’s eleventh grade year." Detailed Analysis: Criterion T requires that outcomes be measured at least one academic term after the intervention begins. The intervention begins in ninth grade (AY 2019–20), and the focal standardized outcome (SBAC) is measured in spring 2021 during eleventh grade. This is far beyond a single academic term after the start of the intervention, and the paper also reports tracking through twelfth grade. Final summary sentence: Criterion T is met because outcomes are measured years after the ninth-grade start of the intervention.
  • D

    Documented Control Group

    • The control condition is clearly described (business-as-usual placement into Algebra Readiness or standard Algebra tracks) and the paper provides baseline/control-group descriptive detail.
    • "Below-grade-level students not assigned to the A1 Initiative were enrolled in a prealgebra remedial course called Algebra Readiness."
    • Relevant Quotes: 1) "In its inaugural year, eligible ninth grade students were randomized into either the control condition (i.e., their conventional assignment to a remedial prealgebra class or to algebra, based on prior achievement) or to the treatment condition." 2) "Below-grade-level students not assigned to the A1 Initiative were enrolled in a prealgebra remedial course called Algebra Readiness." 3) "Table 3 reports summary statistics for the ITT sample of 1,039 students on ninth grade entry." Detailed Analysis: Criterion D requires that the control group be well documented, including what instruction/services they received and baseline characteristics to support comparability checks. The paper clearly defines the business-as-usual control condition as conventional placement into remedial prealgebra (Algebra Readiness) or Algebra tracks based on prior achievement, and it provides descriptive statistics for the randomized (ITT) sample at ninth-grade entry. Final summary sentence: Criterion D is met because the paper describes the control conditions and reports baseline descriptive information for the randomized sample.

• Level 2 Criteria

  • S

    School-level RCT

    • Randomization occurred within campuses and often at the student level, not by assigning entire schools to treatment/control.
    • "Students at three of the four campuses were individually randomized between the A1 Initiative and the business-as-usual condition."
    • Relevant Quotes: 1) "Students at three of the four campuses were individually randomized between the A1 Initiative and the business-as-usual condition." 2) "At the only campus that did not randomize at the student level, an administrator used a two-stage process." Detailed Analysis: Criterion S requires school-level randomization (i.e., whole schools/sites assigned to treatment vs control). The paper describes student-level randomization on three campuses and within-school section-level reassignment on one campus. There is no indication that entire schools were randomized to implement the A1 Initiative versus remain fully business-as-usual. Final summary sentence: Criterion S is not met because whole schools were not randomized to treatment and control.
  • I

    Independent Conduct

    • The paper describes a district-led reform evaluated using district administrative data and administrator-run assignment, indicating the evaluators were not the intervention delivery team.
    • "We used administrative data from the district to examine the effects of the A1 Initiative on an array of student outcomes."
    • Relevant Quotes: 1) "We used administrative data from the district to examine the effects of the A1 Initiative on an array of student outcomes." 2) "From interviews, we knew that randomization had been conducted by campus-level administrators during the course-assignment process preceding the fall 2019 academic term." Detailed Analysis: Criterion I requires that the study be conducted independently from the designers/providers of the intervention, reducing risks of biased implementation, measurement, and reporting. The paper frames the A1 Initiative as a district reform (with administrator-run assignment) and describes the evaluation as relying on district administrative data. This supports that the program delivery and the evaluation/data analysis roles are separated, with administrators handling assignment and researchers analyzing outcomes. Final summary sentence: Criterion I is met because the paper presents the initiative as district-run and describes an administrator-run assignment process with evaluation using administrative records.
  • Y

    Year Duration

    • The study tracks outcomes from grade 9 through grade 12, which is longer than 75% of an academic year.
    • "We observe enrollment, transcripts, test scores, and attendance for students in the pilot cohort from ninth grade (academic year [AY] 2019–20) through twelfth grade (AY 2022–23)."
    • Relevant Quotes: 1) "We observe enrollment, transcripts, test scores, and attendance for students in the pilot cohort from ninth grade (academic year [AY] 2019–20) through twelfth grade (AY 2022–23)." Detailed Analysis: Criterion Y requires outcomes be measured at least 75% of one academic year after the intervention begins. The paper explicitly states that students are observed from ninth grade through twelfth grade, which necessarily exceeds the minimum one-year duration requirement by multiple years. Final summary sentence: Criterion Y is met because outcomes are tracked across multiple academic years (grades 9–12).
  • B

    Balanced Control Group

    • The treatment includes substantial added teacher supports and planning resources that are explicitly integral to the tested intervention package, so the imbalance is by design.
    • "The A1 Initiative classrooms therefore featured both heterogeneously grouped students and teachers who received unique professional development (e.g., on strategies for instructional differentiation) and additional resources (e.g., more planning time)."
    • Relevant Quotes: 1) "This random-assignment partnership study examined an innovative district-level reform—the Algebra I Initiative—that placed ninth grade students with prior math scores below grade level into Algebra I classes coupled with teacher training instead of a remedial pre-algebra class." 2) "The A1 Initiative classrooms therefore featured both heterogeneously grouped students and teachers who received unique professional development (e.g., on strategies for instructional differentiation) and additional resources (e.g., more planning time). Although we cannot disentangle the separate effects of each component, the bundle of interventions that comprises this novel program was piloted under random assignment." 3) "In addition to ~15 full days of professional development, they received an additional class-section release (i.e., equivalent to five additional planning periods per week), four coaching days per site per semester, a districtwide professional learning community, and a partner teacher at their campus." 4) "Control group students nearly at grade-level proficiency enrolled in Algebra I and a second (also not UC-CSU aligned) block of math instruction called Support." Detailed Analysis: Criterion B evaluates whether time/resources are balanced across treatment and control, unless additional resources are explicitly integral to the treatment being tested (i.e., the study is testing the effect of providing those additional resources). Here, the intervention is explicitly defined as "instructionally supported detracking": moving students into Algebra I "coupled with teacher training" and providing teachers "unique professional development" and "additional resources" such as planning time and coaching. The paper also characterizes the program as a "bundle of interventions" piloted under random assignment, indicating these added resources are part of the treatment package rather than a separable confound. The study also documents that, for one baseline-achievement group, the control condition included an additional "Support" math block (double-dose math time). This difference runs in the opposite direction (control has more student time), and does not undermine the conclusion that the key added teacher resources in treatment are integral to the intervention being evaluated. Final summary sentence: Criterion B is met because the additional teacher training, coaching, and planning resources are explicitly integral components of the treatment package being tested.

• Level 3 Criteria

  • R

    Reproduced

    • No independent replication by a different research team in another context was found in the paper or via internet search.
    • Relevant Quotes: None found. Detailed Analysis: Criterion R requires evidence that an independent research team replicated this study (or closely replicated its intervention and evaluation design) in another context and reported results in a peer-reviewed outlet. The paper reports a single randomized evaluation of one district’s Algebra I Initiative. An internet search for replication studies describing themselves as replications of Dee and Huffaker’s A1 Initiative evaluation did not identify any such independent replication as of the ERCT check date. Final summary sentence: Criterion R is not met because no independent replication study was found.
  • A

    All-subject Exams

    • The study’s standardized exam reporting is not a comprehensive all-core-subject exam set (it centers on math; other subjects are not uniformly covered with standardized exams).
    • "Math proficiency, our focal outcome, was measured using multiple sources of assessment data. Of these, we privileged the SBAC assessment administered under the California Assessment of Student Performance and Progress system in spring 2021, during our pilot cohort’s eleventh grade year."
    • Relevant Quotes: 1) "Math proficiency, our focal outcome, was measured using multiple sources of assessment data. Of these, we privileged the SBAC assessment administered under the California Assessment of Student Performance and Progress system in spring 2021, during our pilot cohort’s eleventh grade year." 2) "We also observed a negative impact from A1 Initiative assignment on the English language arts eleventh grade SBAC test for the nearly-at-grade-level group." Detailed Analysis: Criterion A requires standardized exam-based assessment coverage across all main subjects, and it also requires criterion E to be met (criterion E is met here via SBAC). The paper clearly establishes math as the focal standardized exam outcome (SBAC math). While it references an SBAC English language arts finding when discussing spillovers, it does not present standardized exam impacts across all core subjects as a comprehensive outcome set (e.g., it does not provide an all-core- subject standardized exam battery as a primary outcome framework). Final summary sentence: Criterion A is not met because the study does not report standardized exam outcomes across all main subjects.
  • G

    Graduation Tracking

    • The study follows students through grade 12 and includes an on-time graduation indicator, satisfying graduation tracking.
    • "Twelfth-grade data also include an exit indicator for on-time graduation status."
    • Relevant Quotes: 1) "We observe enrollment, transcripts, test scores, and attendance for students in the pilot cohort from ninth grade (academic year [AY] 2019–20) through twelfth grade (AY 2022–23)." 2) "Twelfth-grade data also include an exit indicator for on-time graduation status." Detailed Analysis: Criterion G requires tracking participants through graduation from the relevant educational stage. The paper explicitly states that the cohort is observed through twelfth grade (AY 2022–23) and that twelfth-grade data include an indicator for on-time graduation status. This constitutes tracking through high school graduation for the cohort. Final summary sentence: Criterion G is met because students are tracked through grade 12 with an explicit on-time graduation outcome.
  • P

    Pre-Registered

    • The paper mentions preregistered questions but does not provide a registry link/ID and no verifiable pre-data-collection registration timing was found.
    • "Preregistered confirmatory research question: For each baseline achievement level, what is the impact of assignment to the A1 Initiative on a standardized test score?"
    • Relevant Quotes: 1) "Preregistered confirmatory research question: For each baseline achievement level, what is the impact of assignment to the A1 Initiative on a standardized test score?" 2) "Because this study was originally intended to run through the pilot cohort’s tenth grade year, our preregistration plan focused on an 11-item district-constructed assessment (i.e., the ICA) taken in the fall following the pilot year with the intent that a longer- term follow-up study would have a new pre-registration." Detailed Analysis: Criterion P requires that the full protocol be publicly pre-registered before data collection begins, with a registry platform and identifier (and/or link) plus a registration date that can be verified as preceding study start. Although the paper states that the confirmatory question was preregistered and references "our preregistration plan", the text provided does not include a registry name (e.g., OSF, AEA RCT Registry), a registration ID/link, or a registration date. A targeted internet search for a public preregistration record for this study did not locate a verifiable registry entry that could be checked against the study start. Final summary sentence: Criterion P is not met because a verifiable public preregistration record (ID/link and date before study start) was not identified.