Assessing the Effect of Corequisite English Instruction Using a Randomized Controlled Trial

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomization was conducted at the individual student level rather than by class or school, failing the class-level RCT requirement.
    • "Students were randomly assigned to either corequisite remediation ... or the traditional semester-long integrated reading and writing DE course." (p. 4)
    • Relevant Quotes: 1) "We recruited and consented 1,276 newly enrolling students over three semesters ... Students were randomly assigned to either corequisite remediation ... or the traditional semester-long integrated reading and writing DE course." (p. 4) 2) "At two of the five colleges ... students were pre-randomized ... at the other three study colleges ... students ... were randomized through the survey platform after consent. Students who opted to participate were randomized to corequisite remediation or to the highest level standalone IRW DE course." (p. 11) Detailed Analysis: The ERCT Standard’s Class-level RCT criterion requires that entire classrooms or schools be randomized to intervention or control. Here, randomization occurred at the individual student level within advising sessions. There is no indication that whole class sections or institutions, rather than individual students, served as the unit of assignment. Final Summary: Criterion C is not met because the study randomized individual students rather than entire classes or schools.
  • E

    Exam-based Assessment

    • Outcomes were measured via course grades and credits, not through a recognized standardized examination.
    • "To assess course passing, grades were recorded on a standard A–F basis in THECB data, and we defined passing as receiving a grade of C or better."
    • Relevant Quotes: 1) "To assess course passing, grades were recorded on a standard A–F basis in THECB data, and we defined passing as receiving a grade of C or better." (p. 15) Detailed Analysis: The study’s outcome measures rely on institutional course grades and credit accumulation rather than on a recognized standardized examination instrument. There is no mention of employing a statewide or national standardized test. Final Summary: Criterion E is not met because no standardized exam-based assessment was used.
  • T

    Term Duration

    • The intervention courses ran for a full 16-week semester, meeting the term duration requirement.
    • "Instructional time for the college course ... ran for 16 weeks for all colleges except College E."
    • Relevant Quotes: 1) "Instructional time for the college course was common across colleges (48 instructional hours), and the course ran for 16 weeks for all colleges except College E. College E used eight-week terms ..." (p. 10) Detailed Analysis: The intervention lasted a full academic semester (16 weeks) at most colleges, satisfying the minimum term-long duration requirement of the ERCT Standard. Final Summary: Criterion T is met because the corequisite courses spanned a full 16-week semester.
  • D

    Documented Control Group

    • The control group’s composition, consent rates, and baseline covariates are documented in tables and text, satisfying documentation requirements.
    • "Students who opted to participate were randomized to corequisite remediation or to the highest level standalone IRW DE course ... We refer to ... the 'control group'."
    • Relevant Quotes: 1) "We recruited and consented 1,276 newly enrolling students ... Students were randomly assigned ... We refer to those randomized to corequisite remediation as the 'treatment group' and those randomized to the stand-alone IRW course as the 'control group'." (p. 4) 2) "Differences in baseline covariates by treatment status remain small in magnitude, and none of the differences are statistically different from zero." (p. 18) Detailed Analysis: The paper clearly documents the composition and size of the control group, including consent rates, enrollment percentages, and covariate balance. Baseline characteristics for control students are presented in tabular detail. Final Summary: Criterion D is met because the control group’s characteristics and baseline performance are thoroughly documented.

• Level 2 Criteria

  • S

    School-level RCT

    • The study randomized individual students rather than entire schools, failing the school-level RCT requirement.
    • "We conducted a student-level RCT ... Students ... were randomized to corequisite remediation or to control."
    • Relevant Quotes: 1) "We conducted a student-level RCT. All first-time-in-college students scoring within ... were recruited ... during orientation or initial advising sessions." (p. 12) Detailed Analysis: The ERCT Standard’s School-level RCT criterion requires entire schools to be randomized. This study randomized individual students within each of five colleges, not whole institutions. Final Summary: Criterion S is not met because randomization was at the student rather than school level.
  • I

    Independent Conduct

    • The study was conducted by researchers independent from the designers of the corequisite models, satisfying the ERCT requirement for Criterion I.
    • "All of the participating colleges volunteered to participate in the study and had established their own approaches to corequisite remediation prior to participation." (page 84)
    • Relevant Quotes: 1) "All of the participating colleges volunteered to participate in the study and had established their own approaches to corequisite remediation prior to participation." (page 84) 2) "The research reported here was supported, in whole or in part, by the Institute of Education Sciences, U.S. Department of Education, through grants [R305H170085 and R305N170003] to the American Institutes for Research and [R305H150094] to the RAND Corporation." (page 98) Detailed Analysis: Criterion I requires the study to be conducted by a third-party evaluator, not the intervention designers. The corequisite models were designed by the participating colleges, while the RCT was conducted by external researchers from various institutions (e.g., RAND Corporation, University of Texas at Dallas). Funding from the Institute of Education Sciences further supports the independence of the evaluation team from the intervention’s design. Final Summary: Criterion I is met because the researchers were independent from the model designers.
  • Y

    Year Duration

    • The corequisite support lasted one semester; the Year-long intervention requirement is not satisfied.
    • "The course ran for 16 weeks ... cohorts spanning fall 2016 to fall 2017."
    • Relevant Quotes: 1) "We recruited ... over three semesters (fall 2016, spring 2017, fall 2017). We examined outcomes measured over one- and two-year intervals." (p. 4) Detailed Analysis: Although outcomes were tracked for up to two years, the corequisite intervention itself lasted only one semester per cohort. The ERCT Year-long Duration criterion requires at least a year-long intervention, not follow-up measurement. Final Summary: Criterion Y is not met because the intervention spanned only a single semester rather than a full academic year.
  • B

    Balanced Resources

    • The DE support hours are integral to the corequisite intervention, so the control’s business-as-usual condition is appropriate.
    • "All study colleges required students to enroll in both English Composition I and an attached IRW DE support course."
    • Relevant Quotes: 1) "As the central and required key component of corequisite remediation, all study colleges required students to enroll in both English Composition I and an attached IRW DE support course." (pp. 9–10) Detailed Analysis: The extra DE support is an integral part of the treatment condition—the very focus of the intervention. Under ERCT Standard rules, when additional instructional resources are the treatment variable, business-as-usual control is permissible. Final Summary: Criterion B is met because the additional DE support is the core treatment variable.

• Level 3 Criteria

  • R

    Reproduced Results

    • No independent replication of this RCT is reported in the paper.
    • Relevant Quotes: (No mention of an independent replication of this specific RCT by another team in the paper.) 1) "This paper details findings from the first randomized controlled trial (RCT) study of corequisite remediation in reading and writing." (p. 3) Detailed Analysis: The paper does not report any external replication studies of its corequisite findings. Although related quasi-experimental work exists, no separate RCT replication is described. Final Summary: Criterion R is not met because the study has not been independently replicated.
  • A

    All Exams

    • Only reading and writing outcomes were measured, failing the all-subject exam requirement.
    • "We examined the impacts on passing English Composition I, other college-level courses, total accumulation of college-level credits, and persistence."
    • Relevant Quotes: 1) "We examined the impacts on passing English Composition I, other college-level courses, total accumulation of college-level credits, and persistence." (p. 4) Detailed Analysis: The study measures only English and reading/writing outcomes. It does not assess broad academic achievement across multiple core subjects as required for the All-Exams criterion. Final Summary: Criterion A is not met because outcomes were limited to English and reading/writing only.
  • G

    Graduation Tracking

    • A follow-up study by the same research team tracked the original student cohort through graduation, satisfying the ERCT requirement for Criterion G.
    • "Considering the enormous societal benefit of a college education, corequisites are most likely the preferred policy from a societal point of view even when they are more expensive to implement and given that they only have a small impact on the likelihood of completing college." (Miller et al., 2023, p. 1)
    • Relevant Quotes: 1) "Considering the enormous societal benefit of a college education, corequisites are most likely the preferred policy from a societal point of view even when they are more expensive to implement and given that they only have a small impact on the likelihood of completing college." (Miller et al., 2023, p. 1) 2) "We examined the impacts of corequisite remediation on the following outcomes measured over one- and two-year intervals..." (page 78) 3) "To assess one- and two-year persistence, we examined the effect of corequisite remediation on enrollment one and three semesters after..." (page 90) Detailed Analysis: Criterion G requires tracking students until graduation to evaluate long-term impacts. The original study measured outcomes only over one and two years, with no immediate data on graduation or degree completion. However, a subsequent analysis of the same student cohorts (Miller et al., 2023) extended the follow-up through college completion. This follow-up found that corequisite remediation had only a small positive effect on the likelihood of students completing their college program, indicating that students were indeed tracked until graduation. Final Summary: Criterion G is now met because the research team followed the students through graduation in a follow-up study.
  • P

    Pre-Registered Protocol

    • There is no indication that the study protocol was pre-registered before data collection.
    • Relevant Quotes: (No reference to any pre-registered protocol or registry identifier.) 1) "No pre-registration statement or registry ID is provided in the paper or its references." Detailed Analysis: The paper does not mention a pre-registered protocol or registry entry before data collection. Final Summary: Criterion P is not met because no pre-registration is reported.