Professional Learning for ESL Teachers: A Randomized Controlled Trial to Examine the Impact on Instruction, Collaboration, and Cultural Wealth

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomization occurred at the school level (a stronger unit than class-level), automatically satisfying the class-level RCT criterion.
    • "In both cohort 1 and cohort 2, schools were randomly assigned to either the full-year PL program or to a waitlist control group after all teacher consent forms had been collected." (p. 6)
    • Relevant Quotes: 1) "Randomization procedures were conducted by the team statistician, who did not have a role in recruiting either the schools or teachers." (p. 6) 2) "In both cohort 1 and cohort 2, schools were randomly assigned to either the full-year PL program or to a waitlist control group after all teacher consent forms had been collected." (p. 6) 3) "In cohort 1, schools were blocked by district and then assigned to treatment vs. control within district using a permuted blocks randomization procedure." (p. 6) Detailed Analysis: The paper explicitly states that the unit of randomisation was the school, not the individual teacher or classroom. Entire elementary schools (with their ESL teacher and grade-level partner teacher) were assigned as a block to either the PL program or the waitlist control. Because school-level randomisation is the stronger criterion required at ERCT Level 2, it automatically satisfies the weaker class-level requirement, and there is no indication that individual students or teachers within the same school were split across conditions. Final: Criterion C is met because randomisation was conducted at the school level, which exceeds the class-level requirement.
  • E

    Exam-based Assessment

    • No standardised exam-based assessment of student (or teacher) outcomes was used; outcomes were classroom observation ratings and self-report teacher surveys.
    • "Trained observers, blind to condition, scored the presence or absence of between three and seven indicators for each of the eight instructional strategies on a classroom observation tool." (p. 8)
    • Relevant Quotes: 1) "Trained observers, blind to condition, scored the presence or absence of between three and seven indicators for each of the eight instructional strategies on a classroom observation tool." (p. 8) 2) "The ESL teachers responded to an online survey at the beginning and end of the school year. The teachers rated themselves on six items related to their collaboration process/planning with the classroom teachers and one question related to the frequency of their collaboration." (p. 8) 3) "Classroom observers rated cultural wealth in the classroom environment on a scale from 1 (not observed at all) to 5 (exemplary effort with high-level examples and clear framework or intentional effort) based on a list of visible indicators of intentional connections to Latino language or culture." (p. 9) Detailed Analysis: The ERCT Standard requires a standardised, widely recognised exam-based assessment of educational outcomes. This study's outcome measures are a custom classroom observation tool (rating instructional strategy indicators), a self-report teacher survey on collaboration, and an observer-rated cultural wealth scale. None of these are standardised exams, and no student academic exam of any kind (standardised or custom) was administered in this study; the outcomes are entirely about teacher practice rather than student exam performance. Final: Criterion E is not met because the study used researcher-developed observation tools and teacher self-report surveys rather than a standardised exam-based assessment.
  • T

    Term Duration

    • Outcomes were tracked across an entire yearlong program (fall through spring), far exceeding one academic term.
    • "The goal of this study was to examine the impact of the yearlong professional learning (PL) program on ESL teachers' instructional practices and collaboration." (p. 6)
    • Relevant Quotes: 1) "The goal of this study was to examine the impact of the yearlong professional learning (PL) program on ESL teachers' instructional practices and collaboration." (p. 6) 2) "Classroom observations were conducted four times during the school year in both the intervention and control group ESL classrooms during the 2018-2019 school year: once at the beginning of the school year prior to the professional learning workshops and three additional times during the winter and spring." (pp. 7-8) 3) "Workshops were provided in four full-day sessions, two in the fall and two in the winter. In addition, the school-based teams met weekly using a semi-structured collaboration framework and monthly with an implementation coach." (p. 5) Detailed Analysis: Outcomes (instructional strategy observations, teacher surveys) were measured at the start of the year and again at multiple points through winter and spring, spanning the full academic year rather than a single short intervention window. This duration far exceeds the minimum one-term requirement. Final: Criterion T is met because measurement spanned an entire academic year, well beyond a single term.
  • D

    Documented Control Group

    • The paper names the control condition as a "waitlist control group" but provides no separate description of its size, demographics, or baseline characteristics distinct from the overall sample.
    • "In both cohort 1 and cohort 2, schools were randomly assigned to either the full-year PL program or to a waitlist control group after all teacher consent forms had been collected." (p. 6)
    • Relevant Quotes: 1) "In both cohort 1 and cohort 2, schools were randomly assigned to either the full-year PL program or to a waitlist control group after all teacher consent forms had been collected." (p. 6) 2) "Accordingly, 29 teachers from 29 different elementary schools were included in these analyses." (p. 7, applies to the combined sample, not broken out by condition) 3) Tables 2 and 3 (district and ESL teacher characteristics) report demographics for the overall sample and by district, but do not present a breakdown of school or teacher characteristics specifically for the control arm versus the intervention arm. Detailed Analysis: The ERCT Standard requires detailed documentation of the control group specifically: its size, demographics, baseline performance, and the conditions it experienced. This paper describes the control condition only as a "waitlist control group" with no further detail on how many schools/teachers were assigned to it, what business as usual entailed for these teachers, or how their baseline characteristics compared to the intervention group beyond generic covariate adjustment in the regression models. Baseline outcome scores are used as covariates in the ANCOVA models, but no explicit descriptive table or quote isolates control-group characteristics, size, or documented conditions. Final: Criterion D is not met because the paper lacks a detailed, separate description of the control group's size, demographics, and conditions.

• Level 2 Criteria

  • S

    School-level RCT

    • Entire schools (with their ESL teacher/grade partner team) were the unit of randomisation, satisfying the school-level RCT criterion.
    • "In both cohort 1 and cohort 2, schools were randomly assigned to either the full-year PL program or to a waitlist control group after all teacher consent forms had been collected." (p. 6)
    • Relevant Quotes: 1) "In both cohort 1 and cohort 2, schools were randomly assigned to either the full-year PL program or to a waitlist control group after all teacher consent forms had been collected." (p. 6) 2) "In cohort 1, schools were blocked by district and then assigned to treatment vs. control within district using a permuted blocks randomization procedure." (p. 6) 3) "In cohort 2, which included schools in only one district, schools were blocked by type of calendar (traditional or year-round) and then assigned to treatment vs. control within block using a permuted blocks randomization procedure." (p. 6) Detailed Analysis: The randomisation unit throughout the study is the school (with 30 elementary schools across three districts). This directly satisfies the ERCT Standard's requirement that randomisation occur among schools rather than only among classes or students within a school. Final: Criterion S is met because entire schools were the unit of random assignment.
  • I

    Independent Conduct

    • The same authors who designed the PL program (and stand to benefit financially from its commercial licensing) also led the study's conceptualization, analysis, and reporting; only randomisation itself was handled by an independent statistician.
    • "The teacher professional learning program described in this study will be licensed to FigStar Learning, LLC, a company co-founded by Babinski and Amendum. In the future, Babinski and Amendum, along with their universities, may benefit financially." (p. 14)
    • Relevant Quotes: 1) "Randomization procedures were conducted by the team statistician, who did not have a role in recruiting either the schools or teachers." (p. 6) 2) "Conflicts of Interest: The teacher professional learning program described in this study will be licensed to FigStar Learning, LLC, a company co-founded by Babinski and Amendum. In the future, Babinski and Amendum, along with their universities, may benefit financially." (p. 14) 3) "Author Contributions: Conceptualization, L.M.B., S.J.A., S.E.K. and M.S.; methodology, M.M.C., L.M.B. and S.J.A.; formal analysis, M.M.C.; investigation, L.M.B., S.J.A., S.E.K. and M.S.; ... project administration, L.M.B., S.J.A. and S.E.K." (p. 13) Detailed Analysis: While allocation to condition itself was handled by a team statistician without a recruiting role, the broader independence requirement concerns whether the evaluation as a whole (design, delivery, analysis, interpretation) is separate from the intervention's designers. Here, the same individuals (Babinski and Amendum, with Knotek and Sánchez) who conceptualised the PL program also led conceptualization, investigation, and project administration of this evaluation, and the disclosed conflict of interest confirms they will financially benefit from the program's commercial licensing. There is no external, third-party evaluation team independent of the program's developers. Final: Criterion I is not met because the intervention's designers, who have a financial stake in the program, led the evaluation, with independence limited only to the randomisation step.
  • Y

    Year Duration

    • The program and its measurement span the full academic year, satisfying the year-duration requirement.
    • "The goal of this study was to examine the impact of the yearlong professional learning (PL) program on ESL teachers' instructional practices and collaboration." (p. 6)
    • Relevant Quotes: 1) "The goal of this study was to examine the impact of the yearlong professional learning (PL) program on ESL teachers' instructional practices and collaboration." (p. 6) 2) "Classroom observations were conducted four times during the school year ... once at the beginning of the school year prior to the professional learning workshops and three additional times during the winter and spring." (pp. 7-8) 3) "Workshops were provided in four full-day sessions, two in the fall and two in the winter. In addition, the school-based teams met weekly using a semi-structured collaboration framework and monthly with an implementation coach." (p. 5) Detailed Analysis: The PL program and its associated weekly/monthly collaboration and coaching activities run across the full school year, from fall workshops through spring follow-up observations. This meets the ERCT requirement of tracking outcomes across at least 75% of an academic year. Final: Criterion Y is met because the intervention and its outcome measurement covered the full academic year.
  • B

    Balanced Control Group

    • The additional professional-learning time, coaching, and collaboration structure given to the intervention group are the explicit treatment variable being tested against a business-as-usual waitlist control.
    • "In both cohort 1 and cohort 2, schools were randomly assigned to either the full-year PL program or to a waitlist control group after all teacher consent forms had been collected." (p. 6)
    • Relevant Quotes: 1) "Workshops were provided in four full-day sessions, two in the fall and two in the winter. In addition, the school-based teams met weekly using a semi-structured collaboration framework and monthly with an implementation coach." (p. 5) 2) "In both cohort 1 and cohort 2, schools were randomly assigned to either the full-year PL program or to a waitlist control group after all teacher consent forms had been collected." (p. 6) 3) "The purpose of the current study was to examine the impacts of a professional learning program on ESL teachers' use of evidence-based instructional strategies for MLs; their collaboration frequency, processes, and roles; and the incorporation of Latino MLs' cultural wealth into their classroom environment." (p. 11) Detailed Analysis: The intervention group received substantial additional resources relative to the control group: four full-day workshops, weekly structured collaboration time, and monthly implementation coaching. However, this additional time and support is precisely the professional learning program being tested — the study's explicit purpose is to determine whether providing this PL program (with its built-in extra time and coaching) changes ESL teachers' practices, relative to a "waitlist" control that continues business as usual and receives the program later. Because the extra resources are the treatment variable itself and are clearly framed as such, the control group's lack of these resources during the study period does not create an unaddressed confound; it is the designed comparison of "PL program" vs. "no PL program yet". Final: Criterion B is met because the additional PL time and coaching are integral to the treatment being tested, and the waitlist control appropriately represents the business-as-usual condition.

• Level 3 Criteria

  • R

    Reproduced

    • No independent replication by a different research team was found; internet search of papers citing this study turned up only unrelated reviews and bibliometric work, and the only prior/related trial is by the same authors.
    • "Earlier studies of the PL program showed a significant impact on classroom teachers' instructional practices and MLs' language and literacy outcomes [9,49]. This is the first study examining the impact of the program on ESL teachers' practices." (p. 6)
    • Relevant Quotes: 1) "Earlier studies of the PL program showed a significant impact on classroom teachers' instructional practices and MLs' language and literacy outcomes [9,49]. This is the first study examining the impact of the program on ESL teachers' practices." (p. 6) 2) Reference 9: "Babinski, L.M.; Amendum, S.; Knotek, S.; Sánchez, M.; Malone, P. Improving young English learners' language and literacy skills through teacher professional development: A randomized controlled trial. Am. Educ. Res. J. 2018, 55, 117-143." (p. 14) 3) Reference 49: "Babinski, L.M.; Amendum, S.; Carrig, M.; Knotek, S.; Sánchez, M. Teaching Young Multilingual Learners: Impacts of a Professional Learning Program on Teachers' Practices and Students' Language and Literacy Skills. Manuscript submitted for publication, under review." (p. 15) Internet Research (2026-07-30): 4) Citation search (Semantic Scholar) for this article (DOI 10.3390/educsci14070690) returned six citing papers: a systematic review on school-university research collaboration (Espinosa-Rodríguez et al., 2026); a paper on "Professional Participation and Instructional Strategies of English Teachers" (Gigante, 2026); a study of Khmer-Vietnamese bilingual education in Vietnam (Duong et al., 2026); a systematic review of phonics instruction in ESL (Valladolid & Estremera, 2026); a paper on cultural intelligence in teaching English to young learners in Indonesia (Sisviana et al., 2025); and a bibliometric analysis of cultural literacy programs (Istiq'faroh et al., 2025). None of these papers attempt to replicate this specific school-level RCT of the ESL teacher PL program; they cite it as background literature in unrelated contexts (systematic reviews, bibliometric analyses, or entirely different interventions/countries). 5) The reference 49 manuscript has since been published: Babinski, L.M.; Amendum, S.; Carrig, M.; Knotek, S.; Sánchez, M. "Teaching young multilingual learners: impacts of a professional learning programme on teachers' practices and students' language and literacy skills." Journal of Multilingual and Multicultural Development, 2025 (DOI 10.1080/01434632.2025.2472880). Per its indexed abstract: "This randomized control trial examined a year-long professional learning program designed to enhance literacy instruction for multilingual learners... The study involved 39 teachers and 106 multilingual learners across 13 schools in the Southeastern United States." This is the same author team (Babinski, Amendum, Carrig, Knotek, Sánchez) studying classroom teachers and their MLs, a different sample than the 29 ESL teachers in the present paper, so it is a companion study by the same team rather than an independent replication by a different research group. Detailed Analysis: Criterion R requires independent replication of the study by a different research team, ideally published in a peer-reviewed outlet. The paper itself states this is "the first study examining the impact of the program on ESL teachers' practices," and the only related prior/ companion studies identified (both in the reference list and via citation search) are authored by the same research team (Babinski, Amendum, Knotek, Sánchez, and colleagues). No independent replication by an unrelated research team, in a different context, was located. Final: Criterion R is not met because only the same research team has studied this program, with no independent replication identified.
  • A

    All-subject Exams

    • Because criterion E (exam-based assessment) is not met, the stronger all-subject exam criterion cannot be met either.
    • Relevant Quotes: 1) "Trained observers, blind to condition, scored the presence or absence of between three and seven indicators for each of the eight instructional strategies on a classroom observation tool." (p. 8) Detailed Analysis: Per the ERCT Standard's specific instruction for criterion A, if criterion E is not met, then A is automatically not met. This study did not use any standardised exam-based assessment of student outcomes (or of any outcome); its measures were classroom observation tools, teacher surveys, and a cultural wealth rating scale, none of which qualify as exam-based assessments. Additionally, the study only examined literacy-related instructional practices and collaboration, not performance across all main subjects. Final: Criterion A is not met because the prerequisite criterion E is not met, and no all-subject exam-based assessment was used.
  • G

    Graduation Tracking

    • No evidence of any follow-up tracking beyond the single program year was found in the paper or in subsequent publications by the same authors; no tracking toward graduation is reported anywhere.
    • "Second, school closures due to the COVID-19 pandemic limited the number of classroom observations for the second cohort of ESL teachers." (p. 13)
    • Relevant Quotes: 1) "Second, school closures due to the COVID-19 pandemic limited the number of classroom observations for the second cohort of ESL teachers." (p. 13) 2) "The ESL teachers responded to an online survey at the beginning and end of the school year." (p. 8, indicating measurement ended at the close of the program year) 3) No mention anywhere in the Results, Discussion, or Limitations sections of any follow-up beyond the single program year, for either the ESL teachers or their students. Internet Research (2026-07-30): 4) A search for subsequent publications by the same author team (Babinski, Amendum, Carrig, Knotek, Sánchez) identified one companion paper: "Teaching young multilingual learners: impacts of a professional learning programme on teachers' practices and students' language and literacy skills," Journal of Multilingual and Multicultural Development, 2025 (DOI 10.1080/01434632.2025.2472880). Per its indexed abstract: "This randomized control trial examined a year-long professional learning program designed to enhance literacy instruction for multilingual learners... The study involved 39 teachers and 106 multilingual learners across 13 schools in the Southeastern United States," finding "significantly higher literacy growth on the Measures of Academic Progress (MAP) for MLs whose teachers participated in the PL." This companion study is itself a separate one-year RCT with a different sample of teachers and multilingual learners (not the same 29 ESL teachers or their students from the present paper), and it likewise reports only one year of outcome data with no tracking toward graduation. No paper tracking this study's specific cohort of ESL teachers or their students into later grades or toward graduation was found. Detailed Analysis: The ERCT Standard's Criterion G requires tracking of participants until graduation, either within the paper itself or in subsequent publications by the same authors. All outcome measurement in this paper (classroom observations, teacher surveys) concluded at the end of the single program year. The one identified companion publication by the same team examines a different sample over the same one-year timeframe and does not extend tracking toward graduation. No planned or completed longer-term follow-up was found in the paper's Limitations section or via internet search. Final: Criterion G is not met because tracking stopped at the end of the single program year, with no evidence of follow-up toward graduation in this paper or in any identified subsequent publication.
  • P

    Pre-Registered

    • The paper reports IRB approval but contains no mention of pre-registration of the study protocol, and no matching registry entry was located via internet search.
    • "Institutional Review Board Statement: The study was approved by the Institutional Review Board of Duke University (protocol 2018-0545, approved on 5-4-2018)." (p. 14)
    • Relevant Quotes: 1) "Institutional Review Board Statement: The study was approved by the Institutional Review Board of Duke University (protocol 2018-0545, approved on 5-4-2018)." (p. 14) 2) No statement anywhere in the Methods, Data Analytic Strategy, or supplementary sections referencing a trial registry (e.g., ClinicalTrials.gov, AsPredicted, OSF Registries) or a pre-registered analysis plan. Internet Research (2026-07-30): 3) Searches of OSF Registries and ClinicalTrials.gov for Babinski/Amendum/Knotek/Sánchez and this ESL teacher PL program did not surface a matching pre-registration record. The funding record for IES grant R305A180336 (Institute of Education Sciences, U.S. Department of Education) does not reference a separate pre-registered protocol either. No pre-registration could be confirmed to exist for this study. Detailed Analysis: IRB ethical approval is a distinct requirement from pre-registration of the study's hypotheses, methods, and analysis plan on a public registry before data collection began. The paper documents only IRB approval and gives no indication that the study's protocol, hypotheses, or planned analyses were registered on any platform prior to the study's start, and no such registration was found through independent internet search. Final: Criterion P is not met because no pre-registration of the study protocol is mentioned anywhere in the paper or found via internet search.