Effective Early Literacy Skill Development for Young Spanish-Speaking English Language Learners: An Experimental Study of Two Methods

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomization was explicitly conducted at the student level within shared classrooms, which does not meet the class-level (or stronger) requirement and is not covered by the one-to-one tutoring exception.
    • "Children were assigned to conditions within classrooms to avoid problems with nesting." (p. 706-707)
    • Relevant Quotes: 1) "Ninety-six children were randomly assigned (balancing for gender) to one of three conditions: (a) a control group (n = 32) that received their business-as-usual classroom High/Scope Curriculum only, (b) a group that received their classroom High/Scope Curriculum and the small groups from the Literacy Express Preschool Curriculum in English only (n = 31), and (c) a group that received their High/Scope Curriculum and the small groups from the Literacy Express Preschool Curriculum beginning in Spanish and then transitioning to English instruction (n = 31). Children were assigned to conditions within classrooms to avoid problems with nesting." (p. 706-707) 2) "The participants were 94 Spanish-speaking ELL children ... who were enrolled in 10 classes in a Head Start preschool program located in an inner-city neighborhood of Los Angeles, California." (p. 706) 3) "the assistants delivered the intervention to small groups of four to five children in separate classrooms adjacent to the larger classrooms for approximately 20 min, four times per week" (p. 708) Detailed Analysis: The paper explicitly states that children were randomized to condition "within classrooms," i.e. individual students inside the same 10 Head Start classrooms were split across the control, English-only, and transitional conditions, rather than whole classes or schools being assigned wholesale to a single condition. The authors even give the rationale ("to avoid problems with nesting"), confirming that student-level randomization within class was a deliberate design choice rather than an accident of reporting. The ERCT exception for personal tutoring/one-to-one teaching does not clearly apply here: the intervention was delivered to small pull-out groups of four to five children by research assistants, not as individual one-to-one tutoring, so the tutoring exception is not satisfied. Because randomization occurred at the student level within shared classrooms rather than at the class or school level, and the intervention is not one-to-one personal tutoring, this creates exactly the contamination risk (children from the same class assigned to different conditions) that the Class-level RCT criterion is designed to guard against. Criterion C is not met because randomization was explicitly conducted at the student level within classrooms rather than at the class or school level, and the personal-tutoring exception does not apply to this small-group (four-to-five children) pull-out design.
  • E

    Exam-based Assessment

    • The primary outcome instrument is an author- developed pilot measure (P-CTOPPP), not a widely recognized standardized exam.
    • "The P-CTOPPP is the development version of the Test of Preschool Early Literacy (TOPEL...), and the P-CTOPPP-Spanish is an adapted version of the P-CTOPPP." (p. 707)
    • Relevant Quotes: 1) "Children's oral language, phonological awareness, and print knowledge were measured using the Receptive Vocabulary, Definitional Vocabulary, Blending, Elision, and Print Knowledge subtests of the Preschool Comprehensive Test of Phonological and Print Processing (P-CTOPPP; Lonigan, Wagner, Torgesen, & Rashotte, 2002), and P-CTOPPP-Spanish (Lonigan, Farver, & Eppe, 2002)." (p. 707) 2) "The P-CTOPPP is the development version of the Test of Preschool Early Literacy (TOPEL; Lonigan, Wagner, Torgesen, & Rashotte, 2007), and the P-CTOPPP-Spanish is an adapted version of the P-CTOPPP." (p. 707) 3) "To provide a benchmark by which to compare the impact of the interventions, children's responses on the English Language Pre-CTOPPP measures were converted to the TOPEL (a nationally normed and validated test of early literacy skills; Lonigan, Wagner, et al., 2007) versions of these subtests..." (p. 713) Detailed Analysis: The primary outcome instrument used throughout the paper (Tables 2 and 3, all ANCOVAs and effect sizes) is the P-CTOPPP and P-CTOPPP-Spanish, an unpublished "development version" test authored and normed by two of the study's own investigators (Lonigan is a co-author of both this paper and the P-CTOPPP instrument). It is not itself a widely recognized, published standardized test; rather it is the pilot/developmental version of a later published standardized instrument (TOPEL, 2007) and a researcher-created Spanish adaptation. Note the paper itself is inconsistent in how it labels this instrument, spelling it "P-CTOPPP" in the Method section (p. 707-708) but "Pre-CTOPPP" once in the Results section (p. 713); both spellings refer to the same author-developed instrument. Although the authors additionally converted scores to TOPEL standard scores as a secondary benchmark against national norms, the paper's primary analyses rely on the non-standardized, in-house P-CTOPPP/P-CTOPPP-Spanish measures. Under the ERCT standard, a researcher-developed pilot instrument used as the main outcome measure does not satisfy the exam-based-assessment criterion, even when a standardized-test conversion is performed secondarily. Criterion E is not met because the primary outcome measures (P-CTOPPP and P-CTOPPP-Spanish) are unpublished, author-developed instruments rather than a widely recognized standardized exam.
  • T

    Term Duration

    • Outcomes were measured in May/June, several months after the mid-November intervention start, well beyond the one-term minimum.
    • "Children's preliteracy skills were assessed in both Spanish and English prior to the intervention in October/November and at the end in May/June..." (p. 708)
    • Relevant Quotes: 1) "the assistants delivered the intervention to small groups of four to five children ... for approximately 20 min, four times per week from mid-November to mid-May (approximately 21 weeks)." (p. 708) 2) "Children's preliteracy skills were assessed in both Spanish and English prior to the intervention in October/November and at the end in May/June by trained research assistants who were not involved in the delivery of the intervention and were blind to the children's group assignments." (p. 708) Detailed Analysis: The intervention began in mid-November, and the primary post-intervention assessment took place at the end of the preschool year, in May/June, several months after intervention onset. This interval (roughly five to six months) clearly exceeds the minimum of one academic term (approximately 3-4 months) required by criterion T. Criterion T is met because the interval from intervention start (mid-November) to outcome measurement (May/June) spans several months, exceeding the one-term minimum.
  • D

    Documented Control Group

    • The control group's size, demographics, and baseline literacy scores are documented in detail in Tables 1 and 2, and its business-as-usual treatment is explicitly stated.
    • "a control group (n = 32) that received their business-as-usual classroom High/Scope Curriculum only" (p. 706)
    • Relevant Quotes: 1) "a control group (n = 32) that received their business-as-usual classroom High/Scope Curriculum only" (p. 706) 2) "As shown in Table 1, there were no significant differences among the groups in parents' levels of education, marital status, years of U.S. residence, or characteristics of children's the home literacy environments." (p. 710) 3) "There were no statistically significant group differences in children's scores on the English- or Spanish-language preliteracy skill assessments (all ps > .10)." (p. 710) 4) "Table 1. Means, Standard Deviations, and Ranges for the Demographic Variables by Intervention Group (N = 94)" (p. 710) Detailed Analysis: The paper provides Table 1 with detailed demographic/background baseline statistics (child age, parents' education, home literacy activities, home-language environment, etc.) separately for the control group versus the two treatment groups, and Table 2 with baseline preliteracy scores, confirming no significant differences at baseline. The control condition is explicitly described as receiving the "business-as-usual" High/Scope Curriculum only, with a documented sample size (n = 32). Criterion D is met because the paper documents the control group's size, demographic composition, and baseline performance in detail (Tables 1 and 2), and explicitly describes its "business-as-usual" classroom experience.

• Level 2 Criteria

  • S

    School-level RCT

    • Only a single preschool program (site) was involved, with randomization at the student level within its classrooms, not at the school level.
    • "enrolled in 10 classes in a Head Start preschool program located in an inner-city neighborhood of Los Angeles, California." (p. 706)
    • Relevant Quotes: 1) "The participants were 94 Spanish-speaking ELL children ... who were enrolled in 10 classes in a Head Start preschool program located in an inner-city neighborhood of Los Angeles, California." (p. 706) 2) "Children were assigned to conditions within classrooms to avoid problems with nesting." (p. 706-707) Detailed Analysis: All 94 participating children were drawn from a single Head Start preschool program (10 classes at one site) in Los Angeles, and assignment to condition was performed at the individual-student level within classrooms, not by randomizing separate schools or sites to different conditions. There is no school-level randomization anywhere in the design; only one program/site was involved, and children within it were split across conditions. Criterion S is not met because the study involves a single preschool site with student-level assignment within classrooms, not randomization of multiple schools/sites to different conditions.
  • I

    Independent Conduct

    • Two of the three study authors (Lonigan and Farver) co-designed the Literacy Express curriculum being tested, and the third author personally supervised its delivery, so there is no independent conduct.
    • "The Literacy Express Preschool Curriculum (Lonigan, Clancy-Menchetti, Phillips, McDowell, & Farver, 2005)..." (p. 706)
    • Relevant Quotes: 1) "The Literacy Express Preschool Curriculum (Lonigan, Clancy-Menchetti, Phillips, McDowell, & Farver, 2005) focuses on young children's oral language, emergent literacy, basic math and science, and socio-emotional development." (p. 706) 2) "In August and September, four bilingual graduate research assistants were trained to deliver the curriculum in Spanish and English by the third author, who also served as the intervention supervisor." (p. 708) 3) Reference list: "Lonigan, C. J., Clancy- Menchetti, J., Phillips, B. M., McDowell, K., & Farver, J. M. (2005). Literacy express: A preschool curriculum. Tallahassee, FL: Literacy Express." (p. 717) Detailed Analysis: The intervention curriculum (Literacy Express) was itself designed by two of the study's three authors, Lonigan and Farver, per the cited 2005 curriculum reference. The same author team that created the curriculum also designed the trial, and the third author (Eppe) personally trained the research assistants and served as intervention supervisor. There is no independent, third-party organization conducting the trial or overseeing its implementation. While outcome assessments were collected by research assistants blind to condition (addressing measurement bias only), this does not satisfy the broader Independent Conduct requirement, which concerns separation between the intervention's designers and the team running and evaluating the study. Criterion I is not met because the study was designed, supervised, and evaluated by the same author team that created the Literacy Express curriculum being tested, with no independent third party conducting the trial.
  • Y

    Year Duration

    • The documented intervention-to-measurement interval of roughly five to six months does not clearly meet the 75%-of-a-year threshold required for Y.
    • "the assistants delivered the intervention ... from mid-November to mid-May (approximately 21 weeks)." (p. 708)
    • Relevant Quotes: 1) "the assistants delivered the intervention ... from mid-November to mid-May (approximately 21 weeks)." (p. 708) 2) "Children's preliteracy skills were assessed in both Spanish and English prior to the intervention in October/November and at the end in May/June..." (p. 708) 3) "Additionally, it is unfortunate the children in this study were not followed up longitudinally." (p. 716) Detailed Analysis: The authors explicitly quantify the intervention duration as "approximately 21 weeks" (roughly 4.8 months), running from mid-November to mid-May. Even allowing for final testing to have extended somewhat into "May/June," the documented interval from intervention onset to final measurement is at most about five to six months. This falls short of the roughly 6.75-7.5 months (75% of a 9-10 month academic year) required by the Year Duration criterion. The paper provides no quote establishing that the intervention-to-measurement interval reached 75% of a full academic year; on the contrary, its own text frames the tracked intervention period as "approximately 21 weeks," and the authors separately note that no longitudinal follow-up was conducted. Criterion Y is not met because the documented intervention-to-measurement interval (about five to six months) does not clearly reach 75% of a full academic year.
  • B

    Balanced Control Group

    • The extra small-group instructional time is the explicit treatment variable being tested, so the control group's business-as-usual curriculum is an acceptable baseline under the ERCT exception.
    • "a control group (n = 32) that received their business-as-usual classroom High/Scope Curriculum only" (p. 706)
    • Relevant Quotes: 1) "a control group (n = 32) that received their business-as-usual classroom High/Scope Curriculum only" (p. 706) 2) "Throughout the intervention, all children continued to receive whatever instructional activities were a part of their classroom curriculum, which was High/Scope in all cases." (p. 708) 3) "the assistants delivered the intervention to small groups of four to five children ... for approximately 20 min, four times per week from mid-November to mid-May" (p. 708) 4) "We used the core small-group activities from the Literacy Express Preschool Curriculum, and we created parallel Spanish-language versions of these activities and materials." (p. 708) Detailed Analysis: Applying the ERCT decision tree for Criterion B: extra resources are present (the two intervention groups received about 20 extra minutes of small-group instruction, four times weekly, for approximately 21 weeks, that the control group did not receive), and this difference is not negligible. However, these additional resources are the explicit treatment variable being tested: the study's stated purpose is to contrast the added Literacy Express small-group curriculum (delivered English-only or transitionally) against standard classroom High/Scope instruction alone. The added small-group sessions are not a separable, supplementary add-on to some other primary treatment; they are themselves the core intervention under investigation, comparable to the DPL-tool and PAX-GBG exception examples in the ERCT standard where extra time/technology integral to the tested package is compared against a business-as-usual control. The control condition explicitly received the standard "business-as-usual" classroom curriculum, which is the permitted baseline when additional resources are the explicit treatment variable. Criterion B is met because the additional small-group instructional time is the explicit, integral treatment variable being tested against a "business-as-usual" classroom control, consistent with the ERCT standard's exception for interventions whose added resources are the primary treatment variable.

• Level 3 Criteria

  • R

    Reproduced

    • The authors describe this as the first study of its kind, and neither the paper nor a targeted internet search identified an independent replication of this specific trial by a different research team.
    • Relevant Quotes: 1) "To our knowledge, this is the first study to test such an intervention with Spanish-speaking ELL preschool children and that used a design that allowed for causally interpretable findings." (p. 715-716) Detailed Analysis: The authors themselves characterize this study as the first of its kind for this specific population and design, and the paper contains no reference to any prior or subsequent independent replication of this particular comparative trial (English-only vs. transitional Spanish-to-English Literacy Express instruction vs. control) by a different research team. Internet Search (2026-07-28): Searches for citing literature and replication studies of this specific trial (e.g., "Farver Lonigan Eppe Literacy Express Spanish English replication", "Effective Early Literacy Skill Development Farver Lonigan Eppe 2009 replication") returned only references to the original 2009 study itself, related work by the same research group on the broader Literacy Express curriculum (e.g., the WWC Intervention Report on Literacy Express, and Lonigan et al.'s own school-level Literacy Express trials in Florida and California), and unrelated bilingual-literacy studies. No independent replication of this specific transitional-vs-English-only-vs-control design with Spanish-speaking ELL Head Start preschoolers, conducted by a different research team and published in a peer-reviewed journal, was identified. Criterion R is not met because the paper itself reports no independent replication of this specific study, and a targeted internet search found no subsequent independent replication by a different research team in a peer-reviewed outlet.
  • A

    All-subject Exams

    • Criterion E is not met, and independently the study assessed only literacy-related outcomes, not all core subjects.
    • Relevant Quotes: 1) "Children's emergent literacy skills were assessed before and after the intervention in Spanish and English." (abstract, p. 703) 2) "Children's oral language, phonological awareness, and print knowledge were measured using the Receptive Vocabulary, Definitional Vocabulary, Blending, Elision, and Print Knowledge subtests..." (p. 707) Detailed Analysis: Per the ERCT standard, if Criterion E (Exam-based Assessment) is not met, Criterion A is automatically not met, and E was found not met above because the primary outcome instrument (P-CTOPPP) is an author-developed pilot measure. Independently, the study measured only literacy-domain outcomes (oral language, phonological awareness, and print knowledge) in English and Spanish; no other core subjects such as mathematics or science were assessed at all, and no rationale for such a narrowing is provided beyond the study's inherent focus on early literacy. Criterion A is not met, both because Criterion E is not met and because the study exclusively measured early-literacy-related outcomes rather than all core subjects.
  • G

    Graduation Tracking

    • Criterion Y is not met, and the authors explicitly state no longitudinal follow-up was conducted, with no tracking toward graduation found in the paper or in a search for follow-up publications.
    • "it is unfortunate the children in this study were not followed up longitudinally." (p. 716)
    • Relevant Quotes: 1) "Additionally, it is unfortunate the children in this study were not followed up longitudinally. Follow-up results are needed to provide information regarding the long-term effects of the training provided, particularly with regard to the later possible cross-linguistic transfer of skills as well as the transfer of gains in emergent literacy skills to children's later reading skills." (p. 716) Detailed Analysis: Per the ERCT standard, if Criterion Y (Year Duration) is not met, Criterion G is automatically not met, and Y was found not met above. Independently, the authors explicitly acknowledge that no longitudinal follow-up was conducted beyond the end of the intervention year, and there is no tracking of children through kindergarten, elementary school, or any later graduation milestone. Internet Search (2026-07-28): Searched for subsequent publications by Farver, Lonigan, or Eppe tracking this specific Head Start cohort toward kindergarten or later graduation (e.g., "Farver Lonigan Eppe Literacy Express follow-up graduation kindergarten longitudinal"). No follow-up paper tracking this same 94-child Spanish-speaking Head Start sample toward graduation was found; the results only surfaced the original 2009 paper and unrelated longitudinal studies by other author teams on different Spanish-speaking cohorts (e.g., a kindergarten-to-grade-8 study in Reading and Writing by different authors). No evidence of graduation tracking for this cohort was found in any source. Criterion G is not met because Criterion Y is not met, and because neither the original paper nor a targeted internet search for follow-up publications found evidence that this cohort was tracked toward graduation.
  • P

    Pre-Registered

    • No pre-registration statement, registry ID, or date appears anywhere in the paper, and none was found via internet search.
    • Relevant Quotes: No quotes referencing a trial registry, protocol registration platform, registration date, or pre-specified analysis plan published before data collection were found anywhere in the paper. Detailed Analysis: The paper contains no statement of pre-registration of hypotheses, methods, or planned analyses on any registry (e.g., ClinicalTrials.gov, AEA RCT Registry, ISRCTN) prior to data collection. This is consistent with the study predating the widespread adoption of pre-registration norms in education research (data collection occurred in the 2007-2008 school year, published in 2009). Internet Search (2026-07-28): Searched for a pre-registration record for this trial (e.g., "Farver Lonigan Eppe pre-registration ClinicalTrials preschool literacy Spanish ELL"). No registry entry referencing this study, its authors, or the Literacy Express Spanish-transitional/English-only/control design was found on any trial registry. Criterion P is not met because no pre-registration statement, registry ID, or date is present anywhere in the paper, and no registry record was found via internet search.