The Effect of e-Book Vocabulary Instruction on Spanish–English Speaking Children

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomisation was conducted at the student level within grade and school, not at the class or school level, and no tutoring exception applies.
    • "Students were randomly assigned to conditions within grade and schools." (p. 1948)
    • Relevant Quotes: 1) "Students were randomly assigned to conditions within grade and schools." (p. 1948) 2) "Half of the children were randomly assigned to the e-book read-only condition, and the other half received intensive vocabulary strategies embedded within the same e-books." (p. 1948) 3) "The question of feasibility involved two aspects of interest to the investigators: (a) if it was feasible to administer technology-enhanced intervention in small groups within or outside the classroom for a 15- to 20-min dosage across 20 weeks in low–socioeconomic status schools..." (p. 1948) Detailed Analysis: The paper explicitly states that randomisation occurred "within grade and schools," meaning individual students (stratified via a principal component analysis of baseline vocabulary scores) were assigned to treatment or comparison conditions inside the same classrooms and schools, rather than whole classes or whole schools being randomised as a unit. This creates a risk of contamination, since treatment and comparison children attended the same classrooms and could interact with or observe each other. The exception for personal tutoring-style interventions does not clearly apply here: the e-book sessions were delivered to children individually via computer but administered in small-group rotations "within or outside the classroom," not as one-to-one personal tutoring by a teacher or tutor, and the paper never frames the intervention as a tutoring exception. Final sentence: Because randomisation occurred at the student level within classrooms/schools without an applicable tutoring exception, criterion C is not met.
  • E

    Exam-based Assessment

    • The study used the PPVT-4, a widely recognised, standardized, norm-referenced vocabulary assessment, as a primary outcome measure.
    • "The PPVT-4 (Dunn & Dunn, 2007) was administered in the fall and spring. The PPVT-4 is an untimed, norm-referenced measure of receptive vocabulary normed through a sample of 3,540 participants for use with individuals 2–90 years old." (p. 1951)
    • Relevant Quotes: 1) "The PPVT-4 (Dunn & Dunn, 2007) was administered in the fall and spring. The PPVT-4 is an untimed, norm-referenced measure of receptive vocabulary normed through a sample of 3,540 participants for use with individuals 2–90 years old." (p. 1951) 2) "Split-half reliability by age for Forms A and B has a mean of .94 (SD = 3.6) and ranges from .90 to .97 for ages 5–11 years, based on normative data on monolingual English-speaking children." (p. 1951) 3) "In addition to the standardized assessments, research assistants administered informal researcher-designed vocabulary probes as proximal measures of word learning through labeling and expressive definitions." (p. 1952) Detailed Analysis: The PPVT-4 is a well-established, nationally normed, standardized measure of receptive vocabulary widely used in research and clinical practice, with documented reliability statistics reported in the paper. It served as one of the study's primary (distal) experimental outcome measures. Although the study also used experimenter-created labeling and definition probes as supplementary proximal measures, the presence of a genuinely standardized, widely-recognised exam among the outcome measures satisfies criterion E, which only requires that a standard exam-based assessment be used, not that every measure be standardized. Final sentence: Criterion E is met because the study used the PPVT-4, a standardized and widely recognised vocabulary assessment, as a primary outcome measure.
  • T

    Term Duration

    • Baseline testing occurred in the fall and the primary outcome (PPVT-4) was measured in the spring, an interval spanning well over one academic term.
    • "The PPVT-4 (Dunn & Dunn, 2007) was administered in the fall and spring." (p. 1951)
    • Relevant Quotes: 1) "Research assistants administered standardized assessments of language, literacy, and nonverbal intelligence in September as descriptive measures of the participants' baseline skills." (p. 1951) 2) "The PPVT-4 (Dunn & Dunn, 2007) was administered in the fall and spring." (p. 1951) 3) "The comparison group received 17.69 weeks of treatment on average (SD = 3.89 weeks), and the intervention group received 16.83 weeks on average (SD = 4.35 weeks)." (p. 1953) Detailed Analysis: Baseline/descriptive testing (including the PPVT-4 pretest) began in September, and the primary outcome measure (PPVT-4 posttest) was collected in spring, after participants received an average of roughly 17 weeks (about four months) of thrice-weekly e-book sessions. Even taking only the delivered-instruction duration as the relevant interval, this already reaches approximately one academic term (3-4 months); accounting for the full fall-to-spring measurement window, the interval clearly exceeds one term. Final sentence: Criterion T is met because the interval between the fall baseline and the spring outcome measurement spans at least one full academic term.
  • D

    Documented Control Group

    • The comparison (read-only) group's activities, baseline characteristics, and equivalence to the treatment group are documented in detail.
    • "For the comparison condition, the children listened to the same recorded e-books three times a week in English but without any embedded instruction, directions, or additional language content other than a recording of the text as it appeared on the page." (p. 1951)
    • Relevant Quotes: 1) "For the comparison condition, the children listened to the same recorded e-books three times a week in English but without any embedded instruction, directions, or additional language content other than a recording of the text as it appeared on the page." (p. 1951) 2) "Examination of background characteristics between the treatment and comparison conditions after randomization revealed similar language backgrounds between the groups. No differences were noted in reported mother educational level, χ2(18, N = 211) = 19.15, p = .383, or in father educational level, χ2(18, N = 192) = 14.52, p = .695." (p. 1951) 3) "Table 3. Descriptive data for standardized measures by group," presenting separate columns for "Read 2 (Treatment; N = 155)" and "Read 1 (Control; N = 133)" across multiple baseline measures. (p. 1952) Detailed Analysis: The paper explicitly describes what the comparison (control) group received (read-only e-book sessions with no embedded instruction), its sample size, and statistically tests baseline equivalence between groups on demographic and language variables as well as standardized pretest scores, with results broken down by condition in Table 3. Final sentence: Criterion D is met because the control group's composition, baseline characteristics, and conditions received are clearly and quantitatively documented.

• Level 2 Criteria

  • S

    School-level RCT

    • Randomisation occurred among individual students within each school, not among whole schools.
    • "Students were randomly assigned to conditions within grade and schools." (p. 1948)
    • Relevant Quotes: 1) "Students were randomly assigned to conditions within grade and schools." (p. 1948) 2) "Children were enrolled in eight participating elementary schools located in Florida (196 children) and Kansas City, Kansas (95 children)." (p. 1949) Detailed Analysis: All eight participating schools contained both treatment and comparison children; no school as a whole was assigned entirely to one condition. The unit of randomisation was the individual student within grade and school, which is weaker than the school-level randomisation required by criterion S. Final sentence: Criterion S is not met because randomisation occurred at the student level within schools rather than across whole schools.
  • I

    Independent Conduct

    • The same research team that designed the BLOOM e-book intervention also implemented, collected data for, and analysed the study, with no independent evaluator described.
    • "The current study was part of Project BLOOM (Bridging for Language Outcomes in the Classroom), funded through a grant from the Institute of Education Sciences, U.S. Department of Education..." (p. 1948)
    • Relevant Quotes: 1) "The current study was part of Project BLOOM (Bridging for Language Outcomes in the Classroom), funded through a grant from the Institute of Education Sciences, U.S. Department of Education, focusing on language and literacy interventions for bilingual elementary children." (p. 1948) 2) "a Florida State University, Tallahassee" — all six authors are listed with the same institutional affiliation. (p. 1945) 3) "Examiners recorded responses, which were scored by three research assistants who were blind to the participants' assignment condition." (p. 1953) Detailed Analysis: The intervention was designed, implemented, and evaluated by the same Florida State University research team (Project BLOOM), funded directly to that team via an IES grant. While the three research assistants who scored the open-ended definition probes were blind to condition (a partial rater-blinding safeguard for one outcome measure), the trial overall — including intervention design, delivery, primary data collection, and statistical analysis — was conducted in-house by the intervention's developers, with no external/independent evaluation organisation described. Final sentence: Criterion I is not met because the same team that designed the intervention also conducted the trial, with only partial rater-blinding rather than fully independent conduct.
  • Y

    Year Duration

    • The paper does not document that outcomes were tracked for at least 75% of an academic year from intervention start, and delivered intervention duration varied widely and averaged only about four months.
    • "Several schools did not enroll in the project until later in the fall, resulting in fewer weeks of treatment being delivered to specific schools." (p. 1953)
    • Relevant Quotes: 1) "Several schools did not enroll in the project until later in the fall, resulting in fewer weeks of treatment being delivered to specific schools." (p. 1953) 2) "The comparison group received 17.69 weeks of treatment on average (SD = 3.89 weeks), and the intervention group received 16.83 weeks on average (SD = 4.35 weeks)." (p. 1953) 3) "School D ... 10.39 (2.92)" weeks of treatment on average, the lowest of all participating schools. (Appendix C, p. 1965) Detailed Analysis: The paper never states specific calendar start and end dates for the intervention, only that baseline testing occurred "in September" and the PPVT-4 posttest occurred "in spring." The actually delivered intervention averaged 16.83–17.69 weeks (roughly four months), with some schools (e.g., School D) delivering as few as about 10 weeks, well short of the ~75% of an academic year (about 7-9 months) required by criterion Y. There is no evidence of tracking or measurement extending across a near-full academic year. Final sentence: Criterion Y is not met because the documented intervention and follow-up period falls well short of 75% of an academic year.
  • B

    Balanced Control Group

    • The additional instructional time in the treatment e-books is integral to the specific instructional package being tested, while both groups received the same books at the same weekly frequency.
    • "This design allows for the examination of the added impact of the enhanced instructional components embedded in the e-books, controlling for the impact of repeated reading alone." (p. 1951)
    • Relevant Quotes: 1) "Treatment sessions were approximately 25 min in length." (p. 1950) 2) "The comparison condition sessions were approximately 10–15 min in length." (p. 1951) 3) "This design allows for the examination of the added impact of the enhanced instructional components embedded in the e-books, controlling for the impact of repeated reading alone." (p. 1951) 4) "All children received e-book readings approximately 3 times a week for 10–20 weeks using the same books." (Abstract, p. 1945) Detailed Analysis: Applying the criterion B decision procedure: extra resources are present (treatment sessions run roughly 25 minutes versus 10-15 minutes for comparison, reflecting the added Spanish bridging, definitions, video, and word-map components). However, these additional minutes are not a separable resource added on top of the intervention — they are simply the time needed to deliver the specific instructional components (bridging, scaffolding, morphology highlighting) that constitute the treatment itself, and the authors explicitly frame the design as isolating "the added impact of the enhanced instructional components ... controlling for the impact of repeated reading alone." Both groups received identical book titles at the identical weekly frequency (3x/week) for a comparable number of weeks, with the comparison condition serving as the "business as usual" shared-reading baseline against which the enriched instructional package is tested. Final sentence: Criterion B is met because the additional instructional time is integral to the treatment package under test, and both groups otherwise received equivalent book exposure and session frequency.

• Level 3 Criteria

  • R

    Reproduced

    • No independent replication of this specific study or intervention by a different research team was found in the paper or in an external search.
    • Detailed Analysis: The paper itself makes no reference to any prior or subsequent independent replication of the BLOOM e-book bridging intervention. An external search (2026) confirmed a What Works Clearinghouse (WWC) review of this same original study (Wood et al., 2018, Study ID 86137), which explicitly rates the original study as "meets WWC standards without reservations" and is a methodological quality review/rating of the existing study, not an independent empirical replication of its findings by a different research team in a different context. A related e-book vocabulary study, "Vocabulary enrichment using an E-book with and without kindergarten teacher's support among LSES children" (Segal-Drori et al., Early Child Development and Care, Vol. 192, No. 9, 2022), was also located; it tests a different intervention design (teacher-supported e-book vs. e-book alone, 103 Israeli LSES kindergarten children) with a different population and does not replicate this specific study's design, population (Spanish-English ELs), or L1-bridging methodology. No independent, peer-reviewed replication of this specific study was identified. Final sentence: Criterion R is not met because no independent replication of this specific study was found in the paper or through external search.
  • A

    All-subject Exams

    • The study assessed only vocabulary/language outcomes and did not measure performance in other core academic subjects, with no specialization exception offered.
    • "The primary aim of the current study was to examine the English vocabulary growth of kindergarten and first-grade EL students who participated in an intensive, L1-enhanced vocabulary intervention delivered via e-book three times a week." (p. 1948)
    • Relevant Quotes: 1) "The primary aim of the current study was to examine the English vocabulary growth of kindergarten and first-grade EL students who participated in an intensive, L1-enhanced vocabulary intervention delivered via e-book three times a week." (p. 1948) 2) Table 3 lists only vocabulary/language- related outcome and descriptive measures (TVIP, WRMT-III subtests, BESA sentence repetition, PPVT-4, labeling, and definitions); no mathematics, science, or other core subject assessments are reported. (p. 1952) Detailed Analysis: While criterion E is met via the PPVT-4, the study's outcome measures are entirely confined to English (and Spanish) vocabulary and related language constructs. No other core subjects (e.g., mathematics, science, social studies) were assessed, and the paper offers no rationale of the kind allowed for specialized upper- secondary/vocational interventions to justify this narrow focus, since the participants are kindergarten and first-grade children in general education classrooms. Final sentence: Criterion A is not met because only vocabulary/language outcomes were assessed, with no justified exception for excluding other core subjects.
  • G

    Graduation Tracking

    • Outcomes were tracked only within the single school year of the intervention, with no follow-up toward graduation and no year-long tracking established; criterion Y is also not met, so this criterion cannot be met per the standard's dependency rule.
    • "Furthermore, the current design did not allow for measurement of retention over a longer period. Examination of the children's word knowledge after the summer or a school year later would inform the broader importance of this work." (p. 1958)
    • Relevant Quotes: 1) "Furthermore, the current design did not allow for measurement of retention over a longer period. Examination of the children's word knowledge after the summer or a school year later would inform the broader importance of this work." (p. 1958) Detailed Analysis: The authors explicitly identify the lack of any longer-term follow-up as a limitation, with measurement ending at the spring posttest within the same school year that the intervention took place. There is no tracking toward graduation from kindergarten/first grade or any subsequent educational stage. An external search (2026) for follow-up publications by the same author team (Wood, Fitton, Petscher et al. / Project BLOOM, Florida State University) tracking this same cohort of 288 children into later grades did not identify any such longitudinal follow-up paper; only unrelated subsequent work by overlapping authors on different samples was found. Because criterion Y (Year Duration) is also not met, this criterion cannot be met per the standard's dependency rule. Final sentence: Criterion G is not met because the study explicitly lacks any longer-term or graduation-level follow-up, no subsequent cohort-tracking publication was found, and criterion Y is not met.
  • P

    Pre-Registered

    • The paper documents institutional ethics approval but provides no evidence of a pre-registered study protocol, and no registration record was located in an external search.
    • Relevant Quotes: 1) "The study was reviewed and approved by the university's institutional review board for research involving human subjects (HSC 2016.18265)." (p. 1948) Detailed Analysis: The only regulatory reference in the paper is to institutional review board (IRB) approval, which concerns ethical oversight of human-subjects research rather than pre-registration of hypotheses, methods, and analysis plans on a public registry (e.g., a registry ID or registration date is never mentioned anywhere in the manuscript). An external search (2026) of common trial and study registries (e.g., ClinicalTrials.gov, OSF Registries) did not locate any pre-registration record for this study or Project BLOOM under the authors' names. Final sentence: Criterion P is not met because no pre-registration reference, registry ID, or registration date is provided anywhere in the paper, and no external registration record was found.