Unique Effects of a Family Literacy Program on the Early Reading Development of English Language Learners

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • The control group was not randomly assigned but was self-selected based on families' inability to attend, so this is not a randomised controlled trial.
    • "A third group of families who agreed to participate in the research but did not attend the fall or spring program (because of parents' work schedule, conflicts with other activities, child care issues, etc.) were assigned to a no-program control group." (p. 996)
    • Relevant Quotes: 1) "Because of the large number of interested families, families were randomly assigned to either the fall or spring program, which were identical." (p. 996) 2) "A third group of families who agreed to participate in the research but did not attend the fall or spring program (because of parents' work schedule, conflicts with other activities, child care issues, etc.) were assigned to a no-program control group." (p. 996) 3) "Future studies investigating the effects of Family Literacy programs should attempt to randomly assign families to experimental and control groups." (p. 1005) Detailed Analysis: The only randomisation described in this study is between two identical timings of the SAME experimental program (fall vs. spring). The control group was not formed through randomisation at all; it was composed of families who self-selected out of attending because of scheduling conflicts, work, or child-care issues. This is a non-equivalent, self-selected comparison group, not a randomly allocated control condition. The authors themselves explicitly acknowledge this limitation in the Discussion, recommending that "future studies... attempt to randomly assign families to experimental and control groups," which confirms that the present study did not do so. Since there is no random assignment to intervention versus control at any level (student, class, or school), and the intervention here is a parent/family-directed program rather than a one-to-one tutoring exception, the class-level RCT criterion is not met. All three quotes were checked against the source PDF and are verbatim. Criterion C is not met because families were not randomly assigned to intervention versus control; the comparison group was self-selected based on non-attendance.
  • E

    Exam-based Assessment

    • The study used the TERA-3, a standardized and validated commercially published test of early reading ability.
    • "Children's pre- and posttest scores from the alphabet, conventions, and meaning subtests of the Test of Early Reading Ability-Third Edition (TERA-3...) were examined..." (p. 993-994)
    • Relevant Quotes: 1) "Children's pre- and posttest scores from the alphabet, conventions, and meaning subtests of the Test of Early Reading Ability-Third Edition (TERA-3; Reid, Hresko, & Hammill, 2001) were examined individually..." (p. 993-994) 2) "The authors of the TERA-3 (Reid et al., 2001) have reported sufficient content, construct, and criterion-predictive validity and strong reliabilities of .91, .83, and .90 for the alphabet, conventions, and meaning subtests, respectively." (p. 996) 3) "Children completed Version A of the TERA-3 at pretest and Version B at posttest in order to control for potential practice effects." (p. 997) Detailed Analysis: The outcome measure used is the TERA-3, a commercially published, standardized, and independently validated instrument (published by PRO-ED) with documented reliability and validity statistics, not a custom test created specifically for this study. Two parallel forms were used to control for practice effects, which is standard practice for a recognised psychometric instrument. This satisfies the requirement for a standardised, widely recognised assessment tool. All quotes were checked against the source PDF and are verbatim (quote 1 spans the page break from 993 to 994). Criterion E is met because the study used the TERA-3, a standardized, independently published and validated early reading assessment.
  • T

    Term Duration

    • The paper does not clearly document a term-long interval between intervention start and outcome measurement, and the spring cohort's own window appears to be about 9-11 weeks, shorter than a full term.
    • "The program ran for 90 min, once a week, for 9 weeks... Following the completion of the spring program, the researchers returned to the school and individually administered the same language and literacy tests..." (p. 997)
    • Relevant Quotes: 1) "Before the fall program began, researchers visited the schools and met with each student individually to administer a selection of assessments related to language and literacy." (p. 996) 2) "Parents and children in the experimental group then participated in the 9-week Family Literacy program during the fall or spring term." (p. 996-997) 3) "The program ran for 90 min, once a week, for 9 weeks..." (p. 997) 4) "Following the completion of the spring program, the researchers returned to the school and individually administered the same language and literacy tests to all children involved in the study." (p. 997) Detailed Analysis: The paper does not give explicit calendar dates for when the fall or spring programs began and ended, so the exact intervention-to-measurement interval cannot be precisely calculated. For families in the spring cohort, the 9-week (approximately 2-month) program is described as running "during... the spring term," and the posttest was administered "following the completion of the spring program" -- i.e., immediately/soon after their own intervention ended. This implies an intervention-to-measurement interval for the spring cohort of roughly 9-11 weeks, which is shorter than one academic term (approximately 3-4 months) as defined by the ERCT standard. While the fall cohort's own posttest would occur later (since posttesting is a single wave after the spring program concludes), the paper does not clearly and separately document a term-long intervention-to-measurement interval for all participants, and the shortest documented case (the spring cohort) falls below the term threshold. Because the paper does not provide unambiguous dates confirming a term-long interval for the whole sample, and the clearest reading of the spring cohort's own timeline suggests a shorter-than-term interval, this criterion is not clearly satisfied. All four quotes were checked against the source PDF and are verbatim. Criterion T is not met because the paper does not clearly document a term-long interval between intervention start and outcome measurement for the full sample, and the spring cohort's own intervention-to -posttest window appears to be only around 9-11 weeks.
  • D

    Documented Control Group

    • The control group's composition, reasons for non-participation, and baseline demographic/score comparability with the experimental group are clearly documented.
    • "At pretest there were no significant differences between children in the experimental and control groups related to age... gender... parents' education... ELL status... or school..." (p. 999)
    • Relevant Quotes: 1) "A third group of families who agreed to participate in the research but did not attend the fall or spring program (because of parents' work schedule, conflicts with other activities, child care issues, etc.) were assigned to a no-program control group." (p. 996) 2) "At pretest there were no significant differences between children in the experimental and control groups related to age... gender... parents' education... ELL status... or school..." (p. 999) 3) "Table 1. TERA-3 Means (SD) for ELL and EL1 Experimental and Control Children at Pre- and Posttest (N = 132)" (p. 1000, Table 1) 4) "It should also be noted that the control group was composed of families who expressed an interest in participating in the program but were unable to attend because of parents' work schedules, conflicts with other activities, or child care issues." (p. 1005) Detailed Analysis: The paper provides a clear description of who constitutes the control group (interested families unable to attend, for stated reasons), reports formal statistical comparisons showing no significant baseline differences between experimental and control groups on age, gender, parents' education, ELL status, and school, and provides baseline (pretest) means and standard deviations for the control group broken down by language status in Table 1. This constitutes a reasonably detailed documentation of the control group's composition, demographics, and baseline performance, even though the group was not randomly formed (a separate concern captured under Criterion C). All four quotes were checked against the source PDF and are verbatim. Criterion D is met because the control group's composition, baseline demographic comparability, and baseline test scores are clearly documented.

• Level 2 Criteria

  • S

    School-level RCT

    • Randomisation occurred at the family level within the same seven schools, not at the school level, and control status was not randomly assigned.
    • "Because of the large number of interested families, families were randomly assigned to either the fall or spring program, which were identical." (p. 996)
    • Relevant Quotes: 1) "Parents and children were recruited from seven schools that agreed to host a lunchtime Family Literacy program." (p. 996) 2) "Because of the large number of interested families, families were randomly assigned to either the fall or spring program, which were identical." (p. 996) 3) "A third group of families who agreed to participate in the research but did not attend the fall or spring program... were assigned to a no-program control group." (p. 996) Detailed Analysis: Although seven schools hosted the program, randomisation (such as it exists, between fall and spring timing) was at the family/individual level, not at the school level, and no schools were randomly assigned wholesale to intervention or control conditions -- all seven schools hosted the intervention program for some families. The control group, as established under Criterion C, was also not randomly formed. There is no school-level random allocation of entire schools to intervention or control status. All three quotes were checked against the source PDF and are verbatim. Criterion S is not met because randomisation, where present, occurred at the family level within schools, not at the school level, and the control group was self-selected rather than randomly assigned.
  • I

    Independent Conduct

    • The intervention curriculum was co-developed by one of the study's own authors (Pelletier), and there is no evidence of independent, third-party conduct of the evaluation.
    • "The Family Literacy program (Doyle, Hipfner-Boucher, & Pelletier, 2008) was a research-informed curriculum..." (p. 997)
    • Relevant Quotes: 1) "The Family Literacy program (Doyle, Hipfner-Boucher, & Pelletier, 2008) was a research-informed curriculum that featured joint parent-child activities related to language and literacy development..." (p. 997) 2) "Sarah Harper, Amy Platt, and Janette Pelletier, Department of Human Development and Applied Psychology, Ontario Institute for Studies in Education, University of Toronto" (p. 989) 3) "Doyle, A., Hipfner-Boucher, K., & Pelletier, J. (2008). Learning begins at home: A research-based family literacy program curriculum. Unpublished curriculum." (References, p. 1006) Detailed Analysis: The Family Literacy curriculum used as the intervention was itself co-authored by Janette Pelletier, who is also a co-author (and evidently senior/supervising author, given the correspondence and affiliation) of the present evaluation paper. There is no mention of an external, third-party team conducting data collection or analysis independent of the intervention's designers; the same research group that created the "Learning begins at home" curriculum also designed, implemented, and evaluated this study. No statement of independent or blinded assessment administration is provided. All three quotes were checked against the source PDF and are verbatim. Criterion I is not met because the same author (Pelletier) who co-developed the Family Literacy program curriculum also co-authored and conducted this evaluation, with no indication of independent, external data collection or analysis.
  • Y

    Year Duration

    • Because criterion T (Term Duration) is not met, criterion Y (Year Duration) is automatically not met per the ERCT standard's dependency rule.
    • Relevant Quotes: 1) "The program ran for 90 min, once a week, for 9 weeks, during the school's scheduled lunch period." (p. 997) 2) "Following the completion of the spring program, the researchers returned to the school and individually administered the same language and literacy tests to all children involved in the study." (p. 997) Detailed Analysis: Per the ERCT specification, if criterion T (Term Duration) is not met, criterion Y (Year Duration) is automatically not met. As established above, the paper does not clearly document a term-long interval between intervention start and outcome measurement for the whole sample, with the spring cohort's own window appearing to be only about 9-11 weeks. There is no explicit statement that outcomes were tracked for at least 75% of an academic year from each participant's own intervention start date. Both quotes were checked against the source PDF and are verbatim. Criterion Y is not met because criterion T is not met, and there is no clear documentation of a near-year-long tracking interval from intervention start to measurement.
  • B

    Balanced Control Group

    • The additional time and resources given to the experimental group are the explicit treatment variable (the Family Literacy program itself) being tested against a business-as-usual no-program control.
    • "Families in the experimental group participated in a 9-week intervention program that was designed to promote early literacy in the home, whereas those in the control group did not receive the intervention." (p. 989)
    • Relevant Quotes: 1) "Families in the experimental group participated in a 9-week intervention program that was designed to promote early literacy in the home, whereas those in the control group did not receive the intervention." (p. 989, Abstract) 2) "The aim of the current study was to evaluate the effects of a Family Literacy program on ELL children's early reading development..." (p. 993) 3) "A pizza lunch was provided to all families at the start of the program each week." (p. 997) 4) "A third group of families who agreed to participate in the research but did not attend the fall or spring program... were assigned to a no-program control group." (p. 996) Detailed Analysis: The intervention group received additional resources relative to the control group: 90-minute weekly sessions for 9 weeks, program materials, facilitator time, and a weekly pizza lunch. However, the explicit purpose of this study, as stated in the abstract and aim, is to evaluate the effect of providing the Family Literacy program itself, compared against a "no-program" business-as-usual condition. The additional time, materials, and facilitator support are not a separable add-on alongside some other core intervention; they ARE the treatment variable being tested (attendance in the Family Literacy program versus no program). Re-applying the current ERCT decision tree for Criterion B: EXTRA_RESOURCES_PRESENT is true (extra facilitator time, materials, pizza); the difference is not negligible; but RESOURCES_ARE_TREATMENT is true, because the study explicitly and centrally tests the effect of providing the Family Literacy program (with its associated time and materials) against a business-as-usual "no-program" control that receives none of those resources, and this is clearly stated as the study's intent throughout the abstract, aim, and methods. Per the decision tree, this satisfies the criterion regardless of whether the control group received matching resources. All quotes were checked against the source PDF and are verbatim. Criterion B is met because the additional time and resources given to the experimental group constitute the explicit treatment variable (participation in the Family Literacy program) being tested against a business-as-usual no-program control.

• Level 3 Criteria

  • R

    Reproduced

    • No independent replication of this specific study's design and findings was identified in the paper or in a citation-record search, and the authors themselves call for future replication of the results.
    • Relevant Quotes: 1) "Although there is currently no quantitative research that directly measures ELL children's reading development as a result of a Family Literacy program..." (p. 993) 2) "An attempt to replicate the findings with other programs, other measures, and samples of diverse language groups might provide additional evidence for the positive effects of Family Literacy on ELL and EL1 children's early literacy development." (p. 1005, Discussion/Limitations) Detailed Analysis: The authors themselves state in the Discussion that this is a novel finding requiring future replication ("An attempt to replicate the findings... might provide additional evidence"), confirming that no independent replication existed at the time of publication. Verification search (Semantic Scholar/OpenAlex citation records, checked August 2026): reviewed the approximately 38 papers citing this study for evidence of an independent replication. None reproduces this specific design (the differential effect of the "Learning Begins at Home" family literacy curriculum on ELL versus EL1 kindergartners' TERA-3 gains). The closest candidate, Vojtkova, Morris, Cara, & Marshall (2016), "Research Protocol: A cluster randomised controlled trial to evaluate the Family SKILLS programme for reception year students from families in which English is an additional language," evaluates a different intervention (Family SKILLS, UK reception-year families) rather than replicating Harper, Platt, and Pelletier's curriculum or findings, so it does not constitute an independent reproduction of this specific study. No other peer-reviewed replication by an independent research team was identified in available sources. Criterion R is not met because no independent replication of this specific study's design and findings was found, and the authors themselves call for future replication.
  • A

    All-subject Exams

    • Only reading/literacy-related outcomes were measured; no other core subjects such as early numeracy were assessed.
    • "Dependent variables included children's gains in alphabet knowledge, conventions of print, and meaning." (p. 989)
    • Relevant Quotes: 1) "Dependent variables included children's gains in alphabet knowledge, conventions of print, and meaning." (p. 989, Abstract) 2) "Children's pre- and posttest scores from the alphabet, conventions, and meaning subtests of the Test of Early Reading Ability-Third Edition (TERA-3...) were examined individually to determine whether language group differences existed in these three component areas." (p. 993-994) Detailed Analysis: All outcome measures in this study are subtests of a single reading assessment (TERA-3): alphabet knowledge, conventions of print, and meaning/comprehension. No other core early-years subjects (e.g., early numeracy/math, general knowledge/science) were assessed, even though the program's final session ("Fun With Math and Celebration") touches on early math concepts informally. There is no stated rationale that this is a highly specialised secondary/vocational intervention that would justify narrowing the outcome measures to a single domain; rather, it is a general early-literacy program for young children. As all outcomes fall within the single subject area of early reading/literacy, the all-subject requirement is not satisfied. Both quotes were checked against the source PDF and are verbatim. Criterion A is not met because only early reading/literacy outcomes (alphabet, print conventions, meaning) were assessed, with no measurement of other core early subjects such as numeracy.
  • G

    Graduation Tracking

    • The study only tracked children through a single posttest within the same kindergarten year, with no follow-up toward graduation found in the paper or in a search of subsequent publications by the same authors, and criterion Y is also not met.
    • Relevant Quotes: 1) "Following the completion of the spring program, the researchers returned to the school and individually administered the same language and literacy tests to all children involved in the study." (p. 997) 2) "There are a number of limitations associated with the findings of the current study." (p. 1004) Detailed Analysis: The study's outcome data collection ends with a single posttest administered shortly after the spring program concludes, i.e., still within the same kindergarten year as the intervention. There is no mention of any follow-up assessment in subsequent grades, let alone tracking through to graduation from primary or secondary school. Verification search (Semantic Scholar/OpenAlex authorship and citation records, checked August 2026): reviewed subsequent publications by Harper, Platt, and Pelletier for evidence of a longer-term follow-up of this same 132-child kindergarten cohort. No such follow-up study was identified. The authors' other related work found in citation records (e.g., Harper & Pelletier, 2010, on parent involvement; Harper, 2011, on a separate French Immersion Grade 1 sample) evaluates different samples and does not track this cohort toward graduation. No follow-up publication reporting graduation-stage outcomes for this cohort was found in available sources. Criterion G is not met because tracking ended with a single within-kindergarten-year posttest, with no evidence of any longer-term or graduation follow-up identified, and because criterion Y is not met.
  • P

    Pre-Registered

    • The paper contains no statement or reference indicating that the study protocol was pre-registered before data collection began, and no matching registration was found in a registry search.
    • Relevant Quotes: (No quotes found relevant to pre-registration; the paper contains no mention of a trial registry, registration date, or pre-specified analysis plan anywhere in the Method, Results, or Discussion sections.) Detailed Analysis: A full review of the Method and Procedures sections found no reference to any pre-registration platform (e.g., ClinicalTrials.gov, ISRCTN, OSF Registries, AEA RCT Registry) or any statement that the study's hypotheses, methods, or analysis plan were registered prior to data collection. Verification search (checked August 2026): searched ClinicalTrials.gov for a registration matching this study (title, authors, or the "Learning Begins at Home" family literacy curriculum); no matching registration was found. Given the paper's 2011 publication date, prior to widespread adoption of pre-registration norms in education research, and the complete absence of any registry reference in the paper itself, this criterion cannot be considered met. Criterion P is not met because no pre-registration statement, registry link, or registration date is provided anywhere in the paper, and no matching registration was found in a registry search.