Nuffield Early Language Intervention: Evaluation report and executive summary

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomisation was conducted at the individual pupil level within each nursery/school rather than at the class or school level, and the authors themselves flag this as creating a risk of contamination between treatment and control pupils.
    • "The trial was designed as a randomised controlled trial, with randomisation at the pupil level within nurseries attached to schools." (p. 11)
    • Relevant Quotes: 1) "The trial was designed as a randomised controlled trial, with randomisation at the pupil level within nurseries attached to schools." (p. 11) 2) "The randomisation was undertaken independently by members of the impact evaluation team who randomly allocated pupils within each nursery to one of three groups—a control group or one of two treatment groups." (p. 11) 3) "The main downside of within-school randomisation is that spillover effects are more likely resulting from, for example, TAs applying the intervention techniques to pupils in the control group." (p. 11) 4) "Randomisation was at the child level within nurseries/schools." (p. 4, Executive summary) Detailed Analysis: The ERCT 'C' criterion requires randomisation at the class level or stronger (school level), with an exception only where the intervention is designed for personal, one-to-one tutoring. Here, randomisation was explicitly conducted at the individual pupil level within each nursery, meaning treatment and control children came from the same classes/settings. The intervention is delivered mainly in small groups of two to four children (3 x 20-30 minute sessions per week), supplemented in reception only by two 15-minute individual sessions per week; it is therefore not a purely one-to-one personal tutoring intervention that would qualify for the stated exception. Critically, the authors themselves acknowledge that this within-nursery/school, pupil-level design creates a real risk of spillover, i.e. TAs delivering intervention techniques to control pupils in the same setting, which is precisely the contamination problem criterion C is designed to prevent. Final sentence: Criterion C is not met because randomisation occurred at the individual pupil level within the same nursery/school, with no valid one-to-one tutoring exception, and the authors explicitly acknowledge the resulting contamination risk.
  • E

    Exam-based Assessment

    • The primary and secondary outcomes were measured using established, externally-valid standardised instruments (CELF, Renfrew APT, YARC), not custom-built tests.
    • "The primary outcome for this evaluation is a language skills score which is a composite score of four different externally-valid measures of language skill:" (p. 13)
    • Relevant Quotes: 1) "The primary outcome for this evaluation is a language skills score which is a composite score of four different externally-valid measures of language skill: Renfrew Action Picture Test (APT)... CELF-Preschool 2 UK: Expressive Vocabulary... Listening Comprehension (based on the York Assessment of Reading Comprehension test, YARC)." (p. 13) 2) "YARC: Letter Knowledge... YARC: Early Word Reading... Spelling: children are asked to write a series of simple words." (p. 13) 3) "All these component measures are standardised, age appropriate, and were chosen by the project team to be consistent with the measures used in their Randomised Controlled Trial (Fricke et al., 2013), and the aims of the intervention to improve language and literacy skills." (p. 13) 4) "All tests were administered and scored by research assistants trained by the project team who were blind to the allocation of children to groups." (p. 13) Detailed Analysis: The outcomes are drawn from recognised, externally-published standardised psychometric instruments (the Renfrew Action Picture Test, CELF-Preschool, and YARC), each with established norms and prior use in other studies, rather than being custom-designed solely for this trial. Tests were administered by blinded, trained assessors, further supporting objectivity. Final sentence: Criterion E is met because the study used widely-used, externally-valid standardised assessments (CELF, Renfrew APT, YARC) rather than bespoke measures.
  • T

    Term Duration

    • Outcomes were measured well over a full academic term after intervention start in both arms; since the stronger Year Duration criterion is met, Term Duration is automatically considered met.
    • "Pre-test: this was conducted in April 2013 just before the start of the intervention phase... Post-test: undertaken between May and July 2014 after the end of the intervention phase." (p. 13)
    • Relevant Quotes: 1) "Pre-test: this was conducted in April 2013 just before the start of the intervention phase." (p. 13) 2) "Post-test: undertaken between May and July 2014 after the end of the intervention phase." (p. 13) 3) "Pupils in one treatment group received a 30-week programme starting in the final term of nursery and continuing for the first two terms of reception year in primary school. Pupils in the second treatment group received a 20-week programme that ran during the first two terms of primary school." (p. 6) Detailed Analysis: For both the 30-week arm (starting April 2013, measured May-July 2014, over a year later) and the 20-week arm (starting September 2013, measured May-July 2014, roughly 8-9 months later), the interval from intervention start to outcome measurement clearly exceeds one academic term. As the Year Duration criterion (Y) is also met (see below), the ERCT standard states the weaker Term Duration criterion is automatically satisfied. Final sentence: Criterion T is met because the measured interval from intervention start to post-test substantially exceeds one academic term in both trial arms, and because Year Duration (Y) is separately met.
  • D

    Documented Control Group

    • The control ("waitlist") group is extensively documented, including its business-as-usual status and detailed baseline demographic and language characteristics compared with treatment groups.
    • "Children in the control group received no additional language support during the trial beyond that normally received in a business-as-usual scenario, but their language and word-level literacy skills were monitored in the same way as the two treatment groups." (p. 8)
    • Relevant Quotes: 1) "Children in the control group received no additional language support during the trial beyond that normally received in a business-as-usual scenario, but their language and word-level literacy skills were monitored in the same way as the two treatment groups." (p. 8) 2) Table 3: "% female 49.2% (30-week) / 48.1% (20-week) / 48.8% (Control); Average age in months 47.4 / 47.4 / 47.4; Average language composite score -0.065 / -0.073 / -0.090." (p. 16) 3) Table 9 provides extensive baseline characteristics for the control group versus both treatment groups, including EAL status, known speech/language difficulties, FSM eligibility, SEN status, ethnicity, and pre-test scores on all language and literacy components. (pp. 25-27) 4) "A waitlist control group was chosen to address the ethical issues of identifying a group of struggling pupils and then not providing them any additional support." (p. 11) Detailed Analysis: The control group's composition, size, demographic characteristics, baseline test scores, and treatment status (business-as-usual, later offered an alternative literacy intervention) are all thoroughly documented and directly compared with the treatment arms across multiple detailed tables. Final sentence: Criterion D is met because the control group is comprehensively documented, including demographics, baseline scores, and confirmation of business-as-usual conditions.

• Level 2 Criteria

  • S

    School-level RCT

    • Randomisation was conducted at the individual pupil level within nurseries/schools, not at the school level.
    • "The trial was designed as a randomised controlled trial, with randomisation at the pupil level within nurseries attached to schools." (p. 11)
    • Relevant Quotes: 1) "The trial was designed as a randomised controlled trial, with randomisation at the pupil level within nurseries attached to schools." (p. 11) 2) "Randomisation was at the child level within nurseries/schools." (p. 4, Executive summary) 3) "The main reasons for adopting within-school randomisation, as opposed to an across-school randomisation, were the risks created by differential attrition and the improved statistical power resulting from within-school randomisation." (p. 11) Detailed Analysis: The paper explicitly and repeatedly confirms that randomisation occurred within each nursery/school, at the level of the individual pupil, and that this was a deliberate design choice made instead of across-school randomisation (which the authors explicitly considered and rejected). This directly contradicts the requirement for school-level randomisation. Final sentence: Criterion S is not met because the study used within-school, pupil-level randomisation rather than randomising whole schools.
  • I

    Independent Conduct

    • The trial was independently evaluated by the Institute for Fiscal Studies and NatCen Social Research, separate from the University of York/I CAN team that designed and delivered the intervention, with blinded test administration.
    • "The project was independently evaluated by a team from the Institute for Fiscal Studies. The lead evaluator was Luke Sibieta." (p. 2)
    • Relevant Quotes: 1) "The project was independently evaluated by a team from the Institute for Fiscal Studies. The lead evaluator was Luke Sibieta." (p. 2) 2) "The randomisation was undertaken independently by members of the impact evaluation team who randomly allocated pupils within each nursery to one of three groups—a control group or one of two treatment groups." (p. 11) 3) "All tests were administered and scored by research assistants trained by the project team who were blind to the allocation of children to groups." (p. 13) 4) "Project and delivery team Professor Charles Hulme, University College London Dr. Silke Fricke, University of Sheffield Dr. Claudine Bowyer-Crane, University of York Mandy Grist, I CAN Communication Advisor Professor Margaret Snowling, University of Oxford." (p. 10) Detailed Analysis: The intervention was developed and delivered by a separate "project and delivery team" (University of York/Sheffield/Oxford researchers and I CAN), while the impact evaluation, randomisation, and testing were conducted independently by the Institute for Fiscal Studies (quantitative) and NatCen Social Research (process evaluation), with test administrators blind to group allocation. This separation of intervention delivery from independent evaluation and analysis satisfies the independence requirement, similar to accepted exception cases where the developer and evaluator are organisationally distinct. Final sentence: Criterion I is met because randomisation, testing, and analysis were conducted independently by IFS/NatCen, separate from the team that designed and delivered the intervention, with blinded test administrators.
  • Y

    Year Duration

    • Outcomes were measured over 13-14 months for the 30-week arm and roughly 8-9 months for the 20-week arm, both close to or exceeding a full academic year from intervention start.
    • "Pre-test: this was conducted in April 2013 just before the start of the intervention phase... Post-test: undertaken between May and July 2014 after the end of the intervention phase." (p. 13)
    • Relevant Quotes: 1) "Pre-test: this was conducted in April 2013 just before the start of the intervention phase." (p. 13) 2) "Post-test: undertaken between May and July 2014 after the end of the intervention phase." (p. 13) 3) "20-week intervention begins" - September 2013; "Post-test data collected" - May-July 2014 (Table 5, Project Timeline, p. 21) 4) "Pupils in one treatment group received a 30-week programme starting in the final term of nursery and continuing for the first two terms of reception year in primary school." (p. 6) Detailed Analysis: For the 30-week arm, the interval from intervention start (April 2013) to post-test (May-July 2014) is around 13-14 months, well over a full academic year. For the 20-week arm, the intervention began in September 2013 and was tracked through to the May-July 2014 post-test, i.e. essentially the whole reception academic year (~9-10 months), which the standard's 75% threshold (~7 months) is comfortably met by. Final sentence: Criterion Y is met because outcome measurement occurred at or beyond a full academic year after intervention start for both the 30-week and 20-week arms.
  • B

    Balanced Control Group

    • The additional teaching time is the core treatment variable explicitly being tested against a business-as-usual control, consistent with the ERCT exception for interventions that test additional resources as the treatment itself.
    • "Children in the control group received no additional language support during the trial beyond that normally received in a business-as-usual scenario." (p. 8)
    • Relevant Quotes: 1) "Children in the control group received no additional language support during the trial beyond that normally received in a business-as-usual scenario, but their language and word-level literacy skills were monitored in the same way as the two treatment groups." (p. 8) 2) "This shorter version was included in the evaluation as it potentially represents a lower cost, and therefore more scalable, version of the intervention." (p. 9) 3) "Group and individual sessions were usually delivered during lesson time meaning that pupils were taken out of another class... We should therefore interpret the effects of the interventions relative to a business-as-usual scenario where pupils would have spent the time in the classes." (p. 12) 4) "Schools in the trial were given the opportunity to deliver the Reading and Language Intervention (RALI) to pupils from the control group after the end of the trial." (p. 8) Detailed Analysis (applying the criterion B decision procedure): Step 1 - are extra resources present? Yes: the intervention groups receive a large amount of dedicated additional teaching time (up to 110 hours across 30 weeks) that the control group does not receive at all, so this is not a negligible difference. Step 2 - are the extra resources the treatment variable itself, integral to the intervention as defined? The whole of the "Nuffield Early Language Intervention" IS a structured programme of additional small-group and individual teaching sessions; the study's stated aim is "to test whether the provision of the Nuffield Early Language Intervention improved the language and literacy skills of children." There is no separate pedagogical technique being tested independently of the extra time invested - the additional teaching time is not a supplementary add-on to some other core method, it is the intervention. This mirrors the ERCT exception examples (e.g. PAX-GBG reward activities, RAMSR teacher training, DPL devices) where additional resources integral to a named intervention package do not require a matched control. Step 3 - control group treatment: consistent with resources-as-treatment designs, the control group received the standard "business as usual" level (no extra sessions), which is the appropriate comparator, with schools later offered a compensatory alternative intervention (RALI) as an ethical safeguard rather than as part of the trial comparison itself. Final sentence: Criterion B is met because the additional teaching time is explicitly the treatment variable under investigation and is integral to how the intervention itself is defined, so the control group's business-as-usual condition is the appropriate comparator under the ERCT resource-as-treatment exception.

• Level 3 Criteria

  • R

    Reproduced

    • No research team independent of the original intervention developers (Hulme, Snowling and colleagues) has published a peer-reviewed replication of this trial; all identified follow-on and scale-up studies share authors with the original development team.
    • "A trial of this intervention found statistically significant and positive impacts on oral language and spoken narrative skills (Fricke et al., 2013)." (p. 8)
    • Relevant Quotes: 1) "A trial of this intervention found statistically significant and positive impacts on oral language and spoken narrative skills (Fricke et al., 2013)." (p. 8) -- this earlier pilot was run by the same core research team (Bowyer-Crane, Fricke, Hulme, Snowling) that designed the intervention, so it is not an independent replication. 2) Internet search of Crossref and Europe PMC identified a series of subsequent NELI trials, all co-authored by Charles Hulme and/or Margaret Snowling -- the same academics credited on p. 10 of this report as the intervention's "Project and delivery team" -- typically with Gillian West and colleagues affiliated with the University of Oxford and OxEd and Assessment Ltd (the organisation that now licenses NELI): West, G., Snowling, M.J., Lervag, A., Buchanan-Worster, E., Duta, M., Hall, A., McLachlan, H. and Hulme, C. (2021) "Early language screening and intervention can be delivered successfully at scale: evidence from a cluster randomized controlled trial", Journal of Child Psychology and Psychiatry, DOI: 10.1111/jcpp.13415 -- a 193-school, ~1,173-pupil scale-up cluster RCT (effect sizes d=0.26-0.32); West, G. et al. (2022) "Early language intervention improves behavioral adjustment in school", Journal of School Psychology, DOI: 10.1016/ j.jsp.2022.04.006; West, G. et al. (2024) "Oral language enrichment in preschool improves children's language skills", Journal of Child Psychology and Psychiatry, DOI: 10.1111/ jcpp.13947; Hulme, C., West, G., Rios Diaz, M., Hearne, S., Korell, C., Duta, M. and Snowling, M.J. (2025) "The Nuffield Early Language Intervention (NELI) programme is associated with lasting improvements in children's language and reading skills", Journal of Child Psychology and Psychiatry, 66(9), 1357-1365, DOI: 10.1111/ jcpp.14157 -- a 2-year follow-up reporting durable effects (d=0.16-0.33); and West, G., Titiunik, R., Hearne, S., Korell, C., Duta, M. and Hulme, C. (2026) "Real-World Effectiveness of Early Language Intervention: Evidence From a Nationwide Rollout of the Nuffield Early Language Intervention in England", American Journal of Speech-Language Pathology, DOI: 10.1044/2025_AJSLP-25-00205 -- a nationwide regression-discontinuity analysis of 19,936 children (Hedges' g=0.40). 3) No paper by a research team independent of Hulme, Snowling and their immediate collaborators, replicating NELI's effects in a peer-reviewed outlet, was found despite repeated targeted searches of Crossref and Europe PMC (including a search explicitly excluding Hulme as an author). Detailed Analysis: Criterion R requires independent replication by a different research team, in a different context, published in a peer-reviewed outlet. Although NELI has since been tested at very large scale and shown to produce durable effects at a 2-year follow-up, every one of these subsequent trials is co-authored by Charles Hulme and/or Margaret Snowling, the same academics named as the programme's original developers on p. 10 of this report, generally together with Gillian West and colleagues at OxEd and Assessment Ltd, the organisation that commercially licenses NELI. This is a continuation of the developers' own research and commercial programme rather than an independent replication by an unconnected team, so it does not satisfy the "different research team" requirement. No qualifying independent replication was located. Criterion R is not met because, despite an extensive published record of subsequent NELI trials, none of them was conducted by a research team independent of the original intervention developers.
  • A

    All-subject Exams

    • The study measured only oral language and word-level literacy skills; no other core subjects (e.g. numeracy) were assessed, and no exception for specialised upper-secondary/vocational education applies at this early-years level.
    • "The primary outcome for this evaluation is a language skills score... In addition, we define a secondary composite outcome of word-level literacy skills." (p. 13)
    • Relevant Quotes: 1) "The primary outcome for this evaluation is a language skills score which is a composite score of four different externally-valid measures of language skill." (p. 13) 2) "In addition, we define a secondary composite outcome of word-level literacy skills based on the following three measures: YARC: Letter Knowledge... YARC: Early Word Reading... Spelling." (p. 13) 3) "This evaluation did not provide reliable evidence that either version of the programme had a positive impact on children's word-level literacy skills." (p. 4) Detailed Analysis: Only language/oral communication skills and word-level literacy were assessed; no numeracy, science, or other subject outcomes were measured at all. The ERCT exception for narrow subject coverage applies only to highly specialised interventions in upper secondary or vocational education, which does not describe this nursery/reception-age oral language programme. Since criterion E is met but coverage is limited to a single (language/literacy) domain without a qualifying exception, the all-subject requirement is not satisfied. Final sentence: Criterion A is not met because only language and literacy outcomes were assessed, with no other core subjects measured and no qualifying exception for the age group and intervention type.
  • G

    Graduation Tracking

    • Follow-up testing in this study stopped six months after the intervention ended; no publication was found tracking this specific cohort through to graduation, and the only related long-term NELI evidence found concerns a different, later effectiveness-trial cohort followed for only about two years post-intervention, well short of graduation.
    • "Follow-up test: these tests were undertaken about six months after the end of the intervention phase between October and December 2014." (p. 13)
    • Relevant Quotes (original report): 1) "Follow-up test: these tests were undertaken about six months after the end of the intervention phase between October and December 2014." (p. 13) 2) "By this stage some control pupils had already begun to receive the alternative intervention (RALI). This is not ideal if we are interested in the extent to which differences across treatment and control pupils persist after the end of the intervention." (p. 13) 3) "It will be useful to follow these pupils over time to examine any lasting impacts at Key Stage 2 (and hopefully more schools will provide the necessary data to enable such analysis)." (p. 45) Findings from internet research: An internet search for follow-up publications tracking the 350 pupils (34 schools, Yorkshire and the South East, screened 2013, tested 2013-2014) evaluated in this specific report did not locate any paper that continues to track this cohort, whether via the National Pupil Database or otherwise, through to the end of primary school. The search identified further NELI research co-authored by two of this study's programme designers (Hulme, Snowling), but this concerns a different, later, much larger "effectiveness trial" cohort (West, Snowling, Lervag, Buchanan-Worster, Duta, Hall, McLachlan, & Hulme, 2021, 193 schools, 5,879 children): Hulme, C., West, G., Rios Diaz, M., Hearne, S., Korell, C., Duta, M., & Snowling, M.J. (2025). "The Nuffield Early Language Intervention (NELI) programme is associated with lasting improvements in children's language and reading skills." Journal of Child Psychology and Psychiatry, 66(9), 1357-1365. This paper reports that "approximately 2 years" after completion of that separate, effectiveness-trial intervention, children receiving NELI still showed better oral language (d = 0.22-0.33), reading comprehension (d = 0.16-0.24) and word reading (d = 0.16-0.22) than controls, concluding that "a widely used language intervention produces durable improvements in language and reading skills." Detailed Analysis: Criterion G requires that the participants of this specific study be tracked until graduation, or that such tracking be reported in subsequent papers by the same authors covering the same cohort. No such publication was found for the 350 pupils evaluated in this 2016 IFS/NatCen report; the report itself only records a six-month follow-up and states an aspiration (not a completed follow-up) to link pupils to the National Pupil Database for future tracking. The only related long-term evidence located, Hulme et al. (2025), concerns a different and later effectiveness-trial cohort, followed for only about two years post intervention (broadly Key Stage 1, not the end of primary schooling), which would in any case fall well short of "graduation" even if it were the same cohort. Final sentence: Criterion G is not met because no evidence was found of this study's own cohort being tracked to graduation, and the only related long-term NELI follow-up identified is of a different cohort followed for a much shorter period than to graduation.
  • P

    Pre-Registered

    • The report explicitly states the trial was not registered by either the evaluation or project delivery teams, and an internet search found no trial-registry entry for this 2013-2014 study; the only registered NELI trials located are later, separate effectiveness trials.
    • "The trial was not registered by the evaluation or project delivery teams." (p. 10)
    • Relevant Quotes: 1) "Trial registration: The trial was not registered by the evaluation or project delivery teams." (p. 10) 2) "The evaluation protocol was published on the EEF website: https://educationendowmentfoundation. org.uk/projects/language-for-learning/" (p. 9) Findings from internet research: A search of trial registries did not find any pre-registration of this specific 2013-2014 efficacy trial. The only ISRCTN registrations located for the Nuffield Early Language Intervention are for later, separate trials: ISRCTN12991126 (2018, "The effectiveness of the Nuffield Early Language Intervention programme: a randomised controlled trial"), ISRCTN14881793 (2020, "Evaluating the effectiveness of the Nuffield Early Language Intervention"), and ISRCTN17801846 (2024, an effectiveness trial for preschool). None of these correspond to, or predate, this study, and none was found dated before this trial's 2013 start. Detailed Analysis: The report contains an explicit, unambiguous statement that the trial was not registered on any trial registry by either the delivery or evaluation teams. While an evaluation protocol was published on the EEF's own website, this is not equivalent to formal pre-registration on a recognised registry with a verifiable timestamp before data collection, and no such registration is claimed. Internet research confirms no registry record for this specific trial exists; only unrelated, later NELI trials are registered. Final sentence: Criterion P is not met because the authors explicitly confirm the trial was not registered, and no internet-search evidence of registration for this trial was found.