The efficacy of grapheme-phoneme correspondence instruction in reducing the effect of orthographic forms on second language phonology

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Whole classes (eight classes in four matched pairs) were randomly allocated to conditions by an independent researcher, satisfying class-level randomisation.
    • "Then, one class within each pair was randomly allocated to the EGPC condition and one to the PE condition. To ensure concealed allocation, randomization was carried out by an independent researcher with no knowledge of the project." (p. 690)
    • Relevant Quotes: 1) "Using a group randomized trial design, we tested 100 learners of English as a Second Language studying in eight classes organized in four pairs (please see Randomization section below). One of the two classes in each pair was randomly selected to receive the explicit GPC instruction (EGPC condition), and the other the passive exposure intervention (PE condition)." (p. 689) 2) "First, the eight classes of students were paired: there were two pairs where both classes in a pair had the same teacher (and same school) and two pairs where both classes in a pair were in the same school but had different teachers. Then, one class within each pair was randomly allocated to the EGPC condition and one to the PE condition. To ensure concealed allocation, randomization was carried out by an independent researcher with no knowledge of the project." (p. 690) 3) "A group randomized trial (GRT) was conducted with eight classes of students." (p. 686) 4) "group randomized trials, where classes or schools are randomly allocated to conditions, are widely employed in situations like the one in the present study, where randomization of individual participants to conditions was not logistically possible" (p. 689) Detailed Analysis: Criterion C requires randomisation at the class level (or stronger) with a clearly described randomisation process. The paper explicitly describes a group (cluster) randomised trial in which the unit of randomisation was the intact class: eight fourth-year high-school classes were arranged into four matched pairs (matched on school and, where possible, teacher), and one class in each pair was randomly assigned to the explicit GPC condition and the other to the passive exposure control. Allocation was concealed and performed by an independent researcher with no knowledge of the project. This design prevents within-class contamination between treatment and control students, which is exactly the concern the criterion addresses. The intervention was delivered to whole classes during normal English lessons, consistent with class-level implementation. Criterion C is met because entire classes, not individual students within a class, were randomly allocated to conditions with a clearly described and concealed randomisation procedure.
  • E

    Exam-based Assessment

    • Outcomes were measured with researcher-developed experimental tasks (delayed word repetition and rhyme judgment), not with any widely recognised standardised exam.
    • "each participant was assessed in a production task (delayed word repetition), then a phonological awareness task (rhyme judgment)... All the tasks were developed by Bassetti et al. (2020) and are described in detail in that study." (p. 691)
    • Relevant Quotes: 1) "On both occasions, each participant was assessed in a production task (delayed word repetition), then a phonological awareness task (rhyme judgment)." (p. 691) 2) "All the tasks were developed by Bassetti et al. (2020) and are described in detail in that study. Task materials are available in two online repositories" (p. 691) 3) "The production task involved delayed word repetition and tested whether participants produced a sound as longer when spelled with a digraph than when spelled with a single letter." (pp. 691-692) 4) "The rhyme judgment task involved asking participants to decide whether two words contained the same sounds. The targets were 12 C-CC word pairs, such as very-cherry, and 12 V-VV word pairs, such as rule-cool." (p. 692) 5) "The study also showed that delayed word repetition and rhyme judgment tasks, developed by Bassetti (2017) and Bassetti et al. (2020), can reliably measure effects of number of letters on both L2 production and phonological awareness." (p. 701) Detailed Analysis: Criterion E requires the use of standardised, widely recognised exam-based assessments rather than instruments created by the researchers for the study. The outcome measures here were a delayed word repetition task (acoustic duration ratios measured with Praat) and a rhyme judgment task, both explicitly described as tasks developed by the same research team in prior work (Bassetti 2017; Bassetti et al. 2020). These are bespoke experimental instruments aligned with the intervention's target phenomenon (sound-length contrasts), not national or state-wide standardised exams. No standardised English proficiency examination was used as an outcome measure. Criterion E is not met because outcomes were assessed with custom researcher-designed tasks rather than a recognised standardised exam.
  • T

    Term Duration

    • The intervention was a single one-hour session in mid-April with the posttest in May-June, an interval of only about one to two months, which is shorter than a full academic term.
    • "the eight entire classes received a one-hour teaching session during a normal English language session in their classroom in mid-April. The posttest assessment took place between May and June." (p. 692)
    • Relevant Quotes: 1) "Preintervention assessments were conducted between November and March. Then, after being allocated to the EPGC or PE groups (see Randomization, above), the eight entire classes received a one-hour teaching session during a normal English language session in their classroom in mid-April. The posttest assessment took place between May and June." (p. 692) 2) "The sessions for each of the eight classes lasted one hour, and the intervention focused on clarifying just one concept" (p. 688) 3) "Since the posttest revealed no effects of the intervention, and because teachers were unwilling to give up one hour of teaching at the end of the year, we did not administer a planned delayed posttest" (p. 692) 4) "Arguably the main limitation of the study was the limited amount of time dedicated to the intervention. Originally, two one-hour sessions had been planned" (p. 701) Detailed Analysis: Criterion T requires that outcomes be measured at least one full academic term (approximately 3-4 months) after the intervention begins. Here the intervention consisted of a single one-hour teaching session delivered in mid-April, and the primary outcomes were collected at posttest between May and June, roughly one to two months after the intervention started. Although the overall project spanned several months (preintervention lesson at the start of the school year, pretests November-March), the follow-up interval from intervention start to outcome measurement is what the criterion measures, and that interval is clearly shorter than a full term. A planned delayed posttest that could have extended tracking was cancelled. Criterion T is not met because outcomes were measured only about one to two months after the one-hour intervention began, less than a full academic term.
  • D

    Documented Control Group

    • The passive exposure control group is documented in detail, including its size, demographics, baseline matching, and the exact alternative instruction it received.
    • "The students who received the EGPC and PE interventions were found to be well matched on the variables, and independent t-tests confirmed that there were no significant differences." (pp. 689-690)
    • Relevant Quotes: 1) "The final sample of students whose data were included in the analyses comprised 43 participants in the EGPC condition (one of them did not complete the posttest production task) and 45 in the PE condition. Table 1 gives a summary of demographic variables for the participants in the two conditions." (p. 689) 2) "Table 1. Demographic variables for the participants in the explicit GPC (EPGC) and passive exposure (PE) conditions ... N 43 45; Age (years) 16.84 (±0.37) 16.98 (±0.50); English language age of onset of acquisition (years) 5.74 (±1.95) 5.78 (±1.77); English language length of study (years) 10.35 (±1.89) 10.69 (±2.32)" (p. 690) 3) "The students who received the EGPC and PE interventions were found to be well matched on the variables, and independent t-tests confirmed that there were no significant differences." (pp. 689-690) 4) "the PE control group learnt about English word formation ... Students practiced the same words using the same produce-listen-repeat tasks as the EPGC group, but with no mention of spelling" (pp. 692-693) 5) "Figure 1 gives an outline of the number of students involved in each stage of the study from initial recruitment to analysis of the results, following CONSORT recommendations." (p. 689) Detailed Analysis: Criterion D requires detailed documentation of the control group's characteristics, size, and conditions. The paper reports the control (passive exposure) group's size (n = 45), demographic characteristics (age, age of onset of English acquisition, years of English study) in Table 1, confirms baseline equivalence with the intervention group via t-tests and prior assessments from Bassetti et al. (2020) (memory, proficiency, attitudes, etc.), provides a CONSORT flow chart of participants through the study, and describes precisely what the control group received (a one-hour word-formation lesson practicing the same words). This is thorough documentation permitting proper comparison. Criterion D is met because the control group's size, demographics, baseline equivalence, and treatment are clearly and thoroughly documented.

• Level 2 Criteria

  • S

    School-level RCT

    • Randomisation was carried out at the class level within schools (classes paired within the same school), not by randomly assigning whole schools to conditions.
    • "First, the eight classes of students were paired: there were two pairs where both classes in a pair had the same teacher (and same school) and two pairs where both classes in a pair were in the same school but had different teachers. Then, one class within each pair was randomly allocated to the EGPC condition and one to the PE condition." (p. 690)
    • Relevant Quotes: 1) "They were recruited from fourth-year classes of three high schools in Rome, Italy. Six English language teachers at the schools, teaching a total of eight classes, consented for their students to take part in the study." (p. 689) 2) "First, the eight classes of students were paired: there were two pairs where both classes in a pair had the same teacher (and same school) and two pairs where both classes in a pair were in the same school but had different teachers. Then, one class within each pair was randomly allocated to the EGPC condition and one to the PE condition." (p. 690) Detailed Analysis: Criterion S requires randomisation at the level of whole schools (or equivalent implementing institutions). In this study only three schools participated, and both conditions were present within each school: classes were paired within the same school and one class per pair was randomised to each condition. Therefore schools were not the unit of randomisation; classes within schools were. This is explicitly a class-level (group randomised) design, which does not satisfy the school-level requirement. Criterion S is not met because randomisation occurred between classes within the same schools, not between schools.
  • I

    Independent Conduct

    • The authors designed the intervention, delivered it (Bassetti herself taught all sessions), administered assessments, and analysed the data, with no independent third-party evaluation team despite internal blinding.
    • "In order to ensure quality and fidelity, the intervention (explicit GPC instruction) and control group (passive exposure) teaching were delivered by an applied linguist with fifteen years' experience of teaching linguistics (Bassetti)." (p. 688)
    • Relevant Quotes: 1) "In order to ensure quality and fidelity, the intervention (explicit GPC instruction) and control group (passive exposure) teaching were delivered by an applied linguist with fifteen years' experience of teaching linguistics (Bassetti)." (p. 688) 2) "Apart from the Bassetti, who delivered the intervention, all the researchers were blind to participants' group allocation: the Cerni who administered the pre- and postintervention assessments and coded responses in the awareness task, and the phoneticians who performed acoustic analyses." (p. 691) 3) "The authors are grateful to ... David Torgerson for advice on intervention set-up and data analysis" (p. 702) 4) "This work was supported by a Leverhulme Trust Research Grant ... awarded to Bassetti and Masterson. The sponsor had no involvement in the conduct of the research or the preparation of the article." (p. 702) Detailed Analysis: Criterion I requires that the study be conducted independently from the designers of the intervention, for example by an external evaluation team. Here the intervention was designed by the authors and was personally delivered to all eight classes by the first author (Bassetti). The pre- and posttest assessments were administered and coded by a co-author (Cerni), and the statistical analyses were carried out by the author team. The paper does document meaningful internal safeguards - concealed allocation by an independent researcher, blinding of Cerni and of the phoneticians who did the acoustic analyses, and blinding of teachers and students - and external advice from David Torgerson on set-up and analysis. However, these are internal blinding measures and informal advice, not an independent third-party conduct of data collection and analysis; the same team that designed the intervention implemented and evaluated it. Criterion I is not met because the intervention designers themselves delivered the intervention, collected the data, and performed the analyses, without an independent external evaluation team.
  • Y

    Year Duration

    • Outcomes were measured only about one to two months after the one-hour mid-April intervention began, far short of 75% of an academic year, and criterion T is already not met.
    • "the eight entire classes received a one-hour teaching session during a normal English language session in their classroom in mid-April. The posttest assessment took place between May and June." (p. 692)
    • Relevant Quotes: 1) "the eight entire classes received a one-hour teaching session during a normal English language session in their classroom in mid-April. The posttest assessment took place between May and June." (p. 692) 2) "Since the posttest revealed no effects of the intervention, and because teachers were unwilling to give up one hour of teaching at the end of the year, we did not administer a planned delayed posttest" (p. 692) 3) "The sessions for each of the eight classes lasted one hour" (p. 688) Detailed Analysis: Criterion Y requires that outcomes be measured at least 75% of an academic year (roughly 9-10 months) after the intervention begins. The intervention began in mid-April and outcomes were collected between May and June of the same school year, an interval of approximately one to two months. A planned delayed posttest was cancelled, so no longer-term measurement exists. Furthermore, per the ranking rules, since criterion T (Term Duration) is not met, criterion Y cannot be met. Criterion Y is not met because the interval from intervention start to outcome measurement was only about one to two months, far below 75% of an academic year.
  • B

    Balanced Control Group

    • The passive exposure control received a matched one-hour session from the same teacher with the same structure, the same words, and the same amount of spoken and written input and practice, differing only in the GPC content being tested.
    • "Both EPGC and PE groups received a one-hour session, delivered by the same teacher, with similar structure and tasks, and crucially practiced the same English words, with the same amount of spoken and written input and practice." (p. 692)
    • Relevant Quotes: 1) "Both EPGC and PE groups received a one-hour session, delivered by the same teacher, with similar structure and tasks, and crucially practiced the same English words, with the same amount of spoken and written input and practice. The difference was that the experimental EGPC group learnt about the GPCs of English consonant and vowel digraphs; the PE control group learnt about English word formation." (p. 692) 2) "The control condition was included for three reasons: (1) to prevent participants from guessing the purposes of the intervention, (2) to avoid placebo or demand effects, and (3) to provide all participants with the same amount of exposure to the target English words." (pp. 692-693) 3) "Students practiced the same words using the same produce-listen-repeat tasks as the EPGC group, but with no mention of spelling, as students worked on English derivation and word-formation" (p. 693) 4) "it was strengthened by having a group that was exposed to the same words for the same amount of time, but without the explicit GPC teaching" (p. 701) 5) "At the beginning of the school year, all participating classes read and listened to specially prepared texts containing one occurrence of each of the target words ... during a normal teaching session delivered by their English language teacher." (p. 692) Detailed Analysis: Criterion B requires that the control group receive balanced time and resources so that the intervention's specific effect is isolated. Re-applying the current decision tree: extra resources are present (a one-hour guest session outside the normal curriculum, EXTRA_RESOURCES_PRESENT = true); this extra session is not itself the treatment variable being tested (RESOURCES_ARE_TREATMENT = false - the treatment is the explicit GPC content, not the extra time as such); however, the control condition received an equivalent active-control session of identical duration (one hour), same instructor (Bassetti), same structure (reflection, explicit teaching, practice), same target words, and the same amount of spoken and written input and practice, differing only in instructional content (GPC rules versus word formation), so CONTROL_MATCHES_RESOURCES = true. Per the decision tree, when control matches the resources, the criterion is met regardless of the treatment-variable question. Both groups also received the identical preintervention vocabulary lesson from their own teachers, further supporting balance. Criterion B is met because the active control was carefully matched to the intervention in time, teacher, materials, and amount of practice, isolating the explicit GPC instruction as the only difference, and this matching satisfies the current ERCT decision tree for criterion B.

• Level 3 Criteria

  • R

    Reproduced

    • The authors state this is the first study of its kind, no replication is mentioned in the paper, and an internet citation-database search of works citing this paper found no independent published replication.
    • "This study is the first, to the best of our knowledge, to have included an intervention aimed at reducing orthographic effects on L2 phonology." (p. 701)
    • Relevant Quotes: 1) "This study is the first, to the best of our knowledge, to have included an intervention aimed at reducing orthographic effects on L2 phonology." (p. 701) 2) "It is, however, too early to rule out the possibility that interventions impact orthographic effects on L2 phonology, due to a number of limitations of this study." (p. 700) 3) "Future research could then investigate effects of a longer intervention, possibly using the same instructional materials prepared in this study." (p. 701) Detailed Analysis: Criterion R requires that the study be independently replicated by a different research team in a different context, with the replication published in a peer-reviewed journal. The authors explicitly describe this as the first intervention study targeting orthographic effects on L2 phonology, and the paper contains no reference to any replication. Related prior studies cited (e.g., Showalter & Hayes-Harb 2015; Hayes-Harb et al. 2018; Showalter 2020) concern novel-language word learning, not replications of this class-randomised GPC teaching trial with Italian high-school learners of English. Internet-backed verification: this paper's citation record (DOI 10.1017/S014271642200008X, OpenAlex work ID W4220929188, 11 citing works as of this check) was retrieved via the OpenAlex and Semantic Scholar citation APIs. The citing works are: a systematic review of phonological awareness and reading (Valle-Zevallos et al., 2024); a book chapter by Bassetti herself ("Orthographic Effects in the Phonetics and Phonology of Second Language Learners and Users," 2024, Cambridge University Press); a study of phonological awareness instruction and EFL learners' pronunciation/motivation (Wardana et al., 2022); an Italian educational-linguistics bibliography update (Balboni, 2023); a study of nasal-vowel pronunciation by L1 Japanese learners of L3 French (Granget et al., 2024); a critical systematic review of L2 pronunciation instruction (Yagiz et al., 2024); and a study of colour cues for pronunciation learning (Caudrelier et al., 2025). None of these is an independent replication of this specific group-randomised GPC teaching trial with Italian high-school EnglishL2 learners - they are reviews, a theoretical chapter by the original author, or studies of different interventions and populations. Criterion R is not met because no independent peer-reviewed replication of this study exists; the authors themselves present it as the first of its kind, and a citation-database search confirms no subsequent replication has been published.
  • A

    All-subject Exams

    • Only L2 English phonology was measured with custom tasks; no standardised exams across all main school subjects were used, and criterion E is already not met.
    • "each participant was assessed in a production task (delayed word repetition), then a phonological awareness task (rhyme judgment)." (p. 691)
    • Relevant Quotes: 1) "On both occasions, each participant was assessed in a production task (delayed word repetition), then a phonological awareness task (rhyme judgment)." (p. 691) 2) "The aims of the study were to investigate whether instruction could reduce the effects of orthographic forms on L2 speech production and L2 speech sound awareness." (p. 686) Detailed Analysis: Criterion A requires standardised exam-based assessment of all main subjects taught at the educational level, and it cannot be met if criterion E is not met. Here criterion E is not met (custom experimental tasks were used), and the outcomes concern only one narrow aspect of one subject (L2 English pronunciation and phonological awareness). No other school subjects (Italian, mathematics, science, etc.) were assessed, and no specialised-education exception applies to this general high-school population. Criterion A is not met because assessment was limited to custom L2 phonology tasks with no standardised exams covering all main subjects.
  • G

    Graduation Tracking

    • Tracking ended with the May-June posttest; even the planned delayed posttest was cancelled, and an internet search for follow-up publications tracking this cohort found none, while prerequisite criterion Y is also not met.
    • "Since the posttest revealed no effects of the intervention, and because teachers were unwilling to give up one hour of teaching at the end of the year, we did not administer a planned delayed posttest" (p. 692)
    • Relevant Quotes: 1) "Since the posttest revealed no effects of the intervention, and because teachers were unwilling to give up one hour of teaching at the end of the year, we did not administer a planned delayed posttest and did not deliver the EPGC intervention to the PE group, which we had planned for ethical reasons" (p. 692) 2) "Fourth-year classes were targeted to ensure a minimum age of 16 years ... (fifth-year students need to prepare for graduation exams)." (p. 689) 3) "The study ended with a debrief session held by the Bassetti for all teachers and students at each school outside of teaching hours" (p. 692) Detailed Analysis: Criterion G requires participants to be tracked until graduation from their educational stage. The participants were fourth-year high-school students, at least one year from graduation, and measurement ended with the posttest in May-June of the same school year in which the intervention was delivered. The planned delayed posttest was explicitly cancelled, and the study concluded with a debrief. Per the ranking rules, criterion G also cannot be met because criterion Y is not met. Internet-backed verification: the same OpenAlex/Semantic Scholar citation search used for criterion R (11 works citing this paper, DOI 10.1017/S014271642200008X) was reviewed for any follow-up publication by Bassetti, Cerni, or Masterson tracking this same cohort of Italian high-school students toward graduation. No such follow-up was found; the sole subsequent work by Bassetti in the citing-works list is a 2024 theoretical book chapter on orthographic effects on L2 phonology (likely the published version of "Bassetti, B. (2022). Effects of orthography on second language phonology," cited in the reference list as a manuscript in preparation), which is a review/theory chapter, not a cohort follow-up study, and contains no graduation-tracking data for these participants. Criterion G is not met because tracking stopped shortly after the intervention, no graduation follow-up was conducted or published, and a targeted search found no such follow-up publication.
  • P

    Pre-Registered

    • The paper contains no mention of pre-registration of the trial protocol on any registry, and an internet check confirmed the OSF links in the paper are unregistered materials repositories, not a pre-registration.
    • Relevant Quotes: 1) "The study was approved by the University of Warwick Research Ethics Committee (Number: 118/14-15, Project Title: Effects of Orthography on Phonology in Second Language Speakers of English: Pronunciation, Phonological Awareness, Speech Perception and Spelling)." (p. 691) 2) "Task materials are available in two online repositories (https://osf.io/p3q6d and www.iris-database.org)." (p. 691) 3) "Preintervention lesson materials and intervention materials are available at https://osf.io/eubpz and www.iris-database.org." (p. 693) Detailed Analysis: Criterion P requires pre-registration of the full study protocol (hypotheses, methods, planned analyses) on a registry before data collection began. The paper reports ethics approval and shares task and intervention materials on OSF and IRIS, but sharing materials is not protocol pre-registration: no trial registry (e.g., ClinicalTrials.gov, ISRCTN, OSF Registrations) is cited, no registration ID or date is given, and no statement of pre-registration appears anywhere in the paper. Internet-backed verification: the OSF nodes referenced in the paper were checked directly via the OSF API. The node at https://osf.io/eubpz is titled "Bassetti, B., Cerni, T., & Masterson, J. (2021) Materials for 'Efficacy of a teaching intervention...'", is categorized as an ordinary "Project" (not a Registration), was created 2021-01-18, and its OSF metadata explicitly returns "registration": false, confirming it is a materials-sharing repository, not a pre-registered protocol. No trial registry entry for this study (e.g. on ClinicalTrials.gov, ISRCTN, or OSF Registries) was located. Notably, several planned elements (a second teaching session, a delayed posttest) were changed or dropped, with no pre-registered protocol against which such deviations could be checked. Criterion P is not met because no pre-registration of the study protocol is mentioned in the paper, and a direct check of the linked OSF repositories confirms they are unregistered materials projects rather than a pre-registration.