The effects of Processing Instruction on the acquisition of English simple past tense: Age and cognitive task demands

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Participants were pre-existing intact groups (a school class and an adult language-centre class) and no random assignment to treatment versus control is described anywhere in the paper; both groups also received the identical Processing Instruction treatment.
    • "The participants belonged to two different age groups: school-age learners (fifth-grade pupils – mean age 10.5 years old) and adults (mean age 26 years old) enrolled in two different institutions in Germany..." (p. 253)
    • Relevant Quotes: 1) "The participants belonged to two different age groups: school-age learners (fifth-grade pupils – mean age 10.5 years old) and adults (mean age 26 years old) enrolled in two different institutions in Germany: Maria Ward Gymnasium Nymphenburg (German grammar school) and the Language Centre of University of Munich (LMU), Germany." (p. 253) 2) "A pre-test/post-test design was used (see Figure 1)." (p. 254) 3) "During the instruction period the groups were taught by the same instructor (the researcher)." (p. 254) Detailed Analysis: Criterion C requires a clearly described and properly implemented randomisation process at the class level (or stronger). Nowhere in the Methodology, Procedures, or Participants sections does the paper describe randomised allocation of participants to conditions. The "two groups" are naturally occurring intact cohorts distinguished by age and institution (a secondary-school class vs. an adult language-centre cohort), not randomly assigned experimental arms. Crucially, both groups received the same Processing Instruction treatment - there is no randomised treatment-vs-control comparison at all, only a comparison between two pre-existing age groups exposed to the same intervention. The tutoring exception does not apply, as this is classroom-based group instruction, not one-to-one tutoring. Criterion C is not met because there is no description of randomisation to treatment and control conditions at any level; the design is a quasi-experimental intact-group comparison, not an RCT.
  • E

    Exam-based Assessment

    • Both interpretation tasks and the written production task were researcher-developed or adapted instruments piloted specifically for this line of Processing Instruction research, not widely recognised standardised exams.
    • "One interpretation task (see sample in Figure 5) was adopted from a previous study (Benati 2005) and a second interpretation task (see sample in Figure 6) was developed especially for this experiment to account for the factor 'cognitive task demands'." (p. 259)
    • Relevant Quotes: 1) "One interpretation task (see sample in Figure 5) was adopted from a previous study (Benati 2005) and a second interpretation task (see sample in Figure 6) was developed especially for this experiment to account for the factor 'cognitive task demands'." (p. 259) 2) "The written production task (see sample in Figure 7) was developed and used to measure learner's ability to produce correct sentences using the target feature." (p. 261) 3) "All tests were balanced in terms of difficulty and vocabulary as they were all previously piloted." (p. 259) Detailed Analysis: Criterion E requires a standardised, widely-recognised exam rather than a custom-built instrument. All three outcome measures here (two sentence-level interpretation tasks and a cloze-style written production task) were designed by the researchers specifically for this and prior Processing Instruction studies, then piloted internally for difficulty/vocabulary balance. None of them are national, state, or internationally recognised standardised assessments; they are bespoke experimental instruments targeting a single grammatical feature (-ed past tense). Criterion E is not met because the assessment tools are custom-made research instruments, not standardised exams.
  • T

    Term Duration

    • The tracked interval runs from one week before a two-day instructional treatment to a delayed post-test two weeks after instruction - only a few weeks total, far short of a full academic term.
    • "The teaching schedule in the two institutions allowed for the treatments to take place over two consecutive days for a total of two hours of instruction for each group." (p. 254)
    • Relevant Quotes: 1) "Pre-tests were administered one week before the beginning of the instructional period." (p. 254) 2) "The teaching schedule in the two institutions allowed for the treatments to take place over two consecutive days for a total of two hours of instruction for each group." (p. 254) 3) "2 HOURS (TWO CONSECUTIVE DAYS- 4 HOURS IN TOTAL)" (Fig. 1, p. 255) 4) "Post-tests were used immediately after instruction and two weeks after the instructional period." (p. 254) Detailed Analysis: Criterion T requires outcomes to be measured at least one full academic term (roughly 3-4 months) after intervention start. Here the instructional treatment itself lasted only two consecutive days (two to four hours of instruction total), and the final ("delayed") measurement occurred just two weeks after instruction ended. Counting from the pre-test (one week before instruction) to the delayed post-test, the entire study spans roughly three weeks, nowhere near the one-term threshold. Criterion T is not met because the total tracked interval from intervention start to final measurement is only a few weeks, far short of a full academic term.
  • D

    Documented Control Group

    • The study has no untreated or business-as-usual control group at all; both the school-age and adult cohorts received the Processing Instruction treatment, so there is no control condition to document.
    • "Two groups were formed: school-age (n = 36); and young adults (n = 13)." (p. 254)
    • Relevant Quotes: 1) "Two groups were formed: school-age (n = 36); and young adults (n = 13)." (p. 254) 2) "A pre-test/post-test design was used (see Figure 1)." (p. 254) 3) "During the instruction period the groups were taught by the same instructor (the researcher)." (p. 254) Detailed Analysis: Criterion D requires a well-documented control group that did not receive the intervention, including its demographics, baseline performance, and conditions. In this study, the only two groups defined are the school-age and adult Processing Instruction groups; both received the identical PI treatment described in Section 2.4. No group in this design serves as an untreated or alternative-instruction control against which the intervention's effect could be isolated - the comparison throughout the paper is between two treated age groups, not between treatment and control. Criterion D is not met because the study contains no documented (or any) control group that did not receive the Processing Instruction intervention.

• Level 2 Criteria

  • S

    School-level RCT

    • There is no school-level (or any level) randomisation described; the two groups are naturally occurring intact cohorts from two different institutions, both of which received the same treatment.
    • "...enrolled in two different institutions in Germany: Maria Ward Gymnasium Nymphenburg (German grammar school) and the Language Centre of University of Munich (LMU), Germany." (p. 253)
    • Relevant Quotes: 1) "The participants belonged to two different age groups: school-age learners (fifth-grade pupils – mean age 10.5 years old) and adults (mean age 26 years old) enrolled in two different institutions in Germany: Maria Ward Gymnasium Nymphenburg (German grammar school) and the Language Centre of University of Munich (LMU), Germany." (p. 253) 2) "A pre-test/post-test design was used (see Figure 1)." (p. 254) Detailed Analysis: Criterion S requires randomisation of entire schools (or equivalent institutional units) to intervention versus control conditions. This study does not randomise institutions to conditions at all; the school and the language centre are simply the pre-existing sites where the two age cohorts happen to be enrolled, and both sites/cohorts receive the same Processing Instruction treatment. There is no school-versus-school (or institution-versus- institution) comparison of treatment vs. control. Criterion S is not met because no school-level randomisation to intervention/control conditions is described.
  • I

    Independent Conduct

    • The same person who was the researcher also delivered the instructional treatment as instructor, with no independent or third-party party involved in implementation.
    • "During the instruction period the groups were taught by the same instructor (the researcher)." (p. 254)
    • Relevant Quotes: 1) "During the instruction period the groups were taught by the same instructor (the researcher)." (p. 254) 2) "The instructor acted as a facilitator during the experiment." (p. 254) Detailed Analysis: Criterion I requires that the study be conducted independently from those who designed the intervention, e.g. via an external evaluation team unconnected to the developers. Here, the paper explicitly states that the researcher (i.e., one of the study's authors/designers) personally delivered the instructional treatment to both groups. There is no mention of external test administrators, independent data collectors, or any third-party oversight of implementation or analysis. Criterion I is not met because the researcher who designed the study also personally delivered the intervention, with no independent conduct described.
  • Y

    Year Duration

    • Because criterion T (Term Duration) is not met, criterion Y is automatically not met; the total tracked period of about three weeks is also far short of 75% of an academic year.
    • "Post-tests were used immediately after instruction and two weeks after the instructional period." (p. 254)
    • Relevant Quotes: 1) "Pre-tests were administered one week before the beginning of the instructional period." (p. 254) 2) "Post-tests were used immediately after instruction and two weeks after the instructional period." (p. 254) Detailed Analysis: Per the criteria-specific instruction, if Criterion T is not met then Criterion Y cannot be met either. Independently, the tracked interval here (about three weeks from pre-test to delayed post-test) is vastly shorter than 75% of an academic year (~9-10 months), so the substantive requirement also clearly fails. Criterion Y is not met both because Criterion T is not met and because the actual tracked duration is only a few weeks.
  • B

    Balanced Control Group

    • There is no control or business-as-usual condition anywhere in the design to compare resource/time allocation against, since both groups received the same Processing Instruction treatment.
    • "Two groups were formed: school-age (n = 36); and young adults (n = 13)." (p. 254)
    • Relevant Quotes: 1) "Two groups were formed: school-age (n = 36); and young adults (n = 13)." (p. 254) 2) "During the instruction period the groups were taught by the same instructor (the researcher)." (p. 254) 3) "The teaching schedule in the two institutions allowed for the treatments to take place over two consecutive days for a total of two hours of instruction for each group." (p. 254) Detailed Analysis: Following the criterion B decision procedure: extra instructional resources (two hours of dedicated Processing Instruction) are present for the treated participants, but there is no control group in this design at all - neither a business-as-usual comparison group nor an alternative-instruction group - against which to verify balance of time or resources. The comparison is exclusively between two age groups that both received the identical resource-intensive treatment. Because there is no control condition, the balance requirement cannot be verified as satisfied, and per the decision tree, absent a control group receiving comparable (or intentionally business-as-usual) resources, this falls into the "otherwise -> not met" branch. Criterion B is not met because the study lacks any control condition against which to assess resource/time balance.

• Level 3 Criteria

  • R

    Reproduced

    • All related prior studies cited are by the same author (Benati, alone or with co-authors), the authors themselves state the study needs to be replicated, and a targeted internet search found no independent replication of this specific age x cognitive task demands design by a different research team.
    • "Therefore, this study would need to be replicated before its findings can be generalized." (p. 267)
    • Relevant Quotes: 1) "Benati (2005) measured the effects of Processing Instruction, traditional instruction and meaning output-based instruction on the acquisition of English past simple tense." (p. 250) 2) "Benati (2013) has compared the performance of children and older school-age learners who received Processing Instruction on the English passive construction." (p. 251) 3) "Therefore, this study would need to be replicated before its findings can be generalized. It must be said, however, that the similarities between the results of this study and previous empirical research measuring the relative effects of Processing Instruction in learners of different ages lend validity to the findings presented in this paper." (p. 267) Detailed Analysis: Criterion R requires independent replication by a different research team, ideally published in a peer-reviewed outlet. The prior Processing Instruction/age studies cited throughout the paper (Benati 2005; Benati, Lee and Houghton 2008; Benati 2013; Mavrantoni and Benati 2013) are all authored or co-authored by the same researcher (Benati), so they do not constitute independent replication of this specific study. The authors explicitly acknowledge that "this study would need to be replicated before its findings can be generalized," confirming that no independent replication of this specific study (age x cognitive task demands x past tense acquisition) currently exists. Internet search update (verification pass): a search of Google Scholar, ResearchGate and publisher databases for citing/related studies on "Processing Instruction" and English past simple tense with German L1 learners located only further work by Benati and co-authors (e.g. subsequent Benati-authored Processing Instruction studies) and unrelated studies with different designs (e.g. a 2025 study "Enhancing English Past Tense Acquisition: Comparative Effects of Structured Input, Referential, and Affective Activities" in Languages 10(9), which tests different instructional conditions with a different age group and design and is not a replication of this specific age x cognitive task demands study). No study by a fully independent research team replicating this specific design (school-age vs. adult L1 German learners, two interpretation tasks of differing cognitive complexity, plus a production task) was found. Criterion R is not met because no independent replication by a different research team was found; all related work is by the same author, and internet search confirmed no subsequent independent replication of this specific study.
  • A

    All-subject Exams

    • Because criterion E is not met, criterion A is automatically not met; additionally, only a single grammatical feature was assessed, not all core subjects.
    • "The grammatical feature chosen for this study is the English past simple tense marking tested on both interpretation and production measures." (p. 249)
    • Relevant Quotes: 1) "The grammatical feature chosen for this study is the English past simple tense marking tested on both interpretation and production measures." (p. 249) Detailed Analysis: Per the criteria-specific instruction, since Criterion E (Exam-based Assessment) is not met, Criterion A cannot be met either. Substantively, the study also only assesses a single narrow grammatical feature (the English simple past tense -ed marker) via interpretation and production tasks, with no assessment of other subjects or broader language domains. Criterion A is not met both because Criterion E is not met and because only one narrow grammatical feature was assessed.
  • G

    Graduation Tracking

    • Because criterion Y (Year Duration) is not met, criterion G is automatically not met; tracking stopped two weeks after instruction, the authors flag this as a limitation, and no follow-up publication tracking these cohorts toward graduation was found via internet search.
    • "...long-lasting effects of instruction in this study were measured only over a period of two weeks." (p. 267)
    • Relevant Quotes: 1) "Post-tests were used immediately after instruction and two weeks after the instructional period." (p. 254) 2) "The second limitation of the present study is that the long-term effects of the variables under investigation should be re-examined as long-lasting effects of instruction in this study were measured only over a period of two weeks." (p. 267) Detailed Analysis: Per the criteria-specific instruction, since Criterion Y (Year Duration) is not met, Criterion G cannot be met either. Substantively, Criterion G requires tracking of participants until graduation from their educational stage. Here, the final data collection point is a delayed post-test only two weeks after the two-day instructional treatment. The authors themselves identify this short follow-up window as a limitation of the study and call for further research on longer-term effects. Internet search update (verification pass): a search for subsequent publications by Benati and/or Angelovska tracking the same school-age (Maria Ward Gymnasium Nymphenburg) or adult (LMU Language Centre) cohorts from this study found no such follow-up study. Later related publications by these authors (e.g. subsequent Processing Instruction and structured-input studies, including EEG/working-memory studies with L1 German school-age learners) use different, separately recruited samples rather than continuing to track this study's specific cohorts, and none extend measurement toward graduation. Criterion G is not met because follow-up ended two weeks after instruction, with no tracking toward graduation, and no later publication tracking these cohorts to graduation was found.
  • P

    Pre-Registered

    • No statement of pre-registration, registry platform, or registration date appears anywhere in the paper, and an internet search of common registries found no pre-registration record for this study.
    • Relevant Quotes: No quotes referencing a pre-registration platform, protocol registration, registry ID, or registration date were found anywhere in the Methodology, Procedures, Acknowledgments, or other sections of the paper. Detailed Analysis: Criterion P requires evidence that the full study protocol (hypotheses, methods, planned analyses) was pre-registered on a public registry before data collection began. The paper contains a detailed Methodology section, a Research Questions and Hypotheses section, and an Acknowledgments section (crediting a Leverhulme Trust postdoctoral fellowship grant), but none of these mention a registry, a pre-registration date, or a protocol published ahead of data collection. Internet search update (verification pass): a search for a pre-registration record associated with this study (e.g. via OSF, AsPredicted, or similar registries, and via the authors' names and the study title) found no pre-registration entry. This is unsurprising given the paper's 2015 publication date, predating widespread pre-registration practice in applied linguistics/SLA research, and given the absence of any pre-registration statement in the paper itself. Criterion P is not met because there is no evidence anywhere in the paper, nor in publicly searchable registries, of a pre-registered protocol for this study.