Implicit and Explicit Corrective Feedback and the Acquisition of L2 Grammar

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Although allocation was at the class level, the three intact classes were "arbitrarily assigned" with no described randomisation procedure, so a properly implemented class-level RCT is not demonstrated.
    • "Classes were arbitrarily assigned to one of the two treatment options (group 1 = 12 students, group 2 = 12 students) or to the control group option (group 3 = 10 students)." (p. 351)
    • Relevant Quotes: 1) "Classes were arbitrarily assigned to one of the two treatment options (group 1 = 12 students, group 2 = 12 students) or to the control group option (group 3 = 10 students)." (p. 351) 2) "The study was conducted in a private language school in New Zealand. Three classes of students (n = 34) were involved." (p. 350) 3) "Also, we were forced to use intact groups, with the result that the groups were not equivalent at the commencement of the study, thus obligating the use of ANCOVA." (p. 366) 4) "In an experimental design (two experimental groups and a control group), low-intermediate learners of second language English completed two communicative tasks..." (p. 339, abstract) Detailed Analysis: Criterion C requires a randomised controlled trial with randomisation clearly described and properly implemented at the class level or higher. In this study the unit of assignment was indeed the intact class (three whole classes were allocated to the three conditions), so the unit of allocation would satisfy the class-level requirement. However, the paper explicitly states that classes were "arbitrarily assigned," not randomly assigned, and no randomisation procedure (random number generation, coin toss, stratification, etc.) is described anywhere in the paper. The authors themselves acknowledge in the limitations that they "were forced to use intact groups" and that the groups "were not equivalent at the commencement of the study" - significant pretest differences were found on the oral imitation and grammaticality judgment tests. With only three clusters allocated arbitrarily, the study is better characterised as a quasi-experiment at the class level than a properly implemented class-level RCT. The intervention is whole-class corrective feedback, not one-to-one tutoring, so the tutoring exception does not apply. This lack of true randomisation is also why the top-level "rct" field has been corrected to false during this verification (see 2.9 change summary): nowhere in the paper is a genuine random-assignment procedure described, only an "arbitrary" one. Criterion C is not met because assignment of the three intact classes is described as "arbitrary" rather than random, and no randomisation procedure is described or properly implemented.
  • E

    Exam-based Assessment

    • All three outcome measures were custom instruments designed by the researchers for this study, not widely recognised standardised exams.
    • "Acquisition was measured by means of an oral imitation test (designed to measure implicit knowledge) and both an untimed grammaticality judgment test and a metalinguistic knowledge test (both designed to measure explicit knowledge)." (p. 339)
    • Relevant Quotes: 1) "Acquisition was measured by means of an oral imitation test (designed to measure implicit knowledge) and both an untimed grammaticality judgment test and a metalinguistic knowledge test (both designed to measure explicit knowledge)." (p. 339, abstract) 2) "Oral Imitation Test. This test consisted of a set of 36 belief statements. Statements were grammatically correct (n = 18) or incorrect (n = 18)." (p. 354) 3) "Grammaticality Judgment Test. This was a pen-and-paper test consisting of 45 sentences. Fifteen sentences targeted past tense -ed, and the remainder targeted 30 other structures." (p. 355) 4) "Metalinguistic Knowledge Test. Learners were presented with five sentences and were told that they were ungrammatical. Two of the sentences contained errors in past tense -ed." (p. 356) 5) "For more information about the theoretical rationale for this test and its design, see Erlam (in press)." (p. 355) Detailed Analysis: Criterion E requires that outcomes be measured with widely recognised standardised exams rather than instruments created for the study. All three outcome measures here (the oral elicited imitation test, the untimed grammaticality judgment test, and the metalinguistic knowledge test) were purpose-built research instruments designed by the researchers to separate implicit from explicit knowledge of a single grammatical structure (past tense -ed), drawing on the authors' own prior instrument-development work (R. Ellis 2005; Erlam, in press). No national curriculum exam, state-wide achievement test, or other recognised standardised examination was used. These are exactly the kind of custom, intervention-aligned measures the criterion is designed to guard against. Criterion E is not met because all outcome measures were custom researcher-designed tests rather than recognised standardised exams.
  • T

    Term Duration

    • Outcomes were last measured about two weeks after the 1-hour, 2-day intervention began, far short of the required full academic term of follow-up.
    • "The tests were administered prior to the instruction, 1 day after the instruction, and again 2 weeks later." (p. 339)
    • Relevant Quotes: 1) "For the purposes of the study, each experimental group received the same amount of instruction (i.e., a total of 1 hr over 2 consecutive days during which they completed two different half-hour communicative tasks)." (pp. 351-352) 2) "Five days prior to the start of the instructional treatments, the learners involved in the study signed the consent forms ... and completed all of the pretests. The immediate posttesting was completed the day after the second (and last) day of instruction, and the delayed posttesting was completed 12 days later." (p. 353) 3) "The tests were administered prior to the instruction, 1 day after the instruction, and again 2 weeks later." (p. 339, abstract) 4) "Third, the length of the treatments was very short (approximately 1 hr)." (p. 366) Detailed Analysis: Criterion T requires that outcomes be measured at least one full academic term (roughly 3-4 months) after the intervention begins. Here the intervention consisted of just 1 hour of instruction spread over 2 consecutive days, the immediate posttest was administered 1 day after instruction ended, and the final delayed posttest was administered only about 2 weeks after the intervention began. The total interval from intervention start to final outcome measurement is approximately two weeks, which falls far short of one academic term. The authors themselves flag the very short treatment as a limitation. Criterion T is not met because the final outcome measurement occurred about 2 weeks after the intervention began, far less than one academic term.
  • D

    Documented Control Group

    • The control group's size (n = 10), condition (testing-only with normal instruction and no feedback), and baseline test scores are clearly documented in the text and tables.
    • "The control group continued with their normal instruction. They did not complete the tasks and did not receive any feedback on past tense -ed errors." (p. 352)
    • Relevant Quotes: 1) "Group 1 received implicit feedback (recast group), group 2 received explicit feedback (metalinguistic group), and group 3 (a testing control) had no opportunity to practice the target structure and, thus, received no feedback." (p. 350) 2) "Classes were arbitrarily assigned to one of the two treatment options (group 1 = 12 students, group 2 = 12 students) or to the control group option (group 3 = 10 students)." (p. 351) 3) "The control group continued with their normal instruction. They did not complete the tasks and did not receive any feedback on past tense -ed errors." (p. 352) 4) "Information obtained from a background questionnaire showed that the majority of learners (77%) were of East Asian origin. Most of them had spent less than a year in New Zealand (the mean length of stay was just over 6 months). The mean age of all participants was 25 years." (p. 350) 5) "Test-retest reliability (Pearson's r) was calculated for the control group (n = 10) only." (p. 355) 6) Tables 3, 4, 5, 6, and 7 (pp. 357-360) report the control group's pretest, immediate posttest, and delayed posttest means and standard deviations on all three outcome measures. Detailed Analysis: Criterion D requires clear documentation of the control group: who they are, their size, baseline performance, and what treatment they received. The paper specifies the control group's size (one class of 10 students), its condition (a testing control that continued normal instruction, completed all three tests at all three time points, did not perform the treatment tasks, and received no feedback on the target structure), and its baseline performance (pretest means and SDs reported separately for the control group in Tables 3-5). Sample-level demographics (origin, age, length of English study, proficiency level) are documented, although not broken down by group, and the authors transparently note baseline non-equivalence and handle it with ANCOVA. Taken together, this constitutes adequate documentation of the control group for comparison purposes. Criterion D is met because the control group's size, condition (testing-only, normal instruction, no feedback), and baseline performance are clearly documented.

• Level 2 Criteria

  • S

    School-level RCT

    • The study was conducted within one private language school with three classes allocated to conditions, so there was no school-level randomisation.
    • "The study was conducted in a private language school in New Zealand. Three classes of students (n = 34) were involved." (p. 350)
    • Relevant Quotes: 1) "The study was conducted in a private language school in New Zealand. Three classes of students (n = 34) were involved." (p. 350) 2) "Classes were arbitrarily assigned to one of the two treatment options (group 1 = 12 students, group 2 = 12 students) or to the control group option (group 3 = 10 students)." (p. 351) Detailed Analysis: Criterion S requires randomisation among schools (or equivalent implementing institutions). This study took place within a single private language school, and allocation was of three classes within that one school to the three conditions. There is no mention of multiple schools or sites being randomised. Moreover, as noted under criterion C, even the class-level allocation was "arbitrary" rather than random. Criterion S is not met because the study involved a single language school with classes (not schools) allocated to conditions.
  • I

    Independent Conduct

    • The intervention was designed, delivered, and evaluated by the same research team, with one of the researchers personally teaching the treatment sessions and no independent evaluator involved.
    • "The same instructor - one of the researchers - was responsible for conducting both tasks." (p. 353)
    • Relevant Quotes: 1) "The same instructor - one of the researchers - was responsible for conducting both tasks. The learners had not met the instructor prior to the first treatment session." (p. 353) 2) "An observer sat in the classroom during each session to manually record on paper all instances of the use of the target structure and each instance of corrective feedback." (p. 353) 3) "This research was funded by a Marsden Fund grant awarded by the Royal Society of Arts of New Zealand. Researchers other than the authors who contributed to the research were Jenefer Philip, Satomi Mizutami, Keiko Sakui, and Thomas Delaney." (p. 339, author note) 4) "This study was designed to provide a precise comparison between implicit and explicit corrective feedback by operationalizing these constructs..." (p. 350) Detailed Analysis: Criterion I requires that the study be conducted independently of the intervention's designers, or at least that data collection and analysis be carried out by an external party. In this study the authors designed the feedback treatments, the tasks, and the testing instruments, and one of the researchers personally delivered the instructional treatment to both experimental groups. Data collection, scoring, and analysis were all carried out by the research team itself; the additional named researchers were collaborators on the same project, not an external evaluation body. There is no statement of third-party oversight or an independent evaluation team. Criterion I is not met because the same research team designed the intervention, delivered it (one researcher taught the treatment sessions), and conducted all data collection and analysis without independent oversight.
  • Y

    Year Duration

    • The full tracking period was about two weeks from intervention start, nowhere near 75% of an academic year (and criterion T is already not met).
    • "The immediate posttesting was completed the day after the second (and last) day of instruction, and the delayed posttesting was completed 12 days later." (p. 353)
    • Relevant Quotes: 1) "For the purposes of the study, each experimental group received the same amount of instruction (i.e., a total of 1 hr over 2 consecutive days during which they completed two different half-hour communicative tasks)." (pp. 351-352) 2) "The immediate posttesting was completed the day after the second (and last) day of instruction, and the delayed posttesting was completed 12 days later." (p. 353) Detailed Analysis: Criterion Y requires outcome tracking covering at least 75% of an academic year (roughly 9-10 months) from intervention start. As established under criterion T, the entire study from intervention start to final measurement spanned about two weeks. Since criterion T (one term) is not met, criterion Y cannot be met either, and the two-week tracking window is a tiny fraction of an academic year. Criterion Y is not met because outcomes were tracked for only about two weeks, far short of 75% of an academic year.
  • B

    Balanced Control Group

    • The two treatment groups received identical amounts of instruction (1 hr each), no extra time or budget was added beyond regular schooling, and the testing control simply continued its normal instruction during the same period.
    • "For the purposes of the study, each experimental group received the same amount of instruction (i.e., a total of 1 hr over 2 consecutive days during which they completed two different half-hour communicative tasks). The control group continued with their normal instruction." (pp. 351-352)
    • Relevant Quotes: 1) "For the purposes of the study, each experimental group received the same amount of instruction (i.e., a total of 1 hr over 2 consecutive days during which they completed two different half-hour communicative tasks). The control group continued with their normal instruction. They did not complete the tasks and did not receive any feedback on past tense -ed errors." (pp. 351-352) 2) "Learners received between 3 and 5 hr of English language instruction a day, for which they were enrolled as part-time or full-time students." (p. 351) 3) "Group 1 received implicit feedback (recast group), group 2 received explicit feedback (metalinguistic group), and group 3 (a testing control) had no opportunity to practice the target structure and, thus, received no feedback." (p. 350) 4) "we only included a testing group as a control group (i.e., we did not have a control group that completed the communicative tasks without any corrective feedback)." (p. 366) Detailed Analysis: Applying the criterion B decision tree: the first question is whether the intervention added extra time or budget relative to business as usual. Here the "tasks" replaced 1 hr of the regular daily 3-5 hr instructional schedule that all learners (including the control class) were already enrolled in; the control class simply continued its normal lessons during that same 1 hr block rather than receiving no instruction. No additional out-of-schedule time, budget, or materials were given to the experimental groups beyond what was already part of their enrolled classes. Thus EXTRA_RESOURCES_PRESENT is false and the criterion is met at step 1 of the decision tree. Separately, the two experimental arms (recast vs. metalinguistic) were tightly balanced with each other: both received exactly the same tasks, the same total instruction time (1 hr), and the same instructor, differing only in the type of corrective feedback delivered (the treatment variable itself); Table 2 shows comparable numbers of target forms and feedback episodes elicited in both groups. The acknowledged design limitation (the control did not perform the tasks without feedback) concerns isolating feedback from task practice/exposure, not a time/budget imbalance, since the control received its normal instruction in the same period. Criterion B is met because no extra educational time or budget was added relative to the control group's business-as-usual instruction, and the two treatment arms were exactly balanced with each other.

• Level 3 Criteria

  • R

    Reproduced

    • A different, independent research team (Paraskeva & Agathopoulou, 2022) published a peer-reviewed study replicating the recasts-versus-metalinguistic-feedback contrast on the same target structure (past tense -ed) in a different context; Sauro (2009) offers weaker, conceptual support with a different target structure.
    • Paraskeva, E., & Agathopoulou, E. (2022). "Recasts, metalinguistic feedback, anxiety and grammatical development." Research Papers in Language Teaching and Learning, 12(1), 26-43 - an independently authored, peer-reviewed replication of the recast vs. metalinguistic feedback comparison on regular past tense -ed in a different learner population and context.
    • Relevant Quotes (from the paper): 1) "To the best of our knowledge, this is the first experimental study that has compared the effects of online explicit corrective feedback in the form of metalinguistic explanations and implicit corrective feedback in the form of recasts in classroom-based instruction." (p. 364) External evidence found via internet search (post-publication, other research teams): 2) Sauro, S. (2009). "Computer-Mediated Corrective Feedback and the Development of L2 Grammar." Language Learning & Technology, 13(1), 96-120. Per the ERIC abstract record (EJ827544), this study compares "corrective feedback that reformulates the error in the form of recasts" against "corrective feedback that supplies the learner with metalinguistic information about the nature of the error," with adult learners, using a control group. However, the target structure studied was "omission of the zero article with abstract noncount nouns (e.g., employment, global warming, culture)," not past tense -ed, and the feedback was delivered via text-chat with Swedish university learners rather than in oral classroom interaction. This constitutes only a conceptual replication of the recast-vs-metalinguistic contrast, not a direct replication of this specific study's experimental claim, because the target structure and modality differ substantially. 3) Paraskeva, E., & Agathopoulou, E. (2022). "Recasts, metalinguistic feedback, anxiety and grammatical development." Research Papers in Language Teaching and Learning, 12(1), 26-43. Per the journal's published table of contents and abstract, this study examines oral corrective feedback in the form of recasts (implicit) versus metalinguistic feedback (explicit) on the acquisition of regular past tense -ed (and subject-verb agreement -s) among young EFL learners, treating learner anxiety as a moderating variable. This paper was not found to be accessible in a form that allowed extraction of full in-text verbatim quotes (the source PDF is image-based and blocked full-text extraction), so only the verified bibliographic and abstract-level description above could be confirmed; no invented quotes are included here. Detailed Analysis: Criterion R requires independent replication of the study by a different research team, in a different context, published in a peer-reviewed outlet. The original 2006 paper naturally reports no replication of itself. An internet search located Paraskeva and Agathopoulou (2022), a peer-reviewed study by a different, unaffiliated research team, conducted with a different (younger EFL) population in a different country, that reproduces the central experimental contrast of this study - recasts versus metalinguistic feedback - on the identical target structure (regular past tense -ed). This constitutes a credible independent replication of the central design. Sauro (2009) independently reproduces the same feedback-type contrast by a different team in a different context but on a different target structure and modality, so it is treated as weaker, conceptual support rather than a direct replication. Note: agent1/agent2 blind rankings were not consulted; a previously circulated claim of a fourth, unnamed "further peer-reviewed EFL study" could not be verified via internet search and has been dropped from this analysis, per the instruction not to include quotes or references that cannot be confirmed. Criterion R is met because a verified peer-reviewed study by an independent research team (Paraskeva & Agathopoulou, 2022) replicates the central recasts-versus-metalinguistic-feedback contrast on the same target structure (past tense -ed) in a different context.
  • A

    All-subject Exams

    • Criterion E is not met and the study assessed only one English grammar structure with custom tests, so all-subject standardised assessment is absent.
    • "The target grammatical structure was past tense -ed." (p. 350)
    • Relevant Quotes: 1) "Acquisition was measured by means of an oral imitation test (designed to measure implicit knowledge) and both an untimed grammaticality judgment test and a metalinguistic knowledge test (both designed to measure explicit knowledge)." (p. 339, abstract) 2) "The target grammatical structure was past tense -ed." (p. 350) Detailed Analysis: Criterion A requires standardised exam-based assessment across all main subjects, and explicitly states that if criterion E is not met, criterion A cannot be met. Criterion E is not met here, since all measures were custom research instruments. Furthermore, outcomes were measured only in a single narrow domain - accuracy in one English grammatical structure (regular past tense -ed) - with no assessment of any other subject or even of broader English proficiency using standardised exams. While this is an adult ESL context where English is the focal subject, no standardised exam of any subject was used, so no exception can rescue the criterion. Criterion A is not met because criterion E fails and only a single grammatical structure was assessed with custom tests, with no standardised assessment of any subject.
  • G

    Graduation Tracking

    • Measurement stopped roughly two weeks after the intervention, with no tracking of participants to graduation or course completion, confirmed by internet search to find no follow-up publications on this cohort.
    • "The immediate posttesting was completed the day after the second (and last) day of instruction, and the delayed posttesting was completed 12 days later." (p. 353)
    • Relevant Quotes: 1) "The immediate posttesting was completed the day after the second (and last) day of instruction, and the delayed posttesting was completed 12 days later." (p. 353) 2) "Learners received between 3 and 5 hr of English language instruction a day, for which they were enrolled as part-time or full-time students." (p. 351) Detailed Analysis: Criterion G requires tracking participants until graduation from their educational stage, and per the instructions it cannot be met when criterion Y is not met. Criterion Y fails here. Measurement stopped 12 days after the immediate posttest (about two weeks after instruction), and the paper reports no further follow-up. The participants were adult learners enrolled part-time or full-time in a private language school, a setting without a defined graduation endpoint. An internet search for follow-up publications by Ellis, Loewen, or Erlam tracking this same private-language- school cohort found no evidence of any such study. Criterion G is not met because tracking ended about two weeks after the intervention with no follow-up to any graduation or completion point (and criterion Y is not met).
  • P

    Pre-Registered

    • The paper contains no reference to any pre-registered protocol or trial registry, only ethics approval and grant funding, and no registry record was found via internet search.
    • Relevant Quotes: 1) "Five days prior to the start of the instructional treatments, the learners involved in the study signed the consent forms, as required by the University of Auckland Human Participants Ethics Committee, and completed all of the pretests." (p. 353) 2) "This research was funded by a Marsden Fund grant awarded by the Royal Society of Arts of New Zealand." (p. 339, author note) Detailed Analysis: Criterion P requires that the full study protocol (hypotheses, methods, planned analyses) be publicly pre-registered before data collection began, with a verifiable registry reference and date. The paper mentions only ethics approval and grant funding; it contains no reference to any trial registry (e.g., ClinicalTrials.gov, ISRCTN, OSF, AEA registry), no registration ID, and no registration date. Published in 2006, the study predates the widespread practice of pre-registration in applied linguistics. An internet search found no registry record for this study, consistent with the fact that pre-registration was not yet standard practice in this field at that time. Criterion P is not met because no pre-registration of the study protocol is mentioned in the paper or found via internet search.