Direct versus Indirect Explicit Methods of Enhancing EFL Students' English Grammatical Competence: A Concept Checking-based Consciousness-raising Tasks Model

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Two intact classes were conveniently selected and one was simply "designated" as the experimental group, so no proper class-level (or any) randomisation is described.
    • "Two classes whose English grades in the second semester exam were relatively equal were selected. ... One of the two classes was designated as an experimental group." (p. 115)
    • Relevant Quotes: 1) "Ninety-four eleventh-graders were conveniently selected and randomly assigned into either the experimental group (EG) or the control group (CG)." (Abstract, p. 112) 2) "Two classes whose English grades in the second semester exam were relatively equal were selected. ... One of the two classes was designated as an experimental group. This class, which had 47 members, was more heavily weighted towards females (33) than males (14); the other was treated as the control group having 47 students, 26 of whom were females and the remaining (21) were males." (p. 115) 3) "In this study, the target sample was EFL high school students in two classes that were selected conveniently among the population of 25 Grade-11 at Marie Curie High School in Ho Chi Minh City-Vietnam." (p. 115) Detailed Analysis: Criterion C requires a class-level RCT with a clearly described and properly implemented randomisation procedure. The abstract claims students were "randomly assigned," but the Method section contradicts this: two pre-existing intact classes were "selected conveniently," and one class "was designated" as the experimental group with no description of any random procedure for that designation. The differing gender compositions (33F/14M vs 26F/21M) confirm intact classes rather than random assignment of students. With only two conveniently chosen classes and an unexplained, apparently non-random designation of condition, the study does not document proper class-level randomisation, and the intervention is whole-class instruction, so the tutoring exception does not apply. Criterion C is not met because allocation used two conveniently selected intact classes with one merely "designated" as experimental, without any described randomisation process.
  • E

    Exam-based Assessment

    • Outcomes were measured with researcher-designed rule analysis, grammar, oral, and written tests piloted for the study, not with any recognised standardised exam.
    • "The RAT and the GT were designed in both subjective and objective forms." (p. 116)
    • Relevant Quotes: 1) "The assessment instruments that were involved in this study were a rule analysis test (RAT), a grammar test (GT), an oral test (OT), a written test (WT), and a survey questionnaire." (p. 116) 2) "The teaching plans and instruments were first piloted for their reliability. The trial teaching was conducted with a class of 31 students who did the pilot test before the treatment phase." (p. 115) 3) "The RAT and the GT were designed in both subjective and objective forms. The subjective items were for measuring the students' ability to avoid errors in using tenses and the objective form was used to check the learners' recognition of tenses." (p. 116) 4) "All tests were designed as a means not only to assess the proficiency level of the targeted structure but also to measure knowledge of grammatical form, meaning, and use." (p. 116) Detailed Analysis: Criterion E requires the use of widely recognised standardised exams rather than instruments created for the study. The paper repeatedly states that the tests were "designed" by the researcher and piloted before the experiment, and none of the instruments (RAT, GT, OT, WT) is identified as a national, state, or otherwise recognised standardised examination. These are custom, study-specific measures aligned to the taught tense structures, precisely the situation the criterion is meant to guard against. Criterion E is not met because all outcome measures were custom researcher-designed tests rather than standardised exams.
  • T

    Term Duration

    • The intervention lasted eight weeks with main post-tests one day later and only a single delayed written test about three months after the start, so a full term of tracking is not clearly reached.
    • "The teaching phase lasted eight weeks, each had 90 minutes." (p. 115)
    • Relevant Quotes: 1) "The teaching phase lasted eight weeks, each had 90 minutes." (p. 115) 2) "The pretest was administered a week before the treatments. The post-test including the RAT and the GT was conducted one day after the two instructional methods implemented, and the OT was administered a week after that. ... Finally, the delayed post-test was given a month after the speaking section took place." (p. 116) 3) "Students of both classes did the post-tests after the treatment: the rule analysis test and the grammar test. Then, they took part in the oral test the following week, and a month after that, they continued doing the delayed written test." (p. 115) Detailed Analysis: Criterion T requires that outcomes be measured at least one full academic term (roughly 3-4 months) after the intervention begins. Here the intervention ran for eight weeks (~2 months). The primary post-tests (RAT and GT) were given one day after the intervention ended, i.e. about two months after the start, and the oral test one week later. Only the delayed written retention test, given a month after the oral test, falls roughly 13 weeks (~3 months) after the intervention began, which sits at or below the lower bound of a term, and the paper gives no calendar dates or definition of a term to establish that a full term elapsed. The main outcome measurements clearly occurred well within one term of the start. Criterion T is not met because the principal outcomes were measured about two months after the intervention began and the single delayed test does not clearly reach a full academic term of tracking.
  • D

    Documented Control Group

    • The control group's size, gender mix, proficiency level, baseline equivalence on semester grades, and the instruction it received are all clearly documented.
    • "the other was treated as the control group having 47 students, 26 of whom were females and the remaining (21) were males." (p. 115)
    • Relevant Quotes: 1) "The ninety-four participants in this research project were students whose English levels were pre-intermediate of the 'basic' cohorts (as categorized by the high-school)." (p. 115) 2) "Two classes whose English grades in the second semester exam were relatively equal were selected. Specifically, each class had seven students getting the under-average scores and the other forty students, achieving fair, good, and excellent grades, were nearly the same." (p. 115) 3) "the other was treated as the control group having 47 students, 26 of whom were females and the remaining (21) were males." (p. 115) 4) "Students in two classes received the same amount of time, contextualized input, and practice. Although control students had the same input as that of the counterparts, the techniques were different (i.e., no interaction to raise students' notice of grammatical forms). In addition, while deductive subjects were presented and explained the grammar rules, the inductive students were engaged in 'rule- searched' tasks." (p. 115) Detailed Analysis: Criterion D requires clear documentation of the control group's composition, baseline standing, and the treatment it received. The paper specifies the control group's size (47), gender breakdown (26 female, 21 male), grade level (Grade 11), proficiency band (pre-intermediate 'basic' cohort), and baseline comparability with the experimental class on second semester English grades, including the distribution of under-average versus higher achievers. It also describes exactly what the control group experienced: the same amount of time and contextualised input, but direct explicit (deductive) grammar instruction. Table 1 further reports control group means and SDs on each test. This is adequate documentation for comparison. Criterion D is met because the control group's size, demographics, baseline performance, and instructional conditions are clearly described.

• Level 2 Criteria

  • S

    School-level RCT

    • The study took place in a single high school with only two classes, so there was no school-level randomisation.
    • "the target sample was EFL high school students in two classes that were selected conveniently among the population of 25 Grade-11 at Marie Curie High School in Ho Chi Minh City-Vietnam." (p. 115)
    • Relevant Quotes: 1) "In this study, the target sample was EFL high school students in two classes that were selected conveniently among the population of 25 Grade-11 at Marie Curie High School in Ho Chi Minh City-Vietnam." (p. 115) 2) "One of the two classes was designated as an experimental group. ... the other was treated as the control group." (p. 115) Detailed Analysis: Criterion S requires randomisation among schools or equivalent implementing units. This study involved a single school (Marie Curie High School) and just two classes within it; assignment occurred (non-randomly) at the class level within that one school. No quote indicates that multiple schools or sites were involved, let alone randomised. Criterion S is not met because the entire study was conducted within one school with two conveniently selected classes and no school-level randomisation.
  • I

    Independent Conduct

    • The first author, a teacher at the study school, designed the treatment, taught it, and analysed the data herself with no independent evaluation team.
    • "First, the teacher had four periods (180 minutes) of trial teaching. Then, she interviewed two students..." (p. 115)
    • Relevant Quotes: 1) "Trang Thị Đoan Đặng1 & Hương Thu Nguyễn2" (p. 112, byline); "1 Marie Curie High School, Ho Chi Minh City, Vietnam" (p. 112, affiliation) 2) "First, the teacher had four periods (180 minutes) of trial teaching. Then, she interviewed two students: a male and a female to get their feedback on the method she had applied. ... She recorded the whole trial and interviews with an MP3 player and took some necessary notes so that she could have specific information to adjust the lesson plans and the instruments if needed." (p. 115) 3) "As such, the GT is a kind of test that the researcher used to assess whether or not the students were able to use (practice) the structure learned (i.e. present tenses)." (p. 117) 4) "First, I send my greatest gratitude to my supervisor, Dr. Hương Thu Nguyễn, for his valuable guidance, advice, encouragement, and comments. ... Finally, my special thanks to Mr Tâm Thanh Trần, who has guided me to operate and analyze the SPSS statistics, is my colleague and my husband as well." (pp. 120-121) Detailed Analysis: Criterion I requires that the study be conducted independently of the intervention's designers, or at least with third-party oversight of data collection and analysis. Here the study is a supervised thesis project: the first author is a teacher at the study school who developed the lesson plans and instruments, piloted and delivered the instruction herself, administered the tests, and analysed the data (with statistical help from her husband/colleague). There is no external evaluation team, no blinded assessors, and no statement of independent oversight anywhere in the paper. Criterion I is not met because the same researcher designed, implemented, and evaluated the intervention without any independent conduct or oversight.
  • Y

    Year Duration

    • Total tracking from intervention start to the final delayed test was roughly three months, far short of 75% of an academic year.
    • "The teaching phase lasted eight weeks, each had 90 minutes." (p. 115)
    • Relevant Quotes: 1) "The teaching phase lasted eight weeks, each had 90 minutes." (p. 115) 2) "The post-test including the RAT and the GT was conducted one day after the two instructional methods implemented, and the OT was administered a week after that. ... Finally, the delayed post-test was given a month after the speaking section took place." (p. 116) Detailed Analysis: Criterion Y requires outcome measurement at least 75% of an academic year (roughly 9-10 months) after the intervention begins. The intervention here lasted eight weeks, and even the last measurement point (the delayed written test) came only about five to six weeks after the intervention ended, giving a total span of roughly three months from start to final measurement. This is well under 75% of any academic year definition. Per the prompt rules, criterion T is also not met, which additionally forces Y to be not met. Criterion Y is not met because the full tracking window was about three months, far below 75% of an academic year.
  • B

    Balanced Control Group

    • Both classes received the same amount of instructional time, the same contextualised input, and practice, differing only in the instructional technique being tested.
    • "Students in two classes received the same amount of time, contextualized input, and practice." (p. 115)
    • Relevant Quotes: 1) "The teaching phase lasted eight weeks, each had 90 minutes. Students in two classes received the same amount of time, contextualized input, and practice." (p. 115) 2) "Although control students had the same input as that of the counterparts, the techniques were different (i.e., no interaction to raise students' notice of grammatical forms)." (p. 115) 3) "On the other hand, the direct explicit group received different instruction in step 3; they were presented and explained the grammar rules. This group was also exposed to similar input (i.e., a reading/listening activity) as that of the inductive group (step 2)." (p. 115) 4) "this project largely addressed contextualized use of tense and aspect associated with consciousness-raising tasks for the EG, and to avoid tedious learning conditions, contextualization was also applied for the CG." (p. 113) Detailed Analysis: Criterion B requires that time and resources be balanced between conditions unless extra resources are themselves the treatment variable. Applying the decision tree: the intervention (indirect explicit grammar instruction via consciousness-raising tasks) added no extra time, budget, or materials relative to the control. Both groups received eight weekly 90-minute lessons, the same contextualised reading/listening input, and the same practice stage; the only difference was step 3 of the lesson - rule-discovery tasks with feedback for the EG versus teacher presentation and explanation of rules for the CG. The control condition was an active, time-matched alternative instruction (business-as-usual DEGI), so no resource imbalance exists (branch 1 of the decision tree: no extra time/budget present, criterion met). Criterion B is met because both conditions received identical time, input, and practice, with only the instructional technique differing as the treatment contrast.

• Level 3 Criteria

  • R

    Reproduced

    • No independent replication of this specific study is reported; the paper only builds on earlier related research by Fotos and Ellis rather than being replicated itself.
    • "this project would like to reexamine the results of the previous studies by Fotos and Ellis (1991), Fotos (1993), and Fotos (1994)" (p. 113)
    • Relevant Quotes: 1) "In addition, this project would like to reexamine the results of the previous studies by Fotos and Ellis (1991), Fotos (1993), and Fotos (1994) compared with that of the present research under different settings (i.e. high school learners), grammar structures (i.e. tense and aspects), and kinds of instruments (i.e. oral and written tests)." (p. 113) 2) "This result reinforces the findings of the previous research performed by Fotos and Ellis (1991) and Fotos (1993, 1994) regarding the acquisition of explicit grammatical knowledge." (p. 118) Detailed Analysis: Criterion R requires that this study itself be independently replicated by a different team in a peer-reviewed outlet. The paper positions itself as a partial re-examination of earlier consciousness-raising research (Fotos and Ellis), i.e. it replicates others, not the reverse. No quote in the paper mentions any independent replication of this specific Vietnamese high-school study of concept checking-based consciousness-raising tasks. Internet search for citations and replications of this paper (Dang & Nguyen, 2013, English Language Teaching, 6(1), 112-121) found only one topically related study by a different team: Tilahun, S., Simegn, B., & Emiru, Z. (2022). "Using grammar consciousness-raising tasks to enhance students' narrative tenses competence." Cogent Education, 9(1), 2107471. That study compares consciousness-raising tasks against conventional teacher-fronted instruction for narrative tenses with grade-11 students in Ethiopia over five weeks; it neither cites nor references Dang & Nguyen (2013), does not replicate the specific direct-versus- indirect explicit (DEGI vs IEGI) design or instruments (RAT, GT, OT, WT) used here, and is set in an unrelated context. No other citing or replicating papers were found. General studies on inductive versus deductive grammar teaching by other authors are not replications of this specific trial. Criterion R is not met because no independent published replication of this specific study is documented, and a targeted internet search found no such replication.
  • A

    All-subject Exams

    • Only English tense/grammar outcomes were measured with custom tests; no other school subjects were assessed and criterion E is not met.
    • "This study aims to explore the effects of indirect explicit grammar instruction on EFL learners' mastery of English tenses." (p. 112)
    • Relevant Quotes: 1) "This study aims to explore the effects of indirect explicit grammar instruction on EFL learners' mastery of English tenses." (Abstract, p. 112) 2) "The assessment instruments that were involved in this study were a rule analysis test (RAT), a grammar test (GT), an oral test (OT), a written test (WT), and a survey questionnaire." (p. 116) Detailed Analysis: Criterion A requires standardised assessment of all main subjects taught at the educational level, with criterion E as a prerequisite. Criterion E is not met (all instruments were custom-made), which by itself makes A not met. In addition, the study measured only English grammar (tense and aspect) outcomes; no other Grade 11 subjects (mathematics, literature, science, etc.) were assessed, and no exception rationale for a specialised programme is offered. Criterion A is not met because criterion E fails and only a single narrow English grammar domain was assessed.
  • G

    Graduation Tracking

    • Measurement ended with a delayed written test about a month after the oral test, with no tracking of students to graduation.
    • "Finally, the delayed post-test was given a month after the speaking section took place." (p. 116)
    • Relevant Quotes: 1) "Finally, the delayed post-test was given a month after the speaking section took place." (p. 116) 2) "Then, they took part in the oral test the following week, and a month after that, they continued doing the delayed written test. The main purpose of these tests was to check the retention of the grammar structures..." (p. 115) Detailed Analysis: Criterion G requires following participants until graduation from their educational stage. These were Grade 11 students, and the last data point was a delayed written test roughly five weeks after the intervention ended - well before the end of Grade 11, let alone high school graduation (end of Grade 12). The paper mentions no planned follow-up, and no follow-up publications tracking this cohort are referenced. Internet search for subsequent publications by either author (Trang Thi Doan Dang, Marie Curie High School; Huong Thu Nguyen, Hoa Sen University) tracking this same cohort of 94 Grade-11 students toward graduation found no such follow-up paper. Furthermore, per the prompt rules, criterion Y is not met, which forces G to be not met as well. Criterion G is not met because tracking stopped about a month after the intervention, far short of graduation, no follow-up publications were found, and criterion Y is not met.
  • P

    Pre-Registered

    • The paper contains no mention of any pre-registration, registry, or published protocol.
    • Relevant Quotes: 1) "This project used a trial teaching practice and re- administration method to pilot the instruments before the experiment conducted." (p. 114) 2) "Received: October 13, 2012 Accepted: October 23, 2012 Online Published: December 12, 2012" (p. 112) Detailed Analysis: Criterion P requires that the study protocol, hypotheses, and analysis plan be registered on a public registry before data collection began. The paper describes piloting and administration procedures but never mentions any trial registry (e.g., ClinicalTrials.gov, ISRCTN, OSF), any registration ID, or any published protocol document. For a 2012 classroom thesis study of this type, no pre-registration evidence exists in the text. Internet search for a pre-registration record of this study (by title, authors, and DOI 10.5539/elt.v6n1p112) found no entry in any trial or study registry. This is unsurprising, as pre-registration was not standard practice for TESOL classroom research conducted and published around 2012. Criterion P is not met because no pre-registration or protocol registration is mentioned anywhere in the paper, and none was found in an internet search.