The Effect of Teaching Discourse Markers through Flipped Model on Iraqi EFL Learners' Writing Performance

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Three intact classes were each randomly assigned wholesale to one of the three conditions, so randomisation was at the class level, satisfying the criterion.
    • "One of the classes was randomly taken as the first experimental group ... The second intact class was randomly regarded as the second experimental group ... The third group, randomly selected as the control group, received no instruction on discourse markers neither in the classroom nor in a flipped manner." (Design)
    • Relevant Quotes: 1) "first three intact classes offering writing courses at University of Basra were selected. One of the classes was randomly taken as the first experimental group, who received the instruction of discourse markers through flipped model of instruction." (Methodology, Design) 2) "The second intact class was randomly regarded as the second experimental group who received instruction on discourse markers through the usual, traditional manner in the classroom. The third group, randomly selected as the control group, received no instruction on discourse markers neither in the classroom nor in a flipped manner." (Methodology, Design) Detailed Analysis: The study explicitly describes an entire-class assignment procedure: three separate intact classes at University of Basra were each, as whole units, randomly allocated to one of the three conditions (flipped-model discourse-marker instruction, traditional in-class discourse-marker instruction, or no discourse-marker instruction at all). No single class was internally split so that some students received one condition and others in the same class received a different one; each class received a uniform condition. This is precisely the unit-of-randomisation pattern the ERCT standard requires to avoid within-class contamination. Criterion C is met because randomisation occurred at the level of entire intact classes rather than at the individual-student level within a shared classroom.
  • E

    Exam-based Assessment

    • The discourse-marker task and writing essays were researcher-designed instruments scored with a study-specific rubric, not a standardised exam.
    • "a productive writing task requiring the participants to write the appropriate discourse markers in the blanks was designed based on the list of discourse markers offered by Schiffrin (1995)." (Participants)
    • Relevant Quotes: 1) "In order to measure the participants' knowledge of discourse markers prior to the experiment, a productive writing task requiring the participants to write the appropriate discourse markers in the blanks was designed based on the list of discourse markers offered by Schiffrin (1995)." (Participants) 2) "For the pre-test, the participants of the three groups were required to write a cause and effect essay entitled 'What causes air pollution in your country? What are the harms of air pollution?', and a comparison and contrast essay on the topic of 'Living in the city or in the suburbs? merits and demerits?'." (The Writing Pre-test and Post-test) 3) "the overall writing proficiency was measured in terms of grammar, word choice, discourse markers, organization, thematic organization and style based on an appropriate writing rubric/scale." (Data Collection Procedure) Detailed Analysis: Both outcome instruments used in this study were built by the researchers specifically for this project: a fill-in-the-blank discourse-marker task constructed from Schiffrin's (1995) taxonomy, and two researcher-assigned timed essay prompts scored by two raters against a study-specific writing rubric. Neither instrument is a widely recognised, externally validated standardised exam (such as a national curriculum test); both were purpose- built to align closely with the study's own intervention content and aims, which is exactly the bias risk the E criterion is designed to guard against. Criterion E is not met because a custom-made assessment was used rather than a standardised, widely recognised exam.
  • T

    Term Duration

    • The treatment and follow-up post-tests spanned a full 12-week semester, meeting the one-term interval requirement.
    • "The treatment lasted a whole semester (12 weeks, 90 minutes a week). At the end of the semester, the three groups attended the same productive test of discourse markers as well as the writing post-test." (Data Collection Procedure)
    • Relevant Quotes: 1) "The treatment lasted a whole semester (12 weeks, 90 minutes a week). At the end of the semester, the three groups attended the same productive test of discourse markers as well as the writing post-test." (Data Collection Procedure) 2) "this number of participants were selected from among 144 students who enrolled in the writing course in the fall semester, 2023-2024." (Participants) Detailed Analysis: The intervention was delivered continuously across an entire academic semester (12 weeks at 90 minutes per session), and the primary outcome measures (discourse- marker productive test and writing post-test) were collected at the very end of that same semester. A 12-week semester corresponds to a standard academic term (roughly 3 months), so the interval from intervention start to outcome measurement satisfies the minimum term-length requirement of the criterion. Criterion T is met because outcomes were measured at the close of a full academic semester following the start of the intervention.
  • D

    Documented Control Group

    • The control group's size, baseline scores, demographics, and business-as-usual classroom activity are all clearly documented.
    • "the control group did not receive any instruction on discourse markers. They followed the regular traditional pattern of writing classes: the same expository topic ... was presented to them and they were supposed to write about it in the classroom as the writing task." (Data Collection Procedure)
    • Relevant Quotes: 1) "however, the control group did not receive any instruction on discourse markers. They followed the regular traditional pattern of writing classes: the same expository topic, given to the experimental groups, was presented to them and they were supposed to write about it in the classroom as the writing task." (Data Collection Procedure) 2) "30 students made the control group." / control group pre-test descriptive statistics: "Control 30 12.53 1.13 11.00 14.00" (discourse markers pre-test, Table 2) and writing pre-test "Control 30 12.26 1.04 10.00 14.00" (Table 7). 3) "90 advanced Iraqi EFL learners studying at University of Basra, Iraq participated in this experiment ... They were both male and female students at the age range of 20 to 25." (Participants) Detailed Analysis: The paper reports the control group's exact size (n = 30), its baseline performance on both the discourse-marker pre-test and the writing pre-test (Tables 2 and 7), and a clear statement of what the control group actually did during the study period (writing the same expository prompts in class each session, receiving no discourse- marker instruction of any kind). Broader demographic information (institution, course, age range, gender mix) applies to the whole 90-student sample, of which the control group's 30 students form a documented third. Criterion D is met because the control group's composition, baseline scores, and business-as-usual activity are clearly and specifically documented.

• Level 2 Criteria

  • S

    School-level RCT

    • Randomisation occurred among three classes at a single university, not among multiple schools/institutions.
    • "first three intact classes offering writing courses at University of Basra were selected." (Design)
    • Relevant Quotes: 1) "first three intact classes offering writing courses at University of Basra were selected." (Design) 2) "90 advanced Iraqi EFL learners studying at University of Basra, Iraq participated in this experiment." (Participants) Detailed Analysis: The entire study was conducted with three intact classes drawn from a single institution, University of Basra. There is no description of multiple schools, campuses, or independent educational units being recruited and randomly assigned to conditions; the unit of randomisation was the class, not the school. This is a weaker level of randomisation than the school-level RCT required by criterion S. Criterion S is not met because randomisation was limited to classes within one institution rather than across multiple schools.
  • I

    Independent Conduct

    • No independent or third-party evaluator is described; the same research team appears to have designed, delivered, and assessed the study.
    • "the researchers conducted a semi-structured interview with all thirty participants of the first experimental group." (Results)
    • Relevant Quotes: 1) "the researchers conducted a semi-structured interview with all thirty participants of the first experimental group." (Results) 2) "The validity and appropriacy of the items of the questionnaire was ensured by consulting the supervisor of this study as the expert in the field of language teaching." (The Semi-structured Interview) 3) "In all the three groups, during the classroom time, the students were given an expository topic to write about and then, the writing of one of the students was examined and commented by the teacher and all other students every session." (Data Collection Procedure) Detailed Analysis: There is no statement anywhere in the paper indicating that an external or third-party team, independent of the researchers who designed the flipped-model intervention, carried out data collection, delivery, or analysis. On the contrary, the same researchers appear to have designed the materials, conducted or supervised classroom instruction, coordinated the interviews, and analysed the results, with only the study supervisor consulted internally for questionnaire validity. No acknowledgement of an independent evaluation agency or blinded administrators is present. Criterion I is not met because there is no evidence of independent conduct separate from the intervention designers.
  • Y

    Year Duration

    • The study covered only a single 12-week semester, far short of the 75% academic-year threshold required by this criterion.
    • "The treatment lasted a whole semester (12 weeks, 90 minutes a week)." (Data Collection Procedure)
    • Relevant Quotes: 1) "The treatment lasted a whole semester (12 weeks, 90 minutes a week). At the end of the semester, the three groups attended the same productive test of discourse markers as well as the writing post-test." (Data Collection Procedure) Detailed Analysis: The entire intervention and its outcome measurement took place within one 12-week semester. This is well below the 75% of an academic year (roughly 9-10 months) that criterion Y requires; there is no indication of any tracking beyond the end of this single semester. Criterion Y is not met because the study duration (one semester) does not approach the required academic-year length.
  • B

    Balanced Control Group

    • All groups received equal in-class practice time; the flipped group's extra take-home media is the defining, integral mechanism of the treatment being tested.
    • "The learners of the first experimental group received instruction on discourse markers through flipped model of classroom. This means that the major classroom instructional points were sent to the students in the form of a video clip, an audio clip, a piece of film, or a short voiced PowerPoint file ... two or three days before the classroom session." (Data Collection Procedure)
    • Relevant Quotes: 1) "The learners of the first experimental group received instruction on discourse markers through flipped model of classroom. This means that the major classroom instructional points were sent to the students in the form of a video clip, an audio clip, a piece of film, or a short voiced PowerPoint file recorded by the teacher two or three days before the classroom session." (Data Collection Procedure) 2) "The learners of the second experimental group received instruction on discourse markers in the classroom in the traditional manner. Each session, one category of discourse markers was taught to the learners and then the same expository topic, given to the first experimental group, was presented to them." (Data Collection Procedure) 3) "the control group did not receive any instruction on discourse markers. They followed the regular traditional pattern of writing classes: the same expository topic ... was presented to them and they were supposed to write about it in the classroom as the writing task." (Data Collection Procedure) 4) "In all the three groups, during the classroom time, the students were given an expository topic to write about and then, the writing of one of the students was examined and commented by the teacher and all other students every session." (Data Collection Procedure) Detailed Analysis: All three groups received identical in-class time each session (90 minutes/week) devoted to writing the same expository prompts with teacher feedback; the only structural difference across groups is whether, and how, discourse-marker instruction was delivered. The first experimental group additionally received brief pre-session audio/video/PowerPoint clips two to three days beforehand. This extra take-home resource is not a separable, confounding add-on: it is precisely how the "flipped model" treatment is operationally defined in this study, and the three-way comparison (flipped delivery vs. traditional in-class delivery vs. no discourse-marker instruction) is the explicit research question (RQ1/RQ2). Per the standard's decision logic, when the additional resource is itself the treatment variable being tested, the other conditions may remain at their respective baselines without violating balance. The traditional and control groups, which are not the ones defined by the extra resource, are already matched to each other in classroom time and task structure. Criterion B is met because the only additional resource (short take-home media clips for the flipped group) is integral to the flipped-model treatment under investigation, while in-class time and task structure are otherwise equal across all three groups.

• Level 3 Criteria

  • R

    Reproduced

    • No independent replication of this specific 2024 study by another research team was found after an internet search.
    • Relevant Quotes: 1) "Shahhoseini et al. (2023) attempted to examine the effect of flipped model of instruction on some Iranian EFL learners' use of linking words in writing. ... The results confirmed the outperformance of the experimental group who were exposed to the flipped instruction compared with the control group who received instruction in the traditional manner in the classroom." (Literature Review) Detailed Analysis: This paper was published in late 2024 (accepted 06 November 2024), and no subsequent publication reproducing this specific study (same intervention, population, and design, conducted by a different research team) was identified via internet search, including a check of citing works. The literature review cites several related but distinct studies on flipped instruction applied to writing or discourse- related outcomes (e.g., Shahhoseini et al., 2023, on linking words in an Iranian EFL context), but these are separate original studies with their own designs and samples, predating this paper, not replications of it. A search of works citing this article found only unrelated citing papers (a discourse-analysis study and an unrelated article), none of which replicate this study's design in a different context. No evidence of an independent reproduction of this particular study's results exists. Criterion R is not met because no independent replication of this specific study was found.
  • A

    All-subject Exams

    • Only discourse-marker knowledge and writing were assessed, and the prerequisite standardised-exam criterion (E) is not met.
    • "the overall writing proficiency was measured in terms of grammar, word choice, discourse markers, organization, thematic organization and style based on an appropriate writing rubric/scale." (Data Collection Procedure)
    • Relevant Quotes: 1) "the overall writing proficiency was measured in terms of grammar, word choice, discourse markers, organization, thematic organization and style based on an appropriate writing rubric/scale." (Data Collection Procedure) Detailed Analysis: The study measured only discourse-marker knowledge and writing performance; no other core subjects (e.g., mathematics, science) were assessed, nor would that be expected for a language-course intervention. More importantly, per the criterion-specific rule, criterion A can only be met if the underlying assessment satisfies criterion E (standardised exam-based assessment). Since criterion E is not met (custom instruments were used), criterion A is automatically not met as well. Criterion A is not met because the prerequisite standardised exam-based assessment criterion (E) is not satisfied.
  • G

    Graduation Tracking

    • No tracking beyond the end-of-semester post-tests is reported, no follow-up publication was found, and criterion Y (a prerequisite) is not met.
    • Relevant Quotes: 1) "The treatment lasted a whole semester (12 weeks, 90 minutes a week). At the end of the semester, the three groups attended the same productive test of discourse markers as well as the writing post-test." (Data Collection Procedure) Detailed Analysis: The paper describes no follow-up data collection after the end-of-semester post-tests; there is no mention of tracking participants toward course completion or graduation. An internet search for later papers by the same authors (Al Issa, Amirian, Ketabi) tracking this same cohort of 90 University of Basra students toward graduation did not surface any such follow-up publication. Additionally, per the criterion-specific rule, since criterion Y (Year Duration) is not met, criterion G is automatically considered not met as well. Criterion G is not met because no post-intervention follow-up is reported, no follow-up publication was located, and the prerequisite Year Duration criterion is not satisfied.
  • P

    Pre-Registered

    • No mention of a pre-registered protocol, trial registry, or registration date appears anywhere in the paper, and no registry entry was found via internet search.
    • Relevant Quotes: 1) "Acknowledgements: Not applicable. Funding: Not applicable. Ethics Declarations: Competing Interests: No, there are no conflicting interests." (end matter) Detailed Analysis: The paper's end-matter sections (Acknowledgements, Funding, Ethics Declarations) contain no reference to a pre- registered protocol, a trial registry entry, or any registration date preceding data collection. No hypotheses, methods, or analysis plan are described as having been published in advance of the study. An internet check for a pre-registration record associated with this study or its authors also returned no evidence of registration. Criterion P is not met because no evidence of pre-registration is present in the paper or in any external registry.