The effect of task-based language teaching on analytic writing in EFL classrooms

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomisation was performed on individual students pooled from two language institutes, not on intact classes or schools, and no tutoring exception applies.
    • "Then, they placed the participants randomly into two groups as the control group and the experimental group." (p. 6)
    • Relevant Quotes: 1) "To select the participants in this study, the researchers applied availability-sampling procedure from two English language institutes in Shiraz. To this end, they administered Oxford Quick Placement Test to the participants and selected those with a score of one standard deviation above and below the mean. Accordingly, the research sample included 80 EFL learners. Then, they placed the participants randomly into two groups as the control group and the experimental group." (p. 6) 2) "The number of the participants randomly assigned to the control group was 40 EFL learners (20 males and 20 females); similarly, the same number of the participants, 40 EFL learners (20 males and 20 females), were assigned to the treatment group." (p. 6) Detailed Analysis: The ERCT 'C' criterion requires randomisation at the class (or stronger school) level, unless the intervention is a one-to-one tutoring exception. Here, the authors pooled 80 individual learners from two institutes after placement testing and then randomly assigned these individual students to the control or experimental condition. There is no quote indicating that pre-existing intact classes, sections, or schools were the unit of randomisation. The intervention (group-based TBLT writing instruction delivered to roughly 35-40 students per condition) is not a personal tutoring intervention, so the tutoring exception does not apply. Criterion C is not met because randomisation was conducted at the individual-student level rather than at the class or school level, and no tutoring exception is applicable.
  • E

    Exam-based Assessment

    • The pre-test and post-test were the writing sections of the internationally recognised, standardised TOEFL test.
    • "The researchers included the writing sections of two paper-based TOEFL tests as the pre-test and post-test." (p. 6)
    • Relevant Quotes: 1) "The researchers included the writing sections of two paper-based TOEFL tests as the pre-test and post-test. Each writing test included two writing tasks in which the participants received a topic to write a paragraph about. The total score for the writing test was 30." (p. 6) 2) "Before the administration of these two tests, two EFL professors reviewed the tests to remove any problems with the tests and to ensure their validity." (p. 6) Detailed Analysis: The ERCT 'E' criterion requires a widely recognised, standardised exam rather than a custom-built instrument. TOEFL is an internationally recognised, standardised test of English proficiency, and the authors explicitly used its writing sections (rather than a purpose-built writing prompt) as both pre-test and post-test. Although scoring used the Jacobs et al. analytic composition rubric, the underlying test instrument itself is the standard TOEFL writing task, not a custom assessment designed to favour the intervention. Criterion E is met because the outcome measure was drawn from the standardised, widely recognised TOEFL writing test.
  • T

    Term Duration

    • The intervention and the outcome measurement both concluded within eight weeks, well short of one academic term.
    • "The treatment lasted eight weeks, each week two sessions (16 sessions in total)." (p. 7)
    • Relevant Quotes: 1) "The treatment lasted eight weeks, each week two sessions (16 sessions in total). In addition, each session lasted 90 min." (p. 7) 2) "Upon the completion of the treatment, the instructor administered a writing test as the posttest to the participants in both the control group and the treatment group to find out how their writing ability has improved during the instruction..." (p. 7) Detailed Analysis: The ERCT 'T' criterion requires outcomes to be measured at least one full academic term (roughly 3-4 months) after the intervention begins. Here, the entire intervention ran for eight weeks (about two months), and the post-test was administered immediately upon completion of the treatment, with no further delayed follow-up measurement reported. This interval falls short of the minimum one-term requirement. Criterion T is not met because the interval from intervention start to outcome measurement was only about eight weeks, shorter than one academic term.
  • D

    Documented Control Group

    • The control group's size, demographics, and instructional conditions are clearly described and matched in duration to the treatment group.
    • "In contrast, the participants in the control group learned writing skills through the use of conventional methods of teaching writing... The instruction course for the participants in the control group also lasted eight weeks, each week two sessions (16 sessions)." (p. 7)
    • Relevant Quotes: 1) "The number of the participants randomly assigned to the control group was 40 EFL learners (20 males and 20 females)... the participants' age was 15-20 and their native language was Persian." (p. 6) 2) "In contrast, the participants in the control group learned writing skills through the use of conventional methods of teaching writing. For example, the instructor assigned a topic to students and asked each student to write a passage about. Then, the instructor checked the written assignments, gave the students some feedback, and finally, assigned a score. The instruction course for the participants in the control group also lasted eight weeks, each week two sessions (16 sessions)." (p. 7) 3) "Table 3. Descriptive statistics for groups' performance on components of writing samples as the pretest" reporting Control group means, standard deviations, and standard errors for content, organization, vocabulary, language use, and sentence mechanics. (p. 9) Detailed Analysis: The ERCT 'D' criterion requires clear documentation of the control group's demographics, baseline performance, and conditions. The paper specifies the control group's size (40 learners, 20 male/20 female), age range, native language, baseline writing scores (Table 3), and precisely describes the "business as usual" instructional method they received (topic-assign-and-correct) and its matching eight-week duration. Criterion D is met because the control group is documented in detail across demographics, baseline scores, and instructional conditions.

• Level 2 Criteria

  • S

    School-level RCT

    • Randomisation occurred at the individual-student level within institutes, not at the school level.
    • "Then, they placed the participants randomly into two groups as the control group and the experimental group." (p. 6)
    • Relevant Quotes: 1) "To select the participants in this study, the researchers applied availability-sampling procedure from two English language institutes in Shiraz." (p. 6) 2) "Then, they placed the participants randomly into two groups as the control group and the experimental group." (p. 6) Detailed Analysis: The ERCT 'S' criterion requires randomisation at the school (or equivalent institution) level. Here, individual students drawn from two institutes were pooled and then randomly assigned to condition; there is no indication that whole institutes (or schools) were randomised to the intervention or control condition. This is weaker than even class-level randomisation. Criterion S is not met because the study was randomised at the individual-student level, not at the school level.
  • I

    Independent Conduct

    • One of the study's own researchers delivered the intervention as the instructor, with no independent evaluator involved.
    • "During the pre-task phase, the instructor presented the topic and (one of the researchers) encouraged the participants to activate the related schemata and the background knowledge on the topic." (p. 7)
    • Relevant Quotes: 1) "During the pre-task phase, the instructor presented the topic and (one of the researchers) encouraged the participants to activate the related schemata and the background knowledge on the topic. She asked the students to brainstorm ideas and write freely about the assigned task without having concern for form." (p. 7) 2) "In order to answer research questions, the researchers codified and analyzed the collected data by SPSS Software (Version 19)." (p. 7) Detailed Analysis: The ERCT 'I' criterion requires that the study be conducted independently from those who designed the intervention, to avoid bias in delivery, scoring, or analysis. Here, the text explicitly states that one of the researchers acted as the instructor delivering the TBLT intervention, and the same research team also codified and analyzed the data. There is no mention of an external, third-party evaluator or blinded administrator involved in delivering the intervention or scoring that was independent of the design team. Criterion I is not met because the same research team both designed and delivered the intervention and analyzed the resulting data.
  • Y

    Year Duration

    • The intervention and follow-up lasted only eight weeks, far short of 75% of an academic year, and criterion T was not met.
    • "The treatment lasted eight weeks, each week two sessions (16 sessions in total)." (p. 7)
    • Relevant Quotes: 1) "The treatment lasted eight weeks, each week two sessions (16 sessions in total). In addition, each session lasted 90 min." (p. 7) 2) "Upon the completion of the treatment, the instructor administered a writing test as the posttest to the participants in both the control group and the treatment group..." (p. 7) Detailed Analysis: The ERCT 'Y' criterion requires tracking of outcomes for at least 75% of an academic year (roughly 9-10 months), and per the standard's dependency rule, Y cannot be met unless T is met. Here, the entire study, from intervention start to final outcome measurement, spanned only eight weeks - far below even the one-term threshold for criterion T, let alone the year-long threshold for Y. Criterion Y is not met both because the tracked duration (about two months) is far shorter than 75% of an academic year and because criterion T is not met.
  • B

    Balanced Control Group

    • Both groups received an equal amount of instructional time and duration; only the teaching methodology differed.
    • "The treatment lasted eight weeks, each week two sessions (16 sessions in total)... The instruction course for the participants in the control group also lasted eight weeks, each week two sessions (16 sessions)." (p. 7)
    • Relevant Quotes: 1) "The treatment lasted eight weeks, each week two sessions (16 sessions in total). In addition, each session lasted 90 min." (p. 7) 2) "The instruction course for the participants in the control group also lasted eight weeks, each week two sessions (16 sessions)." (p. 7) 3) "The participants in both groups did the test in 20 min." (p. 7) Detailed Analysis: Applying the criterion B decision procedure: the intervention did not add extra time or budget relative to the control group (EXTRA_RESOURCES_ PRESENT is false). Both the experimental (TBLT) group and the control (conventional writing instruction) group received exactly the same number of sessions (16), the same overall duration (eight weeks), and the same post-test conditions (20-minute writing test). The only difference documented is the pedagogical method used within that identical instructional time (task-based cycle versus assign-write-correct), not any additional educational resource, time, or budget granted to one group over the other. Since no extra resources are present at all, the criterion is satisfied at the first branch of the decision tree without needing to consider the treatment- variable or within-subjects branches. Criterion B is met because no extra time or resources were provided to the treatment group; instructional dosage was matched between conditions.

• Level 3 Criteria

  • R

    Reproduced

    • No independent replication of this specific study or its design was found in the paper, through subsequent literature, or via citation-database search conducted during verification.
    • Relevant Quotes: 1) "Confirming the usefulness of TBLT for improving the writing skills of the Iranian EFL learners, the results of this study are also generally in line with the results of the past research concerning the effectiveness of task-based writing. For instance, Birjandi and Malmir (2009) showed the effectiveness of TBLT in the narrative and expository writing of Iranian EFL learners... In the same way, Marashi and Dadari (2012) demonstrated the positive effects of task-based writing on students' writing performance and creativity." (p. 13) Detailed Analysis: The ERCT 'R' criterion requires that this specific study (its particular design, population, and measures) be independently replicated by a different research team in a peer-reviewed outlet. The paper cites several other TBLT-writing studies (Birjandi & Malmir, 2009; Marashi & Dadari, 2012; Ashari Tabar & Alavi, 2013; Payman & Gorjian, 2014) that examine related but distinct interventions, populations, or writing tasks; none of these is presented as a direct replication of this specific TOEFL-writing pre/post design conducted with Shiraz language-institute learners. A citation-database search (Semantic Scholar citing-papers listing for DOI 10.1080/2331186X.2018.1496627) was conducted during verification and returned no paper that independently replicates this specific study; the only citing work by an overlapping author, Kafipour and Jafari (2021) on procrastination and writing errors in medical students, examines a different population and outcome variable rather than reproducing this study's design. No subsequent replication of this exact study by an independent team was identified. Criterion R is not met because no independent replication of this specific study was found either in the paper or via subsequent literature and citation-database search.
  • A

    All-subject Exams

    • The study measured only writing ability; no other subjects or language skills were assessed as outcomes.
    • "The researchers included the writing sections of two paper-based TOEFL tests as the pre-test and post-test." (p. 6)
    • Relevant Quotes: 1) "The researchers included the writing sections of two paper-based TOEFL tests as the pre-test and post-test." (p. 6) 2) "The main objective of the present study is to investigate the impact of task-based language teaching methodology on analytic writing skills of Iranian EFL learners." (p. 5) Detailed Analysis: The ERCT 'A' criterion requires that all main subjects (or, for narrowly scoped interventions, a justified subset) be assessed using standardised exams. Here, the outcome measure is exclusively the writing section of the TOEFL test; no other subjects or even other core language skills (reading, listening, speaking) were assessed as outcomes. While the study intentionally focuses on writing, no explicit rationale is given that this narrow focus is an acceptable exception under the standard (the exception is intended for highly specialised vocational contexts), so the general rule applies. Criterion A is not met because only a single subject/skill (writing) was assessed, with no justified exception provided.
  • G

    Graduation Tracking

    • No follow-up tracking occurred beyond the immediate post-test, and criterion Y (a prerequisite for G) was not met.
    • "Upon the completion of the treatment, the instructor administered a writing test as the posttest to the participants in both the control group and the treatment group..." (p. 7)
    • Relevant Quotes: 1) "Upon the completion of the treatment, the instructor administered a writing test as the posttest to the participants in both the control group and the treatment group to find out how their writing ability has improved during the instruction and if there was any difference between the writing competence of the two groups." (p. 7) 2) No further quotes describe any follow-up beyond this immediate posttest, and no subsequent follow-up publications tracking this same cohort were identified. Detailed Analysis: The ERCT 'G' criterion requires tracking of participants until graduation from the relevant educational stage, and per the standard's dependency rule, G cannot be met unless Y is met. Here, the study measured outcomes only once, immediately after the eight-week treatment ended, with no subsequent tracking through graduation. A citation-database search (Semantic Scholar citing- papers listing) conducted during verification found no follow-up publication by Kafipour, Mahmoudi, or Khojasteh tracking this same cohort of 69/80 EFL learners toward graduation; the one thematically related later paper by an overlapping author (Kafipour & Jafari, 2021, on procrastination and writing errors in medical students) concerns a different, unrelated cohort. Criterion G is not met because there is no evidence of tracking beyond the immediate post-intervention test in this paper or in any identifiable follow-up publication, and because criterion Y is not met.
  • P

    Pre-Registered

    • There is no mention anywhere in the paper of a pre-registered protocol or registry entry, and no registration was located during verification.
    • Relevant Quotes: No quotes referencing a registry platform (e.g. ClinicalTrials.gov, ISRCTN, OSF, AsPredicted) or a pre-registration date were found anywhere in the Methods, Procedure, or Funding/Author sections of the paper. Detailed Analysis: The ERCT 'P' criterion requires the full study protocol, including hypotheses, methods, and planned analyses, to have been publicly pre-registered before data collection began. The paper contains no statement of pre-registration on any registry, nor any registration date or ID. No registry entry for this study (by title, authors, or DOI) was located during verification, consistent with the absence of any pre-registration statement in the text itself. Criterion P is not met because no evidence of pre-registration was found in the paper or through independent verification.