Effects of Adjunct Questions on L2 Reading Comprehension with Texts of Different Types

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Participants were randomly assigned as individual students drawn from mixed majors into six conditions, not as intact classes or schools, and this is not a one-to-one tutoring intervention.
    • "They were randomly assigned into six different groups. Each group consisted of 25-30 students majoring in chemistry, communication engineering, biology, material engineering, medicine, etc." (p. 5)
    • Relevant Quotes: 1) "One hundred and forty-four intermediate-level Chinese EFL learners participated in this study and were divided randomly into six groups." (Abstract) 2) "They were randomly assigned into six different groups. Each group consisted of 25-30 students majoring in chemistry, communication engineering, biology, material engineering, medicine, etc." (p. 5) 3) "This experiment was a 2 x 3 between-group design. Question and text types were independent variables... We randomly grouped the final 144 participants into one of the six conditions." (p. 6) 4) "During their English classroom learning sessions, their teachers gave them instructions on how to finish the reading tasks. After that, each participant received a set of corresponding materials..." (p. 7) Detailed Analysis: The unit of randomisation here is the individual student. Participants were recruited as a convenience sample spanning many different majors (chemistry, communication engineering, biology, material engineering, medicine, etc.) and were individually allocated to one of six experimental conditions. There is no description of whole intact classes or schools being assigned as a block to a single condition; rather, students appear to have been mixed across conditions even while attending "classroom learning sessions," raising the possibility that students from the same class ended up in different conditions (contamination risk). The intervention (embedded adjunct questions inserted into a reading passage) is a one-time group-administered experimental manipulation, not a personal one-to-one tutoring intervention, so the tutoring exception described in the ERCT standard does not apply. Criterion C is not met because randomisation occurred at the individual-student level rather than the class or school level, and no tutoring exception applies.
  • E

    Exam-based Assessment

    • The outcome measures (ten custom multiple-choice questions per text and a written-recall/pausal-unit scoring scheme) were researcher-designed for this study, not a widely recognised standardised exam.
    • "Both texts had ten multiple-choice questions written in English." (p. 7)
    • Relevant Quotes: 1) "The materials used for the pre-test were taken from TEM4 (Test for English Majors-level 4), a large-scale standardized test in China that evaluates the language proficiency level of English majors in China. The pre-test consisted of four reading passages with a total of 30 multiple-choice questions on reading comprehension." (p. 6) 2) "Both texts had ten multiple-choice questions written in English." (p. 7) 3) "To measure participants' scores in the written recall task, we calculated the number of pausal units participants could recall from the reading texts... There were 33 pausal units in text 1 and 29 in text 2." (p. 7) 4) "The development of the embedded questions for text 2 was under discussion among a group of experienced language teachers in the university... However, the inserted questions for text 1 were adopted from previous researchers [5]." (p. 6) Detailed Analysis: TEM4 is used only as a screening/pre-test instrument to confirm participants' baseline reading proficiency, not as the outcome (post-intervention) assessment. The actual outcome measures used to evaluate the effect of the adjunct questions are (a) ten multiple-choice questions per text, and (b) a written-recall task scored by counting recalled "pausal units," both of which were developed or assembled specifically by the research team for this study (the embedded adjunct questions themselves were also either adapted from a prior study or developed in-house by university teachers, but these are the independent variable, not the outcome test). Neither the ten-item MC test nor the pausal-unit written-recall scoring is a widely recognised, standardised examination. Criterion E is not met because the primary outcome instruments are custom, researcher-assembled measures rather than standardised exams.
  • T

    Term Duration

    • The entire experiment, from reading the passage to completing all assessment tasks, was administered in a single session and completed within 45 minutes, far short of one academic term.
    • "Upon completion, we noted that all participants finished within 45 min." (p. 7)
    • Relevant Quotes: 1) "The materials were arranged in the following order: (1) reading passage; (2) topic familiarity questionnaire; (3) written recall task; (4) multiple-choice questions." (p. 7) 2) "Participants were told not to read back during the experiment and were given enough time to complete all the tasks." (p. 7) 3) "Upon completion, we noted that all participants finished within 45 min." (p. 7) 4) "In the eighth week of the fall semester of 2023, a reading test was given to about 200 recruited participants..." (p. 7) (this refers to the separate screening/pre-test, not the main intervention.) Detailed Analysis: The intervention (reading a single passage with or without embedded adjunct questions) and all outcome measurements (written recall and multiple-choice test) occurred back-to-back within one 45-minute sitting. There is no indication of any delay between the "intervention" (reading with embedded questions) and the measurement of outcomes; they occur in the same session. This is vastly shorter than the minimum one-term (roughly 3-4 month) interval required by the standard. Criterion T is not met because intervention and outcome measurement occurred in the same 45-minute session.
  • D

    Documented Control Group

    • The control ("no question") group's size, composition, and procedure are clearly documented, and baseline reading competence was confirmed to be equivalent across all six randomly assigned groups.
    • "Three groups dealt with expository text, with 21 participants for no question condition... The number of participants who dealt with narrative text was 26... in the same condition order as the expository text." (p. 6)
    • Relevant Quotes: 1) "After taking a reading comprehension test, only about 153 participants who showed no significant differences in their reading competence joined the formal study (F(5, 138) = 0.827; p = 0.533)." (p. 5) 2) "Three groups dealt with expository text, with 21 participants for no question condition, 22 for the what questions condition, and 22 for the why questions condition. The number of participants who dealt with narrative text was 26, 28, and 25, respectively, in the same condition order as the expository text." (p. 6) 3) "no embedded question groups (Groups 1 and 4), embedded what question groups (Groups 2 and 5), and embedded why question groups (Groups 3 and 6)." (p. 7) 4) Table 2 reports Mean, SD and N for the "Control" condition separately for both text types (multiple choice and written recall scores). Detailed Analysis: The paper documents the exact size of the no-question (control) condition for each text (n = 21 for the expository text, n = 26 for the narrative text) and describes precisely what this group did: they read the same passage as the other groups but without any embedded questions, then completed the same questionnaire, written-recall task, and multiple-choice test. Because all 144 participants were drawn from the same screened, demographically homogeneous pool (average age 19.5, 80% male/20% female, all intermediate CET4-level EFL learners) and were randomly assigned to conditions, and because a pre-test confirmed no baseline differences in reading competence across the six groups (F(5, 138) = 0.827, p = 0.533), the control group's characteristics and conditions are adequately documented for comparison. Criterion D is met because the control condition's size, procedure, and baseline equivalence are clearly documented.

• Level 2 Criteria

  • S

    School-level RCT

    • Randomisation occurred among individual students at a single institution rather than among schools.
    • "They were randomly assigned into six different groups." (p. 5)
    • Relevant Quotes: 1) "About two hundred convenient samples were recruited... They were randomly assigned into six different groups." (p. 5) 2) "This experiment was a 2 x 3 between-group design... We randomly grouped the final 144 participants into one of the six conditions." (p. 6) Detailed Analysis: All participants were students recruited from a single university setting (implicitly, given the description of majors such as chemistry, communication engineering, etc., which reads as one technology-focused institution) and were individually randomised to conditions. There is no mention of multiple schools or institutions being randomised as whole units. Criterion S is not met because there is no evidence of school-level randomisation.
  • I

    Independent Conduct

    • The same author team designed the materials, ran the experiment, and performed the analysis; no independent third-party evaluator is described.
    • "Conceptualization, Y.S.; methodology, Y.S., W.Z. and S.T.; software, Y.S.; validation, Y.S., W.Z. and S.T.; formal analysis, Y.S. and S.T.; investigation, Y.S., W.Z. and S.T....; supervision, Y.S.; project administration, Y.S...." (p. 12)
    • Relevant Quotes: 1) "Conceptualization, Y.S.; methodology, Y.S., W.Z. and S.T.; software, Y.S.; validation, Y.S., W.Z. and S.T.; formal analysis, Y.S. and S.T.; investigation, Y.S., W.Z. and S.T.; resources, Y.S.; data curation, Y.S.; writing-original draft preparation, Y.S., W.Z. and S.T.; writing-review and editing, Y.S., W.Z. and S.T.; visualization, Y.S.; supervision, Y.S.; project administration, Y.S.; funding acquisition, Y.S." (p. 12) 2) "The development of the embedded questions for text 2 was under discussion among a group of experienced language teachers in the university..." (p. 6) 3) "Two native English speakers were invited to read the two texts out loud and marked the pausal units in each text." (p. 7) (this concerns scoring rubric creation, not independent conduct of the trial as a whole.) Detailed Analysis: The author contribution statement shows that the same three authors performed every stage of the study, including conceptualisation, methodology, materials design, data collection ("investigation"), formal analysis, and manuscript writing. There is no mention of an external, independent research team or agency conducting or overseeing the data collection or analysis. The "experienced language teachers" who helped write embedded questions for text 2 are University colleagues, not an independent evaluation body, and the two native-English raters were used only to mark pausal units for scoring, not to independently run or evaluate the trial. Criterion I is not met because the study was designed, conducted, and analysed entirely by the same research team.
  • Y

    Year Duration

    • Since the Term Duration (T) criterion is not met (the study was a single 45-minute session), the stronger Year Duration criterion is automatically not met.
    • "Upon completion, we noted that all participants finished within 45 min." (p. 7)
    • Relevant Quotes: 1) "Upon completion, we noted that all participants finished within 45 min." (p. 7) Detailed Analysis: Per the ERCT standard, if criterion T (Term Duration) is not met, criterion Y is automatically not met. Since the entire study, including intervention and outcome measurement, occurred within a single 45-minute session, it falls dramatically short of the 75% of an academic year required by criterion Y. Criterion Y is not met because criterion T is not met and the study duration was a single short session.
  • B

    Balanced Control Group

    • All six groups underwent an identical procedure and time allotment (single 45-minute session, same questionnaire, written recall, and multiple-choice test); the only difference between groups was the presence/type of embedded questions in the text, which is the core variable under investigation, so no extra, unmatched time or budget was given to any condition.
    • "The materials were arranged in the following order: (1) reading passage; (2) topic familiarity questionnaire; (3) written recall task; (4) multiple-choice questions." (p. 7)
    • Relevant Quotes: 1) "The materials were arranged in the following order: (1) reading passage; (2) topic familiarity questionnaire; (3) written recall task; (4) multiple-choice questions." (p. 7) 2) "Three groups dealt with expository text, with 21 participants for no question condition, 22 for the what questions condition, and 22 for the why questions condition." (p. 6) 3) "Upon completion, we noted that all participants finished within 45 min." (p. 7) 4) "Two parallel questions were inserted into each of the two texts for the experimental groups; one was put in the middle of the text, while the other was placed at the end." (p. 6) Detailed Analysis: Applying the criterion B decision tree: the first check is whether the intervention adds extra time or budget relative to control. Here, all six groups (no-question, what-question, why-question, for each of two text types) received the same reading passage, the same topic-familiarity questionnaire, the same written-recall task, and the same multiple-choice test, all within the same single sitting reported to take about 45 minutes in total. The only difference across conditions is the presence and type of two short embedded questions inserted into the passage itself, which is the independent variable the study is designed to test, not a separate/optional add-on resource such as extra tutoring time, materials, or budget provided only to the treatment groups. Since no extra time, budget, or materials were introduced for any condition, this criterion is trivially satisfied under the first branch of the decision tree, without needing to invoke the "resources are the treatment variable" exception. Criterion B is met because no group received additional unmatched time, budget, or materials; the only difference between conditions is the embedded-question manipulation itself, while time, materials, and procedure were otherwise identical for all groups.

• Level 3 Criteria

  • R

    Reproduced

    • No independent replication of this specific study by a different research team was found in the paper or in an internet search of works that cite it.
    • Relevant Quotes: No quotes describing a replication of this specific study (Sun, Zhou & Tang, 2024, on adjunct questions with expository vs. narrative L2 texts in Chinese EFL college learners) were found in the paper itself. Internet Search Results (Semantic Scholar, OpenAlex, Crossref, checked July 2026): Citation databases show three works citing this paper: Alreshidi & Alrashidi (2026), "Investigating the role of a higher-order adjunct question package in supporting learners' germane cognitive load"; Tang, Song & Matsumi (2025), "The impact of vividness of visual imagery on the construction of multi-dimensional situation models by second language learners"; and Msaddek (2025), "Uncovering the Use and Transfer of Cognitive and Metacognitive Reading Strategies Across Text Types (Narrative & Expository Reading Texts) in English (L3) Among Moroccan EFL University Learners." None of these three citing works attempts to reproduce the specific 2 x 3 between-subjects design (text type x question type) with Chinese intermediate EFL learners reported in this paper; they investigate related but distinct questions (cognitive load in digital learning, situation-model construction in L2 Japanese, and strategy transfer among Moroccan EFL learners), and none share authorship with the original study. Detailed Analysis: Criterion R requires that the specific study be independently replicated by a different research team in a different context, published in a peer-reviewed outlet. No such replication was found either within the paper's own references or among the works that have since cited it. Criterion R is not met because no independent replication of this specific study was found.
  • A

    All-subject Exams

    • Since criterion E (Exam-based Assessment) is not met, criterion A is automatically not met; in addition, only English reading comprehension was assessed, not all main subjects.
    • "Both texts had ten multiple-choice questions written in English." (p. 7)
    • Relevant Quotes: 1) "Both texts had ten multiple-choice questions written in English." (p. 7) 2) "To measure participants' scores in the written recall task, we calculated the number of pausal units participants could recall from the reading texts." (p. 7) Detailed Analysis: Per the ERCT standard's dependency rule, if criterion E is not met, criterion A cannot be met either. In addition, on its own merits the study measures only English-language reading comprehension (via multiple-choice and written recall on two short passages); no other subjects (e.g., mathematics, science) were assessed, and no exception rationale for a specialised, single-subject focus was provided. Criterion A is not met because criterion E is not met and only a single subject (L2 reading) was assessed.
  • G

    Graduation Tracking

    • Since criterion Y (Year Duration) is not met, criterion G is automatically not met; no follow-up or graduation-tracking publication by the same authors was found either.
    • "Upon completion, we noted that all participants finished within 45 min." (p. 7)
    • Relevant Quotes: 1) "Upon completion, we noted that all participants finished within 45 min." (p. 7) 2) "Future studies could center around investigating other types of adjunct responses, other test types, the effect of topic familiarity, and using other methods... to explore learners' cognitive processes." (p. 12) (no mention of any planned or completed long-term follow-up of this specific cohort.) Internet Search Results (Semantic Scholar, OpenAlex, Crossref, checked July 2026): A search for follow-up publications by the same authors (Yunmei Sun, Wenhui Zhou, Shifang Tang) tracking the same cohort of 144 Chinese EFL college participants found none. The citation record for this 2024 article shows only three citing works, all by unrelated author teams (Alreshidi & Alrashidi 2026; Tang, Song & Matsumi 2025; Msaddek 2025), none of which report longer-term or graduation tracking of this study's participants. Detailed Analysis: Per the ERCT standard's dependency rule, since criterion Y is not met, criterion G is automatically not met. The study is also a single-session laboratory-style reading experiment with no cohort re-contact mechanism described, and no follow-up publication tracking these participants toward graduation was found. Criterion G is not met because criterion Y is not met and no follow-up or graduation-tracking publication was found.
  • P

    Pre-Registered

    • The paper reports institutional ethics approval but no pre-registration of the study's hypotheses, methods, or analysis plan on a public registry prior to data collection, and no registry entry was found on internet search.
    • "This study was conducted in accordance with the Declaration of Helsinki, and approved by the Ethics Committee of Huazhong University of Science & Technology (protocol code 20231218 and date of approval 2 September 2023)." (p. 12)
    • Relevant Quotes: 1) "This study was conducted in accordance with the Declaration of Helsinki, and approved by the Ethics Committee of Huazhong University of Science & Technology (protocol code 20231218 and date of approval 2 September 2023)." (p. 12) 2) No statement anywhere in the paper references a trial registry (e.g., OSF, AsPredicted, ClinicalTrials.gov, ISRCTN) or a pre-registered hypotheses/analysis plan. Detailed Analysis: The only prospective documentation mentioned is Institutional Review Board (ethics) approval, which concerns participant protection, not pre-registration of the study's hypotheses, design, and planned analyses on a public registry. No registry link, ID, or registration date is provided anywhere in the paper, so there was no specific registry entry to verify; a general check of the citation record and publisher metadata likewise surfaced no pre-registration reference. Criterion P is not met because no pre-registration of the study protocol is reported or found.