Student-generated multiple-choice questions enhance deeper learning in dental materials education: a randomized crossover trial

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomization occurred at the individual student level (not by class or school), and the intervention is not one-to-one tutoring, so ERCT class- level randomization is not satisfied.
    • "Participants were randomized using computer-generated random sequences in blocks of four, stratified by gender."
    • Relevant Quotes: 1) "A prospective, interventional, randomized crossover trial was conducted at the Department of Conservative Dentistry and Endodontics at the Nair Hospital Dental College, Mumbai, India, between July and August 2025." (p. 3) 2) "Participants were randomized using computer-generated random sequences in blocks of four, stratified by gender." (p. 3) 3) "The students were randomly allocated to two groups:" (p. 3) Detailed Analysis: Criterion C requires that the RCT randomize at the class level (or stronger, e.g., school/site), to reduce contamination between students in the same instructional environment. In this study, participants were randomized as individuals within a single institution/cohort (and then crossed over), rather than randomizing whole classes or schools. The intervention is a group-based educational activity (MCQ generation with faculty-moderated discussion), not one-to-one tutoring/personal teaching; therefore, the tutoring exception does not apply. Criterion C is not met because randomization is at the student level rather than at the class (or school) level, with no tutoring exception.
  • E

    Exam-based Assessment

    • Outcomes were assessed using study-created and validated MCQs rather than a widely recognized standardized exam, so the exam-based assessment requirement is not met.
    • "The pre- and post-test MCQs were developed by a panel of three dental materials experts, each with at least 10 years of teaching experience."
    • Relevant Quotes: 1) "Following the lecture, all participants completed a pre-test comprising 30 validated MCQs covering various cognitive domains according to Bloom’s taxonomy." (p. 3) 2) "One week after the initial lecture, all participants completed a post-test comprising 30 validated MCQs of comparable difficulty to the pre-test, but with different questions." (p. 3) 3) "The pre- and post-test MCQs were developed by a panel of three dental materials experts, each with at least 10 years of teaching experience." (p. 5) Detailed Analysis: Criterion E requires that educational outcomes be measured using a standardized, widely recognized exam (e.g., national/state standardized testing) rather than an assessment created for the study. Here, both the pre-test and post-test are sets of MCQs developed by an expert panel for this study and validated (content review, pilot testing, and psychometrics). While this supports internal measurement quality, it is not a widely recognized external standardized exam. Criterion E is not met because the outcome assessments are study-created validated MCQs rather than a widely recognized standardized exam.
  • T

    Term Duration

    • The primary post-test occurred one week after instruction, which is far shorter than an academic term, so the term-duration requirement is not met.
    • "One week after the initial lecture, all participants completed a post-test..."
    • Relevant Quotes: 1) "A prospective, interventional, randomized crossover trial was conducted ... between July and August 2025." (p. 3) 2) "One week after the initial lecture, all participants completed a post-test comprising 30 validated MCQs of comparable difficulty to the pre-test, but with different questions." (p. 3) Detailed Analysis: Criterion T requires outcomes to be measured at least one full academic term (~3 to 4 months) after the intervention begins. In this study, the post-test is administered "One week after the initial lecture," and the overall study timeframe is "between July and August 2025," indicating a short follow-up window measured in weeks, not months. Criterion T is not met because outcomes were measured one week after instruction rather than at least one academic term later.
  • D

    Documented Control Group

    • The control condition and baseline comparability are described (control received additional study time), with demographic and pre-test evidence of comparable baseline status, so the control group is adequately documented.
    • "Control Group (n=32): Received additional study time with standard learning materials."
    • Relevant Quotes: 1) "Control Group (n=32): Received additional study time with standard learning materials." (p. 3) 2) "Participants included 38 females (61.3%) and 24 males (38.7%), with a mean age of 19.6±1.2 years." (p. 6) 3) "No significant differences in baseline characteristics were observed between the initially assigned groups." (p. 6) 4) "Mean pre-test scores demonstrated no significant difference between the intervention group (approximately 52.4% ± 8.7%) and the control group (approximately 53.1% ± 9.2%) (P=0.745), confirming comparable baseline knowledge." (p. 6) Detailed Analysis: Criterion D requires the control group to be well-documented, including what the control group received and evidence that the groups were comparable at baseline. The paper explicitly states the control condition ("additional study time with standard learning materials"). It also reports demographic information and explicitly states no significant baseline differences, and provides pre-test results showing no significant difference at baseline. Criterion D is met because the control condition and baseline characteristics/performance are clearly reported.

• Level 2 Criteria

  • S

    School-level RCT

    • The study was conducted at a single institution with student-level randomization, not school/site-level randomization across multiple schools, so the school-level RCT requirement is not met.
    • "A prospective, interventional, randomized crossover trial was conducted at the Department of Conservative Dentistry and Endodontics at the Nair Hospital Dental College, Mumbai, India..."
    • Relevant Quotes: 1) "A prospective, interventional, randomized crossover trial was conducted at the Department of Conservative Dentistry and Endodontics at the Nair Hospital Dental College, Mumbai, India..." (p. 3) 2) "Participants were randomized using computer-generated random sequences in blocks of four, stratified by gender." (p. 3) Detailed Analysis: Criterion S requires randomization at the school (site) level, typically involving multiple schools/sites assigned to intervention vs control. This study took place at a single dental college department and describes participant-level randomization (students randomized in blocks), with no indication that multiple schools/sites were randomized. Criterion S is not met because randomization was not conducted at the school/site level.
  • I

    Independent Conduct

    • The authors appear to have designed, conducted, and analyzed the study themselves without documenting an independent evaluation team, so independent conduct is not established.
    • "KS participated in the study design, performed statistical analysis, and drafted the manuscript."
    • Relevant Quotes: 1) "A faculty-moderated group discussion was conducted for the intervention group to review, critique, and refine student-generated MCQs." (p. 3) 2) "Data analysis was performed using SPSS version 25.0 (IBM Corp., Armonk, NY, USA)." (p. 5) 3) "PG conceived the study, participated in its design and coordination, and helped draft the manuscript. KS participated in the study design, performed statistical analysis, and drafted the manuscript. HS participated in data collection and helped draft the manuscript. KSB participated in the study coordination and helped draft the manuscript." (p. 11) Detailed Analysis: Criterion I requires that the evaluation be conducted independently from the intervention designers/implementers to reduce potential bias. The paper describes faculty involvement in delivering/moderating the intervention, and the author contribution statement indicates that the author team designed/coordinated the study, collected data, and performed statistical analysis. The paper does not describe an external/third-party evaluation team conducting implementation, data collection, or analysis independently of the authors. Criterion I is not met because independent trial conduct and analysis are not documented.
  • Y

    Year Duration

    • Outcomes were measured within weeks (and T is not met), which is far short of 75% of an academic year, so the year-duration requirement is not met.
    • "One week after the initial lecture, all participants completed a post-test..."
    • Relevant Quotes: 1) "A prospective, interventional, randomized crossover trial was conducted ... between July and August 2025." (p. 3) 2) "One week after the initial lecture, all participants completed a post-test..." (p. 3) Detailed Analysis: Criterion Y requires outcomes measured at least 75% of one academic year after the intervention begins. This study’s timeframe ("between July and August 2025") and outcome timing ("One week after the initial lecture") indicate a short duration measured in weeks. Additionally, per ERCT rules, if Criterion T is not met then Criterion Y is not met. Criterion T is not met here. Criterion Y is not met because the study measures outcomes over weeks rather than ~one academic year (and T is not met).
  • B

    Balanced Control Group

    • The intervention’s additional structure and faculty-moderated discussion are integral components of the educational method being tested against an equal-time study control, so the resource difference reflects the treatment rather than an unintended imbalance.
    • "Control Group (n=32): Received additional study time with standard learning materials."
    • Relevant Quotes: 1) "Control Group (n=32): Received additional study time with standard learning materials." (p. 3) 2) "They were provided with reference materials and faculty support during a 120-minute session." (p. 3) 3) "A faculty-moderated group discussion was conducted for the intervention group to review, critique, and refine student-generated MCQs." (p. 3) Detailed Analysis: Criterion B compares the nature, quantity, and quality of resources (time, materials, adult support) provided to intervention and control conditions, and asks whether any extra inputs are (a) matched in the control, or (b) explicitly integral to the treatment being tested. Here, the intervention is explicitly defined as an MCQ-generation activity combined with faculty support and a faculty-moderated discussion, while the control is explicitly defined as "additional study time with standard learning materials." The study intent is therefore to test the effect of this structured, moderated MCQ-generation approach versus additional study time. Although the intervention includes structured facilitation (a meaningful input), this facilitation is not an accidental add-on; it is part of the intervention package as defined by the authors. The control condition is framed as time for standard studying (a plausible educational substitute), which matches the common ERCT-acceptable design where the additional resources are the treatment being tested. Criterion B is met because the additional structure and faculty moderation are integral to the defined intervention being tested against a study-time control, rather than an unintended resource imbalance.

• Level 3 Criteria

  • R

    Reproduced

    • The paper does not report an independent replication, and an internet search did not identify a peer-reviewed independent replication of this specific trial as of the ERCT check date.
    • "This single-institution study with a moderate sample size may limit generalizability."
    • Relevant Quotes: 1) "This single-institution study with a moderate sample size may limit generalizability." (p. 9) 2) "Multi-institutional investigations across diverse educational settings, class sizes, and student populations would strengthen external validity and clarify whether findings are context dependent." (p. 9) Detailed Analysis: Criterion R requires an independent replication by a different research team in a different context, published in a peer-reviewed outlet. The paper characterizes itself as a single-institution study and explicitly calls for future multi-institutional investigations, but does not cite or describe any completed independent replication of this specific trial (same intervention package and study design). An internet search (by DOI, full title, and key phrases) did not identify a peer-reviewed independent replication of this specific trial as of 2026-02-22. Criterion R is not met because no independent replication of this specific study was found.
  • A

    All-subject Exams

    • Because standardized exams were not used (E not met), the all-subject standardized exam requirement cannot be satisfied, and the assessments focus only on dental materials topics.
    • "Following the lecture, all participants completed a pre-test comprising 30 validated MCQs..."
    • Relevant Quotes: 1) "Following the lecture, all participants completed a pre-test comprising 30 validated MCQs covering various cognitive domains according to Bloom’s taxonomy." (p. 3) 2) "The study comprised two phases, each examining different dental material topics of equivalent difficulty." (p. 3) Detailed Analysis: Criterion A requires all-subject outcomes assessed via standardized exam- based assessments, and per ERCT rules, if Criterion E is not met then Criterion A is not met. This study uses validated MCQs created for the study (not a standardized external exam), and focuses on dental materials topics rather than all core subjects. Criterion A is not met because Criterion E is not met and the study does not use standardized all-subject exams.
  • G

    Graduation Tracking

    • The study measures short-term outcomes only and does not track students through graduation (and Y is not met), so graduation tracking is not satisfied.
    • "This study assessed only immediate post-test performance conducted one week after instruction."
    • Relevant Quotes: 1) "This study assessed only immediate post-test performance conducted one week after instruction." (p. 9) 2) "We did not evaluate long-term knowledge retention at 3, 6, or 12 months." (p. 9) Detailed Analysis: Criterion G requires tracking participants until graduation. The paper explicitly states that it assessed only immediate post-test performance (one week after instruction) and did not evaluate longer-term retention, which indicates that it did not track learners through graduation. Additionally, per ERCT rules, if Criterion Y is not met then Criterion G is not met. Criterion Y is not met here. An internet search for follow-up publications by the same author group that track this cohort to graduation did not identify any such follow-up papers as of 2026-02-22. Criterion G is not met because outcomes are short-term and participants were not tracked to graduation (and Y is not met).
  • P

    Pre-Registered

    • The paper reports a CTRI registration number and date, but registry timing relative to first enrollment could not be independently verified from the CTRI database in this review, so pre-registration cannot be confirmed.
    • "The study was registered with the Clinical Trial Registry of India (https://ctri.icmr.org.in/) with registration number- CTRI/2025/07/091029 dated 16th July 2025."
    • Relevant Quotes: 1) "The study was registered with the Clinical Trial Registry of India (CTRI) with the registration number- CTRI/2025/07/091029 dated 16th July 2025." (p. 3) 2) "The study was registered with the Clinical Trial Registry of India (https://ctri.icmr.org.in/) with registration number- CTRI/2025/07/091029 dated 16th July 2025." (p. 11) 3) "A prospective, interventional, randomized crossover trial was conducted ... between July and August 2025." (p. 3) Detailed Analysis: Criterion P requires that the full protocol be pre-registered before data collection begins, which typically requires verifying the registry’s registration date and the registry-recorded date of first enrollment (or study start) to confirm the ordering. The paper clearly reports a CTRI identifier and a registration date (16th July 2025). However, in this ERCT check, the CTRI database record for CTRI/2025/07/091029 could not be accessed in a way that allowed independent verification of the date of first enrollment for that specific CTRI number (the CTRI website requires human-verification steps for trial lookup, and the specific public record could not be retrieved here). Given this inability to independently verify registry timing relative to first enrollment, the evidence is insufficient to confirm that the protocol was pre-registered before data collection began. Criterion P is not met because a registry ID/date are reported, but the registry timing relative to first enrollment could not be verified in this review.