Student-generated multiple-choice questions enhance deeper learning in dental materials education: a randomized crossover trial

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomization was at the individual student level (crossover), not at the class (or higher) level, so ERCT class-level randomization is not satisfied.
    • "Participants were randomized using computer-generated random sequences in blocks of four, stratified by gender." (p. 3)
    • Relevant Quotes: 1) "A prospective, interventional, randomized crossover trial was conducted at the Department of Conservative Dentistry and Endodontics at the Nair Hospital Dental College, Mumbai, India, between July and August 2025." (p. 3) 2) "Participants were randomized using computer-generated random sequences in blocks of four, stratified by gender." (p. 3) 3) "Following the didactic lectures, all students completed the pre-tests and were randomly allocated to the intervention (MCQ generation activity with faculty-moderated discussion) or control group (additional study time)." (p. 1) Detailed Analysis: Criterion C requires that the unit of randomization be the class (or stronger, e.g., school/site), unless the intervention is inherently one-to-one tutoring/personal teaching (in which case student-level randomization can be acceptable). This study is explicitly a "randomized crossover trial" conducted within a single cohort, and the randomization description refers to "Participants" being randomized (i.e., individual students). The intervention is a structured group learning activity (MCQ generation plus faculty-moderated discussion), not one-to-one tutoring. Because randomization is not at the class level (or higher), the ERCT class-level randomization requirement is not met. Final summary sentence: Criterion C is not met because the unit of randomization is individual students (crossover) rather than classes or schools, and the tutoring exception does not apply.
  • E

    Exam-based Assessment

    • Outcomes were measured using study-developed MCQs (even though validated) rather than a widely recognized standardized exam.
    • "The pre- and post-test MCQs were developed by a panel of three dental materials experts, each with at least 10 years of teaching experience." (p. 4)
    • Relevant Quotes: 1) "Following the lecture, all participants completed a pre-test comprising 30 validated MCQs covering various cognitive domains according to Bloom’s taxonomy." (p. 3) 2) "One week after the initial lecture, all participants completed a post-test comprising 30 validated MCQs of comparable difficulty to the pre-test, but with different questions." (p. 3) 3) "The pre- and post-test MCQs were developed by a panel of three dental materials experts, each with at least 10 years of teaching experience." (p. 4) Detailed Analysis: Criterion E requires a standardized, widely recognized exam-based assessment (e.g., national/state standardized tests or other broadly used external standardized exams), rather than an assessment instrument created specifically for the study. The paper clearly states that the outcome tests were "validated MCQs" and then specifies that these MCQs were developed by an expert panel for this study (with validation steps). This is a bespoke assessment instrument aligned to the course topics, not an external standardized exam. Final summary sentence: Criterion E is not met because the study uses study-developed (though validated) MCQs rather than a recognized standardized exam.
  • T

    Term Duration

    • The primary outcome was measured one week after instruction, which is far shorter than a full academic term.
    • "One week after the initial lecture, all participants completed a post-test..." (p. 3)
    • Relevant Quotes: 1) "One week after the initial lecture, all participants completed a post-test comprising 30 validated MCQs of comparable difficulty to the pre-test, but with different questions." (p. 3) 2) "This study assessed only immediate post-test performance conducted one week after instruction." (p. 9) Detailed Analysis: Criterion T requires that outcomes be measured at least one full academic term after the intervention begins (typically about 3 to 4 months), or at minimum that follow-up extends at least a term even if the intervention itself is brief. Here, the post-test is explicitly administered "One week after the initial lecture," and the limitations section explicitly frames the study as assessing "only immediate post-test performance." This is substantially shorter than one academic term. Final summary sentence: Criterion T is not met because outcomes were measured after about one week rather than at least one academic term after intervention start.
  • D

    Documented Control Group

    • The control condition and baseline/demographic information are reported, enabling meaningful comparison.
    • "The students were randomly allocated to two groups: • Control Group (n=32): Received additional study time with standard learning materials." (p. 3)
    • Relevant Quotes: 1) "The students were randomly allocated to two groups: • Intervention Group (n=32): Participated in MCQ-generation activity. • Control Group (n=32): Received additional study time with standard learning materials." (p. 3) 2) "Participants included 38 females (61.3%) and 24 males (38.7%), with a mean age of 19.6±1.2 years." (p. 5) 3) "Mean pre-test scores demonstrated no significant difference between the intervention group (approximately 52.4% ± 8.7%) and the control group (approximately 53.1% ± 9.2%) (P=0.745), confirming comparable baseline knowledge." (p. 5) Detailed Analysis: Criterion D requires that the control group be documented well enough to support interpretation and comparison, including what the control received and baseline characteristics/performance. The paper explicitly defines what the control group received ("additional study time with standard learning materials") and reports group sizes at allocation. It also reports participant demographics and provides baseline comparability evidence via pre-test score comparison showing no significant difference. Final summary sentence: Criterion D is met because the control condition is described and baseline demographics/performance are reported for comparison.

• Level 2 Criteria

  • S

    School-level RCT

    • This was not a school-/site-level randomized trial; it was conducted in one institution and randomized individual students.
    • "A prospective, interventional, randomized crossover trial was conducted at the Department of Conservative Dentistry and Endodontics at the Nair Hospital Dental College, Mumbai, India..." (p. 3)
    • Relevant Quotes: 1) "A prospective, interventional, randomized crossover trial was conducted at the Department of Conservative Dentistry and Endodontics at the Nair Hospital Dental College, Mumbai, India, between July and August 2025." (p. 3) 2) "Participants were randomized using computer-generated random sequences in blocks of four, stratified by gender." (p. 3) Detailed Analysis: Criterion S requires randomization at the school/site level (i.e., whole schools or comparable sites are assigned to intervention vs control). The study describes a single setting (one dental college department) and describes randomization of participants (students) rather than randomization of schools/sites. Final summary sentence: Criterion S is not met because the study does not randomize schools/sites, only individual students within one institution.
  • I

    Independent Conduct

    • The paper does not document independent third-party conduct of the trial’s implementation and evaluation.
    • "PG conceived the study, participated in its design and coordination, and helped draft the manuscript." (p. 11)
    • Relevant Quotes: 1) "The pre- and post-test MCQs were developed by a panel of three dental materials experts, each with at least 10 years of teaching experience." (p. 4) 2) "Content validity was established through review by two external subject matter experts." (p. 4) 3) "PG conceived the study, participated in its design and coordination, and helped draft the manuscript." (p. 11) 4) "KS participated in the study design, performed statistical analysis, and drafted the manuscript." (p. 11) Detailed Analysis: Criterion I requires that the conduct of the evaluation be independent of the intervention designers (or otherwise have clear third-party independence in implementation and/or evaluation) to reduce bias. The authors’ contributions state that the author team conceived, designed/coordinated, collected data, and analyzed data. The only explicitly external involvement is limited to "review" for content validity of the MCQs, which does not constitute independent trial conduct, implementation, data collection, or independent analysis oversight. Final summary sentence: Criterion I is not met because the paper does not document independent third-party conduct of trial implementation and evaluation.
  • Y

    Year Duration

    • Outcomes were measured on the scale of a week, not at least 75% of an academic year (and T is also not met).
    • "This study assessed only immediate post-test performance conducted one week after instruction." (p. 9)
    • Relevant Quotes: 1) "One week after the initial lecture, all participants completed a post-test comprising 30 validated MCQs of comparable difficulty to the pre-test, but with different questions." (p. 3) 2) "This study assessed only immediate post-test performance conducted one week after instruction." (p. 9) Detailed Analysis: Criterion Y requires outcomes to be measured at least 75% of an academic year after the intervention begins. This study’s post-test is conducted one week after instruction, and the paper explicitly frames the measurement as "only immediate post-test performance." This does not approach year-long tracking, and it also fails the term-duration prerequisite in practice. Final summary sentence: Criterion Y is not met because the study measures outcomes after about one week, far shorter than 75% of an academic year.
  • B

    Balanced Control Group

    • The control group received additional study time, and the additional faculty-moderated discussion/support is an explicit and integral part of the intervention package being tested.
    • "The intervention group received... faculty support during a 120-minute session." (p. 3)
    • Relevant Quotes: 1) "The students were randomly allocated to two groups: • Intervention Group (n=32): Participated in MCQ-generation activity. • Control Group (n=32): Received additional study time with standard learning materials." (p. 3) 2) "They were provided with reference materials and faculty support during a 120-minute session." (p. 3) 3) "A faculty-moderated group discussion was conducted for the intervention group to review, critique, and refine student-generated MCQs." (p. 3) Detailed Analysis: Criterion B compares the nature, quantity, and quality of resources (time, materials, adult/faculty support) provided to intervention and control conditions, and asks whether the control condition provides a comparable substitute for the intervention’s inputs, unless the additional resources are explicitly integral to the treatment being tested. Extra resources are present in the intervention condition (faculty support, faculty-moderated discussion, structured templates, and a specified "120-minute session"). The control condition is not "no-treatment"; it is explicitly described as "additional study time with standard learning materials," which indicates an attempt to create an active comparison based on extra time-on-task via self-study. The paper does not explicitly state the exact duration of the control group’s "additional study time." However, the intervention being evaluated is explicitly defined as the full package of MCQ-generation activity plus faculty-moderated discussion/support. Since this additional support is not an incidental add-on but part of the intervention definition, the resource difference is best interpreted as integral to the treatment contrast rather than an unintended confound. Final summary sentence: Criterion B is met because the control condition includes additional study time and the extra faculty inputs are explicitly integral to the intervention package being tested (MCQ generation with faculty moderation).

• Level 3 Criteria

  • R

    Reproduced

    • No independent, peer-reviewed replication of this specific intervention/trial was identified as of the ERCT check date.
    • "This single-institution study with a moderate sample size may limit generalizability." (p. 9)
    • Relevant Quotes: 1) "This single-institution study with a moderate sample size may limit generalizability." (p. 9) 2) "Multi-institutional investigations across diverse educational settings, class sizes, and student populations would strengthen external validity and clarify whether findings are context dependent." (p. 9) Detailed Analysis: Criterion R requires an independent replication by a different research team in a different context, published in a peer-reviewed outlet, that is clearly a reproduction of this study’s central experimental claim and method. The paper itself describes this work as a single-institution study and explicitly calls for multi-institutional investigations, which indicates replication is not already established within the article. An internet search (performed for this ERCT check on 2026-03-03) for independent replications of this specific trial/intervention package (student-generated MCQ creation with faculty-moderated discussion for dental materials education, randomized crossover design) did not identify a clearly labeled, peer-reviewed independent replication. Final summary sentence: Criterion R is not met because independent replication evidence for this specific study was not found.
  • A

    All-subject Exams

    • Because the study does not use standardized exams (E not met), it cannot meet the all-subject standardized exam requirement.
    • "Following the lecture, all participants completed a pre-test comprising 30 validated MCQs..." (p. 3)
    • Relevant Quotes: 1) "Following the lecture, all participants completed a pre-test comprising 30 validated MCQs covering various cognitive domains according to Bloom’s taxonomy." (p. 3) 2) "The pre- and post-test MCQs were developed by a panel of three dental materials experts, each with at least 10 years of teaching experience." (p. 4) Detailed Analysis: Criterion A requires assessment across all main subjects using standardized exam-based assessments, and ERCT rules specify that if Criterion E is not met, Criterion A is not met. This study uses study-developed MCQs (not standardized external exams) and focuses on dental materials topics rather than all core subjects; therefore it cannot meet the all-subject standardized exam requirement. Final summary sentence: Criterion A is not met because the study does not use standardized exams (E not met), so it cannot satisfy the all-subject standardized exam criterion.
  • G

    Graduation Tracking

    • The study measures only immediate outcomes and does not track participants to graduation (and Y is not met).
    • "This study assessed only immediate post-test performance conducted one week after instruction." (p. 9)
    • Relevant Quotes: 1) "This study assessed only immediate post-test performance conducted one week after instruction." (p. 9) 2) "We did not evaluate long-term knowledge retention at 3, 6, or 12 months." (p. 9) Detailed Analysis: Criterion G requires follow-up tracking until graduation. The paper explicitly states it assessed only immediate post-test performance one week after instruction and did not evaluate longer-term retention at 3, 6, or 12 months, let alone outcomes at graduation. Per ERCT rules, if Criterion Y (year duration) is not met, then Criterion G cannot be met; this study is far shorter than a year. An internet search for follow-up publications by the same author team tracking this cohort to graduation did not identify any such follow-up paper as of 2026-03-03. Final summary sentence: Criterion G is not met because outcomes were assessed only immediately/short-term and there is no evidence of tracking to graduation (and Y is not met).
  • P

    Pre-Registered

    • A registry ID and registration date are provided, but it cannot be verified from the paper (or accessible registry data) that registration occurred before the first data collection/enrollment.
    • "The study was registered with the Clinical Trial Registry of India (CTRI) with the registration number- CTRI/2025/07/091029 dated 16th July 2025." (p. 3)
    • Relevant Quotes: 1) "A prospective, interventional, randomized crossover trial was conducted... between July and August 2025." (p. 3) 2) "The study was registered with the Clinical Trial Registry of India (CTRI) with the registration number- CTRI/2025/07/091029 dated 16th July 2025." (p. 3) Detailed Analysis: Criterion P requires that the study protocol be pre-registered before the study begins (i.e., before first enrollment/data collection). The paper provides a registry name, registration number, and registration date (16th July 2025). However, the paper describes the study as occurring "between July and August 2025" without specifying a precise first enrollment date, pre-test date, or the first intervention date. Therefore, from the paper alone, it is not possible to confirm that registration occurred before the first participant was enrolled or before data collection began. During this ERCT check, attempts to validate the CTRI record using public registry search interfaces were not successful due to access constraints (e.g., CAPTCHA-based search), so the timing could not be independently confirmed from the registry entry. Final summary sentence: Criterion P is not met because prospective registration (before study start) cannot be verified from the paper’s dated information and could not be independently confirmed from accessible registry data during this check.