Randomized, controlled study evaluating multi-disciplinary team-based learning (MDTBL) as optimal teaching paradigm for residents, comparing with problem-based learning (PBL) and lecture-based learning (LBL)

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomization was at the individual participant level within one course, not at the class (or stronger) level, and no one-to-one tutoring exception applies.
    • Using computer-generated randomization, course participants were divided into the MDTBL group, PBL group, and LBL group in a 1:1:1 ratio, with each group consisting of ten students with at least one year of clinical experience and fifteen students with less than a year of clinical experience.
    • Relevant Quotes: 1) "This is a prospective randomized, controlled study." (p. 3) 2) "Using computer-generated randomization, course participants were divided into the MDTBL group, PBL group, and LBL group in a 1:1:1 ratio, with each group consisting of ten students with at least one year of clinical experience and fifteen students with less than a year of clinical experience." (p. 3) 3) "Participants were numbered according to the order of enrollment." (p. 3) Detailed Analysis: Criterion C requires random assignment at the class level (or stronger), unless the intervention is explicitly one-to-one tutoring/personal teaching. Here, the paper states that "course participants" (individual students) were divided into MDTBL, PBL, and LBL groups using "computer-generated randomization." This indicates individual-level randomization rather than randomizing intact classes or entire schools/sites. The intervention is not described as one-to-one tutoring; it is delivered as group-based learning models (MDTBL and PBL use group discussions; LBL is lecture-based). Therefore, the tutoring exception does not apply. Final Summary: Criterion C is not met because the unit of randomization is individual students rather than intact classes or schools.
  • E

    Exam-based Assessment

    • Outcomes were measured using exams and questionnaires developed by the course team for this study rather than a widely recognized standardized exam.
    • All questions were developed by the course director together with clinical instructors based on Bloom's Taxonomy and the course learning objectives and current diagnostic and treatment guidelines for pulmonary nodules.
    • Relevant Quotes: 1) "The pre-test, midterm, and final exam were the same across all groups, comprising theoretical questions and clinical case analysis." (p. 3) 2) "The pre-test included sixteen multiple-choice questions, while both the midterm and final exam consisted of 32 multiple-choice questions, including questions repeated from the pre-test." (p. 3) 3) "All questions were developed by the course director together with clinical instructors based on Bloom's Taxonomy and the course learning objectives and current diagnostic and treatment guidelines for pulmonary nodules." (p. 4) 4) "The items were independently reviewed by two senior instructors to ensure clarity, clinical relevance and alignment with Bloom’s taxonomy." (p. 4) Detailed Analysis: Criterion E requires a standardized, widely recognized exam-based assessment (i.e., not created specifically for this study/course). The paper explicitly states that the test items were "developed by the course director together with clinical instructors" and aligned to course objectives and guidelines for pulmonary nodules. This is strong evidence the exams were locally created rather than an external standardized exam. Although items were reviewed by senior instructors, that is internal review and does not constitute use of an external standardized exam. Final Summary: Criterion E is not met because the outcome exams were created by the course team rather than being a recognized standardized exam.
  • T

    Term Duration

    • The paper reports that participants took part in the course from September 15, 2022 through December 31, 2023, which exceeds a full academic term.
    • Participants took part in a training course on the clinical diagnosis and evaluation of pulmonary nodules from September 15, 2022, through December 31, 2023.
    • Relevant Quotes: 1) "A randomized controlled study was designed to recruit 30 medical students with a minimum of one year of clinical experience and 45 with less than one year of clinical experience to take a course on clinical diagnosis and evaluation of pulmonary nodules from September 15, 2022, to December 31, 2023." (p. 1) 2) "Participants took part in a training course on the clinical diagnosis and evaluation of pulmonary nodules from September 15, 2022, through December 31, 2023." (p. 3) 3) "After all six sessions, students, again in groups, immediately complete the final exam, self-assessment questionnaire, and feedback questionnaire at a scheduled time." (p. 3) Detailed Analysis: Criterion T requires that outcomes be measured at least one academic term after the intervention begins (typically ~3–4 months). The paper explicitly states a participation/course window from 2022-09-15 through 2023-12-31. This span is far longer than a term. The paper also explains that the final exam is completed after all six sessions. The paper does not provide specific calendar dates for when the final exam occurred within that window, but it does describe the course as occurring across the stated dates, which supports that the study’s outcome measurement period extends beyond a term from the course start. Final Summary: Criterion T is met because the paper reports the course participation period spanning well beyond one academic term.
  • D

    Documented Control Group

    • The comparison groups are described with group sizes and baseline characteristics (gender, age, clinical experience) reported and compared.
    • Table 1 compares the basic characteristics of students in each group.
    • Relevant Quotes: 1) "Using computer-generated randomization, course participants were divided into the MDTBL group, PBL group, and LBL group in a 1:1:1 ratio, with each group consisting of ten students with at least one year of clinical experience and fifteen students with less than a year of clinical experience." (p. 3) 2) "Each group included 10 students with at least one year of clinical experience and 15 students with less than one year of clinical experience." (p. 5) 3) "Table 1 compares the basic characteristics of students in each group." (p. 5) 4) "There were no statistically significant differences among the three groups in terms of gender, age, or clinical experience (P>0.05)." (p. 5) Detailed Analysis: Criterion D requires that the control/comparison condition(s) be documented in enough detail to evaluate comparability, including baseline characteristics. This is a three-arm RCT (MDTBL vs PBL vs LBL). The LBL arm serves as a plausible "business as usual" comparator, and the paper documents all three arms. The paper provides per-group sizes and explicitly reports baseline variables (gender, age, clinical experience) and states there were no statistically significant baseline differences across groups. This meets the documentation requirement for the comparison/control condition(s). Final Summary: Criterion D is met because the paper reports group sizes and baseline characteristics and compares them across arms.

• Level 2 Criteria

  • S

    School-level RCT

    • Randomization occurred among individual course participants, not among schools or equivalent educational institutions.
    • A total of forty-five medical students from the First Affiliated Hospital of Xi'an Jiaotong University with less than a year of clinical experience and 30 medical students with at least a year of clinical experience specializing in thoracic surgery, respiratory medicine, radiology, oncology, and pathology were voluntarily recruited as course participants.
    • Relevant Quotes: 1) "A total of forty-five medical students from the First Affiliated Hospital of Xi'an Jiaotong University with less than a year of clinical experience and 30 medical students with at least a year of clinical experience specializing in thoracic surgery, respiratory medicine, radiology, oncology, and pathology were voluntarily recruited as course participants." (p. 3) 2) "Using computer-generated randomization, course participants were divided into the MDTBL group, PBL group, and LBL group in a 1:1:1 ratio..." (p. 3) Detailed Analysis: Criterion S requires school-level (or equivalent site-level) randomization (i.e., multiple institutions/sites randomized to conditions). The paper describes recruiting students from a single institution (the First Affiliated Hospital of Xi'an Jiaotong University) and randomizing individual participants into three instructional arms. There is no evidence that multiple schools/programs/sites were randomized. Final Summary: Criterion S is not met because the unit of randomization is individuals within one site rather than schools/sites.
  • I

    Independent Conduct

    • The paper indicates the authors developed MDTBL and designed the assessments, with no documented independent third-party evaluation team.
    • To replicate this approach in medical education, we developed a Multi-Disciplinary Team-Based Learning (MDTBL) model.
    • Relevant Quotes: 1) "To replicate this approach in medical education, we developed a Multi-Disciplinary Team-Based Learning (MDTBL) model." (p. 2) 2) "In the MDTBL group, the teaching team consisted of one instructor and three assistants (one of whom acts as a simulated patient) to supervise the sessions." (p. 3) 3) "All questions were developed by the course director together with clinical instructors..." (p. 4) 4) "Authors’ contributions Conceptualization, validation, visualization and writing—original draft: Runyi Tao and Shan Gao; ... Funding acquisition, project administration, resources and supervision: Rui Gao." (p. 9) Detailed Analysis: Criterion I requires that the evaluation be conducted independently from the intervention designers/providers to reduce bias (e.g., external evaluators responsible for delivery, measurement, and/or analysis). The paper states "we developed" the MDTBL model, which indicates the study authors are also the intervention developers. The teaching and supervision are described as being carried out by instructors/assistants as part of the course. The assessments were created internally by the course director and instructors. The paper does not provide a clear statement that an independent external evaluation team (separate from the intervention developers/course team) conducted data collection, administered assessments, or led the analysis. Final Summary: Criterion I is not met because the paper does not document independent third-party conduct separate from the MDTBL developers/course team.
  • Y

    Year Duration

    • The reported participation/course window (September 15, 2022 to December 31, 2023) exceeds 75% of an academic year.
    • Participants took part in a training course on the clinical diagnosis and evaluation of pulmonary nodules from September 15, 2022, through December 31, 2023.
    • Relevant Quotes: 1) "Participants took part in a training course on the clinical diagnosis and evaluation of pulmonary nodules from September 15, 2022, through December 31, 2023." (p. 3) 2) "Before, during, and after the learning phases, all participants underwent identical theoretical assessments..." (p. 1) Detailed Analysis: Criterion Y requires outcomes to be measured at least 75% of one academic year after the intervention begins. The paper explicitly reports a course participation window spanning from 2022-09-15 to 2023-12-31 (~15.5 months), and it describes assessments occurring before, during, and after the learning phases within that window. Even using a conservative academic-year length of ~9–10 months, this reported window exceeds the 75% threshold. Final Summary: Criterion Y is met because the paper reports a course/assessment window spanning well over 75% of an academic year.
  • B

    Balanced Control Group

    • Core instructional materials, cases, and exams were matched across groups; the additional staffing and simulated patient in MDTBL are integral parts of the MDTBL intervention definition.
    • In the MDTBL group, the teaching team consisted of one instructor and three assistants (one of whom acts as a simulated patient) to supervise the sessions.
    • Relevant Quotes: 1) "The preparatory materials provided to each group included the same diagnostic and treatment guidelines and ten course-related academic articles." (p. 3) 2) "Each group studied identical cases involving twelve patients." (p. 3) 3) "The pre-test, midterm, and final exam were the same across all groups, comprising theoretical questions and clinical case analysis." (p. 3) 4) "In the MDTBL group, the teaching team consisted of one instructor and three assistants (one of whom acts as a simulated patient) to supervise the sessions. In the PBL group, one instructor and two assistants formed the teaching supervision team, while in the LBL, one instructor and two assistants provided supervision." (p. 3) Detailed Analysis: Criterion B compares the nature, quantity, and quality of resources (time, staffing, materials) across conditions, and asks whether any extra resources in the intervention arm are either (a) balanced by the control arm, or (b) clearly integral to the treatment being tested. The paper documents strong balancing on key educational inputs: identical preparatory materials, identical patient cases, and identical exams across MDTBL, PBL, and LBL arms. There is a staffing difference: MDTBL has one more assistant and includes a simulated patient. However, MDTBL is explicitly defined as a model that "includes a simulated patient to reproduce the interactions of MDT" (described in the Background), and the staffing configuration is part of how MDTBL is implemented. In other words, the added staffing/simulated patient is not an incidental add-on; it is integral to the MDTBL treatment package being compared against PBL and LBL. Final Summary: Criterion B is met because core instructional inputs are matched across arms, and the remaining resource differences (simulated patient and staffing) are integral to the MDTBL intervention definition.

• Level 3 Criteria

  • R

    Reproduced

    • No independent replication study of this specific MDTBL trial (or a clearly equivalent MDTBL intervention) by a different team was found.
    • Relevant Quotes: 1) (No statement describing an independent replication of this trial was found in the paper.) Detailed Analysis: Criterion R requires evidence that the study/intervention has been independently replicated by a different research team in a different context and published in a peer-reviewed journal. The paper presents a single randomized controlled study and does not claim to be a replication, nor does it cite an independent replication of this specific MDTBL intervention and evaluation design. An internet search (Springer Nature Link/PMC and web-wide keyword searches for MDTBL + pulmonary nodules + randomized controlled study) did not identify a peer-reviewed independent replication that explicitly reproduced this study’s intervention and design. Final Summary: Criterion R is not met because no independent replication of this specific study/intervention was identified.
  • A

    All-subject Exams

    • Because the assessment is not a standardized exam (E not met), the study cannot meet the all-subject standardized exam requirement.
    • All questions were developed by the course director together with clinical instructors based on Bloom's Taxonomy and the course learning objectives and current diagnostic and treatment guidelines for pulmonary nodules.
    • Relevant Quotes: 1) "All questions were developed by the course director together with clinical instructors based on Bloom's Taxonomy and the course learning objectives and current diagnostic and treatment guidelines for pulmonary nodules." (p. 4) 2) "This study examined the effectiveness of the MDTBL teaching model in learning diagnostic imaging of pulmonary nodules." (p. 7) Detailed Analysis: Criterion A requires standardized, exam-based assessment across all main subjects, and it has a prerequisite rule: if Criterion E is not met, then Criterion A is not met. In this study, the exams were developed internally by the course team, so E is not met. Therefore A must be not met. Additionally, the outcomes are focused on a single specialized topic (pulmonary nodules) rather than all main subjects assessed with standardized exams. Final Summary: Criterion A is not met because E is not met and the assessments are not standardized all-subject exams.
  • G

    Graduation Tracking

    • The study reports only within-course assessments and recommends future long-term follow-up, and no follow-up papers tracking the cohort to graduation were found.
    • Future studies could include more objective performance-based measures and consider using anonymous, delayed evaluations to reduce the influence of such biases and consider long-term follow-up with objective assessments to more comprehensively evaluate the effectiveness of the MDTBL teaching model.
    • Relevant Quotes: 1) "We compared pre-test, midterm, and final exam scores among the MDTBL, PBL, and LBL groups." (p. 6) 2) "Future studies could include more objective performance-based measures and consider using anonymous, delayed evaluations to reduce the influence of such biases and consider long-term follow-up with objective assessments to more comprehensively evaluate the effectiveness of the MDTBL teaching model." (p. 9) Detailed Analysis: Criterion G requires tracking participants until graduation from the relevant educational stage. The paper reports only course-based assessments (pre-, midterm, final) and questionnaires. The limitations section explicitly frames "long-term follow-up" as a future recommendation, indicating that graduation-linked follow-up was not conducted/reported in this study. Per the ERCT procedure, an internet search was conducted for follow-up publications by the same author team (Tao, Gao, et al.) that tracked this cohort to graduation. No such follow-up paper (with graduation tracking) was identified in the searched sources at the time of this ERCT check. Final Summary: Criterion G is not met because the study does not report tracking the cohort through graduation and no graduation-tracking follow-up paper was found.
  • P

    Pre-Registered

    • The study was retrospectively registered in 2023, after the course/study began in September 2022.
    • Trial registration This study was retrospectively registered in 2023 as No.JG2023-0203.
    • Relevant Quotes: 1) "Participants took part in a training course on the clinical diagnosis and evaluation of pulmonary nodules from September 15, 2022, through December 31, 2023." (p. 3) 2) "Trial registration This study was retrospectively registered in 2023 as No.JG2023-0203." (p. 3) Detailed Analysis: Criterion P requires that the study protocol be pre-registered before data collection (or the intervention) begins. The paper reports the course/intervention starting on September 15, 2022, and states that trial registration was retrospective and occurred in 2023. Retrospective registration necessarily occurs after the study has begun. A web search did not identify a public registry entry providing an earlier (pre-start) registration date for No. JG2023-0203. Final Summary: Criterion P is not met because registration is explicitly retrospective (in 2023) after the study began (in 2022).