Team-based vs. problem-based learning in undergraduate surgery: a randomized controlled trial in Saudi Arabia

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomization was at the individual student level (not class- or school-level), and the tutoring/personal-teaching exception does not apply.
    • All 66 students were double-blinded and randomized into groups (TBL 34 and PBL 32).
    • Relevant Quotes: 1) "Our targeted population (66 medical students) were the final medical students in our faculty of medicine at the University of Tabuk during the period from 15/10/2024 to 15/01/2025." (p. 3) 2) "All 66 students were double-blinded and randomized into groups (TBL 34 and PBL 32)." (p. 3) 3) "We randomly divided the case (TBL 34) group into seven groups in total, six groups consisting of five students, and the final group included four students." (p. 3) 4) "In contrast, the control group (PBL32) was divided randomly into seven groups in total, with six groups comprising five each, and the last group had two students." (p. 3) Detailed Analysis: Criterion C requires randomization at the class level (or stronger school/site level) to reduce contamination between treatment and control conditions. The paper states that all 66 students were randomized into two groups (TBL vs. PBL). The subsequent division into small groups occurs within each treatment arm and does not change the unit of randomization (students). The intervention is group-based pedagogy (TBL vs. PBL), not a one-to-one tutoring/personal-teaching intervention, so the criterion C exception does not apply. Final sentence explaining if criterion C is not met because the unit of randomization is individual students rather than classes (or schools/sites), with no applicable exception.
  • E

    Exam-based Assessment

    • Outcomes were measured using study-specific MCQ questionnaires rather than a standardized, widely recognized exam-based assessment.
    • At the end of the clinical sessions, all 66 students took a summative test to assess their knowledge using 14 multiple-choice questions (MCQs) that were part of an online self-administered, voluntary questionnaire...
    • Relevant Quotes: 1) "At the end of the clinical sessions, all 66 students took a summative test to assess their knowledge using 14 multiple-choice questions (MCQs) that were part of an online self-administered, voluntary questionnaire..." (p. 1) 2) "The case (TBL-34) group was provided the opportunity to read the subject before individually submitting themselves to an individual assurance test (iRAT) based on 14 MCQs—an online, self-administered, voluntary questionnaire." (p. 3) Detailed Analysis: Criterion E requires a standardized exam-based assessment (e.g., recognized external/standard instruments), not a custom or study-specific test. The paper describes a 14-item MCQ test delivered as an online, self-administered questionnaire (including iRAT/tRAT and the summative test). The paper does not identify these MCQs as a standardized exam used beyond the study context. Final sentence explaining if criterion E is not met because the assessment is an online 14-item MCQ questionnaire rather than a standardized, widely recognized exam.
  • T

    Term Duration

    • The study ran from 15/10/2024 to 15/01/2025 with assessment at the end of the sessions, which is approximately one academic term.
    • Our targeted population (66 medical students) were the final medical students in our faculty of medicine at the University of Tabuk during the period from 15/10/2024 to 15/01/2025.
    • Relevant Quotes: 1) "Our targeted population (66 medical students) were the final medical students in our faculty of medicine at the University of Tabuk during the period from 15/10/2024 to 15/01/2025." (p. 3) 2) "At the end of the clinical sessions, all 66 students took a summative test to assess their knowledge..." (p. 1) Detailed Analysis: Criterion T requires outcomes to be measured at least one full academic term after the intervention begins (typically about 3–4 months). The paper provides explicit start and end dates (15/10/2024 to 15/01/2025), which is roughly three months, and states outcomes were measured at the end of the clinical sessions. This meets the "at least one term" intent as operationalized in the ERCT definition (approximately a term-length interval). Final sentence explaining if criterion T is met because the study period spans roughly one academic term and outcomes were measured at the end of that term-length period.
  • D

    Documented Control Group

    • The control (PBL) group is described with sample size, baseline characteristics, and a clear description of what the control condition received.
    • TABLE 1 Baseline characteristics of participants.
    • Relevant Quotes: 1) "The control group (PBL) comprised 32 students, whereas the case group (TBL) comprised 34 students." (p. 3) 2) "TABLE 1 Baseline characteristics of participants." (p. 5) 3) "In contrast, the control group (PBL32) was divided randomly into seven groups in total, with six groups comprising five each, and the last group had two students." (p. 3) 4) "The exercise was supervised by an expert teacher, with minimal interference if required, and a facilitator in each group." (p. 3) Detailed Analysis: Criterion D requires a well-documented control group, including the control group size, baseline/demographic characteristics, and what the control group received. The paper provides control group size (PBL n = 32), reports baseline characteristics (Table 1 includes age, GPA, and prior exposure), and describes how the PBL control condition was run (group structure and supervision/facilitation). Final sentence explaining if criterion D is met because the paper provides control group size, baseline characteristics, and a clear description of the PBL control condition.

• Level 2 Criteria

  • S

    School-level RCT

    • The unit of randomization is students within one institution, not schools or equivalent educational sites.
    • All 66 students were double-blinded and randomized into groups (TBL 34 and PBL 32).
    • Relevant Quotes: 1) "The study was conducted among sixth-year clinical students at the Faculty of Medicine, University of Tabuk." (p. 1) 2) "All 66 students were double-blinded and randomized into groups (TBL 34 and PBL 32)." (p. 3) Detailed Analysis: Criterion S requires randomization at the school/site level (i.e., assignment among multiple schools or comparable educational sites). The paper describes a single-institution study at the University of Tabuk with student-level randomization into two arms. No evidence is provided of multiple schools/sites being randomized. Final sentence explaining if criterion S is not met because the unit of randomization is students within one institution rather than schools/sites.
  • I

    Independent Conduct

    • The paper does not document independent third-party conduct of implementation and/or evaluation separate from the study team.
    • The exercise was supervised by an expert teacher with a facilitator in each group.
    • Relevant Quotes: 1) "The exercise was supervised by an expert teacher with a facilitator in each group." (p. 3) 2) "Data were collected using structured electronic questionnaires administered through Google Forms for each study group." (p. 4) Detailed Analysis: Criterion I requires that the evaluation be conducted independently from the intervention designers/implementers, or that the paper clearly documents independent evaluation (e.g., an external evaluation team, explicit independence statements). The paper describes in-faculty supervision and facilitation and describes data collection by the study process, but it does not state that an independent external evaluation team conducted the implementation, measurement, or analysis, nor does it include an explicit independence statement. Final sentence explaining if criterion I is not met because the paper does not document independent third-party conduct of the study.
  • Y

    Year Duration

    • The study period (15/10/2024 to 15/01/2025) is about three months, which is well below 75% of an academic year, and the paper notes a short follow-up duration.
    • Our targeted population (66 medical students) were the final medical students in our faculty of medicine at the University of Tabuk during the period from 15/10/2024 to 15/01/2025.
    • Relevant Quotes: 1) "Our targeted population (66 medical students) were the final medical students in our faculty of medicine at the University of Tabuk during the period from 15/10/2024 to 15/01/2025." (p. 3) 2) "Another limitation is the short follow-up duration, which was not our objective..." (p. 9) Detailed Analysis: Criterion Y requires outcomes to be measured at least 75% of one academic year after the intervention begins (commonly about 7–8+ months). The stated study period is approximately three months. The paper also explicitly acknowledges a "short follow-up duration." This does not meet the year-duration threshold. Final sentence explaining if criterion Y is not met because the start-to-outcome window is about three months, far less than 75% of an academic year.
  • B

    Balanced Control Group

    • Both conditions had the same in-session duration (4 h) and similar facilitation, and the additional TBL elements (pre-class preparation materials and readiness assurance tests) are integral to the TBL treatment being compared to PBL.
    • The duration of each session was 4 h.
    • Relevant Quotes: 1) "The duration of each session was 4 h." (p. 3) 2) "We provided each (TBL) group with pre-class preparatory material, which included recorded lectures on diabetic foot and abdominal hernias." (p. 3) 3) "The case (TBL-34) group was provided the opportunity to read the subject before individually submitting themselves to an individual assurance test (iRAT)..." (p. 3) 4) "The duration of each session was 4 h." (PBL description) (p. 3) 5) "The control (PBL32) group did not have the privilege of pre-class material preparation." (p. 3) Detailed Analysis: Criterion B compares the nature, quantity, and quality of resources (time, materials, staff support) provided to both conditions, and asks whether the control offers a comparable substitute for the intervention's inputs unless the extra inputs are integral to what is being tested. Here, both groups are described as having equal scheduled in-room time ("4 h") and both are supervised/facilitated. The TBL arm additionally receives pre-class preparatory material and uses iRAT/tRAT readiness assurance testing, while the PBL arm "did not have the privilege of pre-class material preparation." These additional elements are not incidental add-ons; they are central defining components of TBL as implemented in this study and thus are integral to the pedagogical method being evaluated against PBL. Therefore, the resource differences are part of the intended treatment contrast rather than an unacknowledged, separable confound. Final sentence explaining if criterion B is met because in-session time and facilitation are balanced and the extra TBL inputs are integral components of the TBL intervention under test.

• Level 3 Criteria

  • R

    Reproduced

    • No independent replication of this specific University of Tabuk TBL-vs-PBL RCT by a different team in a different context was found.
    • Nevertheless, there is less research comparing TBL to PBL.
    • Relevant Quotes: 1) "Nevertheless, there is less research comparing TBL to PBL." (p. 2) Detailed Analysis: Criterion R requires an independent replication of the study (or a clearly identified replication of the same intervention and claim) by a different research team in a different context, published in a peer-reviewed journal. The paper discusses the broader literature comparing TBL and PBL, but it does not cite any replication of this specific RCT. Additional internet searching by DOI/title did not identify any published independent replication of this specific University of Tabuk RCT as of the ERCT check date. Final sentence explaining if criterion R is not met because no independent replication of this specific RCT has been identified.
  • A

    All-subject Exams

    • Criterion E is not met (no standardized exam), so criterion A is automatically not met; additionally, the study assesses only surgery-specific knowledge rather than all main subjects.
    • The study included general surgery as a subject for final-year MBBS students at the University of Tabuk.
    • Relevant Quotes: 1) "The study included general surgery as a subject for final-year MBBS students at the University of Tabuk." (p. 3) 2) "At the end of the clinical sessions, all 66 students took a summative test to assess their knowledge using 14 multiple-choice questions (MCQs)..." (p. 1) Detailed Analysis: Criterion A requires standardized exam-based assessment across all main subjects, and the ERCT rules specify that if criterion E is not met then criterion A is not met. This study uses a study-specific MCQ assessment (so criterion E is not met) and the content focus is general surgery rather than a comprehensive set of main subjects. Final sentence explaining if criterion A is not met because criterion E is not met and the study does not assess all main subjects using standardized exams.
  • G

    Graduation Tracking

    • The study does not track learners through graduation and notes a short follow-up and lack of long-term retention assessment; also, because Y is not met, G cannot be met.
    • Another limitation is the short follow-up duration, which was not our objective...
    • Relevant Quotes: 1) "It is important to note that our study did not assess the retention rate, as long-term retention tests were not conducted." (p. 7) 2) "Another limitation is the short follow-up duration, which was not our objective..." (p. 9) Detailed Analysis: Criterion G requires tracking participants until graduation from the relevant educational stage. The paper explicitly states that long-term retention was not assessed and that a short follow-up duration is a limitation, which indicates outcomes were not tracked over long horizons (and not through graduation). Additionally, the ERCT rules specify that if criterion Y is not met, then criterion G is not met. Since the study duration is approximately three months, Y is not met, so G cannot be met. Internet searching for later follow-up publications by the same authors reporting graduation tracking for this cohort did not identify any such follow-up paper as of the ERCT check date. Final sentence explaining if criterion G is not met because the study does not include graduation tracking, and criterion Y is not met.
  • P

    Pre-Registered

    • No pre-registration registry, registration ID, or registration date is reported, and no external protocol record was identified.
    • Relevant Quotes: 1) "The Ethical Review Committee for Medical Research Involving Human Subjects at the University of Tabuk approved this study (approval number: UT-448-251-2024) on November 3, 2024." (p. 4) Detailed Analysis: Criterion P requires a publicly pre-registered protocol with a registry link/ID and a registration date before data collection begins. The paper reports ethics approval but does not mention any trial registry, registration number, or registration date. Searching by the paper DOI/title did not identify a public pre-registration record for this study. Final sentence explaining if criterion P is not met because no pre-registration registry/ID/date is reported or found.