Web-Based Virtual Environment Versus Face-To-Face Delivery for Team-Based Learning of Anesthesia Techniques Among Undergraduate Medical Students: Randomized Controlled Trial

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomization occurred at the class (cluster) level, which meets the class-level RCT requirement.
    • "We used cluster randomization at the class level to minimize contamination between teaching groups." (p. 2)
    • Relevant Quotes: 1) "We used cluster randomization at the class level to minimize contamination between teaching groups." (p. 2) 2) "Students were grouped into clusters based on their scheduled learning sessions. Each cluster comprised approximately 20 students." (p. 2) 3) "Clusters were then randomly allocated in a 1:1 ratio to either the WBVE or F2F group using computer-generated random numbers." (p. 2) Detailed Analysis: ERCT Criterion C requires randomization at the class level (or stronger) to reduce contamination within a classroom. The paper explicitly states cluster randomization "at the class level" and explains that clusters were built from scheduled learning sessions (approximately 20 students per cluster), and then randomized 1:1 to WBVE vs F2F. This matches the ERCT intent: entire teaching groups (classes/sessions) are assigned to one condition, rather than mixing treatment and control students in the same live class setting. Criterion C is met because the paper clearly describes class-level (cluster) randomization.
  • E

    Exam-based Assessment

    • Outcomes were measured with a study-specific 15-item MCQ test rather than a widely recognized standardized exam.
    • "Pre- and postintervention knowledge evaluations were assessed using a 15-item MCQ test mapped to a test specification table (Multimedia Appendix 1)." (p. 4)
    • Relevant Quotes: 1) "The primary outcome was the postclass multiple-choice questionnaire score." (p. 1) 2) "Individual knowledge was assessed before and after the session using a 15-item questionnaire containing multiple-choice questions via Google Forms." (p. 1) 3) "Pre- and postintervention knowledge evaluations were assessed using a 15-item MCQ test mapped to a test specification table (Multimedia Appendix 1)." (p. 4) Detailed Analysis: ERCT Criterion E requires a standardized, widely recognized exam-based assessment (eg, national/state standardized tests), not a researcher-constructed instrument aligned to the intervention. Here, the assessment is described as a 15-item MCQ questionnaire administered via Google Forms and mapped to a test specification table in an appendix. The paper does not identify the instrument as a national, licensing, or otherwise widely recognized standardized examination, nor does it cite a validated external test name. Therefore, the evidence indicates a study-specific MCQ test, so the standardized exam requirement is not met. Criterion E is not met because the knowledge assessment is a study-specific MCQ questionnaire rather than a standardized exam.
  • T

    Term Duration

    • The primary outcome was measured immediately after a single 130-minute session, not at least one academic term later.
    • "Immediately after the teaching session, 15 minutes were allocated for outcome measurement." (p. 4)
    • Relevant Quotes: 1) "Both groups participated in a single, standardized 130-minute TBL session." (p. 3) 2) "Immediately after the teaching session, 15 minutes were allocated for outcome measurement." (p. 4) 3) "Satisfaction was measured immediately after class on a 5-point Likert scale." (p. 1) 4) "The study was conducted from August 26, 2024, to January 7, 2025." (p. 2) Detailed Analysis: ERCT Criterion T requires that outcomes be measured at least one academic term (about 3-4 months) after the intervention begins, allowing time for durable learning effects (or fade-out) to appear. Although the overall study period spans months (Aug 2024 to Jan 2025), the intervention itself is described as a single 130-minute session, and the primary outcome measurement is explicitly "immediately after the teaching session" (with 15 minutes allocated). Satisfaction is also measured "immediately after class." This is a very short follow-up window (minutes/hours), not term-length tracking from start to outcome. Criterion T is not met because outcomes were measured immediately after a single session rather than after a full academic term.
  • D

    Documented Control Group

    • The paper clearly documents the comparison (control) group, including group sizes and baseline characteristics.
    • "As shown in Table 1, 78 participants were included, with 38 (49%) assigned to the F2F group and 40 (51%) to the WBVE group." (p. 5)
    • Relevant Quotes: 1) "In total, 79 students were randomized in this study (F2F: n=38, 48%; WBVE: n=41, 52%)." (p. 1) 2) "As shown in Table 1, 78 participants were included, with 38 (49%) assigned to the F2F group and 40 (51%) to the WBVE group." (p. 5) 3) "The groups were comparable in gender, age, and grade point average." (p. 5) 4) "Preclass scores were similar between groups (F2F: mean 6.03, SD 2.05; WBVE: mean 6.20, SD 2.04)." (p. 1) Detailed Analysis: ERCT Criterion D requires that the control/comparison group be well documented, including sample size, baseline characteristics, and what they received. The paper provides explicit group counts (F2F vs WBVE), reports baseline comparability (gender/age/GPA), and provides baseline knowledge scores. The F2F arm functions as the control/comparator condition (standard face-to-face delivery), and it is clearly described and tabulated. This meets the documentation requirement for the control group. Criterion D is met because the comparator group is clearly described with sizes and baseline characteristics.

• Level 2 Criteria

  • S

    School-level RCT

    • Randomization was at the class (cluster) level within one medical school, not across multiple schools.
    • "We used cluster randomization at the class level to minimize contamination between teaching groups." (p. 2)
    • Relevant Quotes: 1) "We conducted a randomized, controlled, assessor-blinded trial at a Thai medical school..." (p. 1) 2) "We used cluster randomization at the class level to minimize contamination between teaching groups." (p. 2) 3) "Each cluster comprised approximately 20 students." (p. 2) Detailed Analysis: ERCT Criterion S requires randomization at the school level (ie, multiple schools randomized to conditions), which increases real-world relevance and captures school-wide factors. This study was conducted at a single Thai medical school, and randomization is explicitly stated to be at the class (cluster) level based on scheduled sessions, not among separate schools. Therefore, the school-level RCT requirement is not satisfied. Criterion S is not met because assignment was by class clusters within a single school, not by school.
  • I

    Independent Conduct

    • While some procedures were blinded and separated from teaching, there is no clear third-party independent evaluation team separate from the intervention designers.
    • "The randomization sequence was generated and implemented by investigators who were not involved in teaching or assessment to reduce the risk of allocation bias." (p. 2)
    • Relevant Quotes: 1) "This study was a randomized, controlled, single-blinded trial." (p. 2) 2) "The randomization sequence was generated and implemented by investigators who were not involved in teaching or assessment to reduce the risk of allocation bias." (p. 2) 3) "Outcome scoring and data analysis were blinded to group assignment." (p. 1) 4) "Formal analysis: DS, SB, PB" (p. 9) Detailed Analysis: ERCT Criterion I requires independent conduct, meaning the evaluation is carried out by a third party independent from the intervention designers/developers to reduce bias in implementation, measurement, and analysis. The paper shows some separation of roles (randomization done by investigators not involved in teaching or assessment) and blinding of outcome scoring/data analysis to group assignment. However, these statements still describe the work as being done by "investigators" and the listed author team performed the formal analysis. There is no explicit statement that an external/third-party evaluation organization (independent from the intervention development and study team) conducted the implementation and analysis. Therefore, the stricter ERCT independence requirement is not met. Criterion I is not met because the paper does not clearly document an independent third-party evaluator separate from the study/intervention team.
  • Y

    Year Duration

    • Because term-duration follow-up is not met and outcomes were measured immediately after the session, year-duration tracking is also not met.
    • "Immediately after the teaching session, 15 minutes were allocated for outcome measurement." (p. 4)
    • Relevant Quotes: 1) "Both groups participated in a single, standardized 130-minute TBL session." (p. 3) 2) "Immediately after the teaching session, 15 minutes were allocated for outcome measurement." (p. 4) Detailed Analysis: ERCT Criterion Y requires outcomes measured at least 75% of an academic year after the intervention begins. Per the ERCT rule provided, if Criterion T (term duration) is not met, then Criterion Y is not met. This study measures outcomes immediately after a single session, so it does not meet term-duration follow-up and cannot meet year-duration tracking. Criterion Y is not met because there is no year-long tracking and (by rule) T is not met.
  • B

    Balanced Control Group

    • Both groups received comparable instructional time and core materials, and the delivery mode (Spatial WBVE vs classroom) was the intended treatment difference.
    • "Both groups received identical 10-section content in a standardized TBL sequence lasting 130 minutes. Only the delivery mode differed (Spatial WBVE vs classroom F2F)." (p. 1)
    • Relevant Quotes: 1) "Both groups received identical 10-section content in a standardized TBL sequence lasting 130 minutes. Only the delivery mode differed (Spatial WBVE vs classroom F2F)." (p. 1) 2) "After recruitment into the study, all participants in both groups were given access to the learning material via online learning in Google Classroom, including a slide presentation and supplementary video material, 2 days before the teaching session." (p. 2) 3) "In each group, students were organized into 4 TBL teams of 5 to 6 students, supervised by 1 instructor. Each session lasted 130 minutes." (p. 2) 4) "Materials were identical to those in the other group (handouts mirroring the slides and identical question stems)." (p. 3) 5) "Sessions ran on the Spatial platform (Spatial Systems, Inc; Thai localization) [19] using desktop computers on the university’s secure local network." (p. 3) Detailed Analysis: ERCT Criterion B examines whether extra resources (time, staffing, budget, materials) are imbalanced between conditions in a way that confounds the intervention effect, unless those additional resources are explicitly the treatment variable. The paper states both groups received the same curriculum content and the same standardized session length (130 minutes), and that "Only the delivery mode differed." It also documents that both groups received the same pre-session access to learning materials (Google Classroom) and that the classroom materials (handouts/question stems) were identical. The WBVE condition uses the Spatial platform and desktop computers, but these technology inputs are integral to the delivery-mode contrast being tested (WBVE vs F2F), not an optional add-on that could be separated from the intervention definition. The control condition (F2F) is the intended comparator delivery mode and receives comparable instructional time and core content. Criterion B is met because time, core content, and core materials are comparable, and the remaining resource differences are integral to the explicitly tested delivery modality.

• Level 3 Criteria

  • R

    Reproduced

    • No peer-reviewed, independent replication of this specific trial was identified as of the ERCT check date.
    • Relevant Quotes: None found in the paper that describe an independent replication of this specific trial. Detailed Analysis: ERCT Criterion R requires that the study be independently replicated by a different research team in a different context and published in a peer-reviewed outlet. The paper does not report any independent replication of this specific WBVE-versus-F2F TBL trial. An internet search (performed as part of this ERCT check on 2026-04-16) did not identify a peer-reviewed, independent replication of this exact trial (same delivery-mode contrast, population, and outcomes). Criterion R is not met because independent replication of this specific trial was not found.
  • A

    All-subject Exams

    • Because a standardized exam was not used (E not met), the all-subject standardized exam requirement is not met.
    • "Pre- and postintervention knowledge evaluations were assessed using a 15-item MCQ test..." (p. 4)
    • Relevant Quotes: 1) "The primary outcome was the postclass multiple-choice questionnaire score." (p. 1) 2) "Pre- and postintervention knowledge evaluations were assessed using a 15-item MCQ test..." (p. 4) Detailed Analysis: ERCT Criterion A requires standardized, exam-based assessments across all main subjects, and the provided ERCT rule states: if Criterion E is not met, then Criterion A is not met. This study uses a study-specific MCQ focused on anesthesia techniques rather than a standardized exam, so it fails the prerequisite for all-subject standardized testing. In addition, the outcomes are specific to the anesthesiology teaching content, not across a full general curriculum. Criterion A is not met because E is not met and the study does not assess all subjects using standardized exams.
  • G

    Graduation Tracking

    • There is no follow-up through graduation, and since year duration (Y) is not met, graduation tracking (G) is also not met.
    • "Immediately after the teaching session, 15 minutes were allocated for outcome measurement." (p. 4)
    • Relevant Quotes: 1) "Immediately after the teaching session, 15 minutes were allocated for outcome measurement." (p. 4) 2) "Satisfaction was measured immediately after class on a 5-point Likert scale." (p. 1) Detailed Analysis: ERCT Criterion G requires tracking participants until graduation to assess long-run outcomes. The provided ERCT rule also states: if Criterion Y is not met, then Criterion G is not met. The only reported measurements occur immediately after the session (knowledge posttest and satisfaction). No later follow-up windows are described, and nothing indicates tracking students through graduation. In addition, an internet search (performed as part of this ERCT check on 2026-04-16) did not identify a subsequent peer-reviewed follow-up paper by the same author team reporting graduation outcomes for this cohort. Criterion G is not met because there is no graduation tracking and (by rule) Y is not met.
  • P

    Pre-Registered

    • The trial reports a public registry ID and states registration occurred before participant enrollment, and an earlier preprint reports a registration date that precedes the study start date.
    • "It was also registered with the Thai Clinical Trials Registry before participant enrollment (TCTR20240708012)." (p. 4)
    • Relevant Quotes: 1) "Trial Registration: Thai Clinical Trials Registry TCTR20240708012; https://www.thaiclinicaltrials.org/show/TCTR20240708012" (p. 1) 2) "It was also registered with the Thai Clinical Trials Registry before participant enrollment (TCTR20240708012)." (p. 4) 3) "The study was conducted from August 26, 2024, to January 7, 2025." (p. 2) 4) "Clinical Trial: TCTR20240708012 (Thai Clinical Trials Registry) registered July 8, 2024." (JMIR Preprints #80097, published 2025-07-04, accepted 2025-12-24) Detailed Analysis: ERCT Criterion P requires a study protocol to be pre-registered before the study begins, with evidence of the registry record and timing. The published paper provides a concrete registry and ID (TCTR20240708012), and explicitly states registration occurred "before participant enrollment." While the Thai Clinical Trials Registry record itself could not be fully retrieved in a text-verifiable way during this ERCT check (likely due to dynamic page rendering), an earlier JMIR preprint associated with this article explicitly reports that the trial was "registered July 8, 2024." The paper reports the study was conducted from August 26, 2024, to January 7, 2025, so the reported registration date (July 8, 2024) is before the reported study start date (August 26, 2024), which supports prospective registration. Criterion P is met because the paper documents a public registry ID and prospective registration, and the related preprint reports a registration date that precedes the reported study start date.