Web-Based Virtual Environment Versus Face-To-Face Delivery for Team-Based Learning of Anesthesia Techniques Among Undergraduate Medical Students: Randomized Controlled Trial

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • The study used cluster randomization at the class (scheduled session) level.
    • "We used cluster randomization at the class level to minimize contamination between teaching groups." (p. 2)
    • Relevant Quotes: 1) "We used cluster randomization at the class level to minimize contamination between teaching groups." (p. 2) 2) "Students were grouped into clusters based on their scheduled learning sessions. Each cluster comprised approximately 20 students." (p. 2) 3) "Clusters were then randomly allocated in a 1:1 ratio to either the WBVE or F2F group using computer-generated random numbers." (p. 2) Detailed Analysis: Criterion C requires that randomization occur at the class level (or a stronger unit such as school level) to reduce contamination between treatment and control students. The paper explicitly reports "cluster randomization at the class level" and defines clusters as scheduled learning sessions of about 20 students, followed by random allocation of clusters to WBVE vs F2F. This directly satisfies the ERCT requirement that intact classroom-like instructional units (scheduled sessions) are randomized rather than individual students within the same class session. Criterion C is met because clusters at the class/scheduled-session level were randomized to conditions.
  • E

    Exam-based Assessment

    • Knowledge was assessed with a study-specific 15-item MCQ rather than a widely recognized standardized exam.
    • "Individual knowledge was assessed before and after the session using a 15-item questionnaire containing multiple-choice questions via Google Forms." (p. 1)
    • Relevant Quotes: 1) "Individual knowledge was assessed before and after the session using a 15-item questionnaire containing multiple-choice questions via Google Forms." (p. 1) 2) "Pre- and postintervention knowledge evaluations were assessed using a 15-item MCQ test mapped to a test specification table (Multimedia Appendix 1)." (p. 4) Detailed Analysis: Criterion E requires outcomes to be measured using a standardized, widely recognized exam-based assessment (not created specifically for the study). The paper describes a 15-item multiple-choice questionnaire/MCQ test administered via Google Forms and mapped to a test specification table. This indicates a locally constructed assessment aligned to the course content rather than an external standardized examination (eg, national board exam or other widely recognized standardized test). The paper does not identify the knowledge test as a known standardized exam; instead, it describes a bespoke 15-item MCQ instrument for this session. Criterion E is not met because the primary knowledge assessment is a study-specific MCQ rather than a standardized exam.
  • T

    Term Duration

    • Outcomes were measured immediately after a single session rather than at least one academic term after the intervention began.
    • "Immediately after the teaching session, 15 minutes were allocated for outcome measurement." (p. 4)
    • Relevant Quotes: 1) "Each session lasted 130 minutes." (p. 3) 2) "Immediately after the teaching session, 15 minutes were allocated for outcome measurement." (p. 4) Detailed Analysis: Criterion T requires that outcomes be measured at least one academic term (approximately 3-4 months) after the intervention begins. Here, the intervention is explicitly a single class session lasting 130 minutes, and outcomes were measured immediately after the teaching session. Because the outcome measurement occurs minutes after completion of the intervention (not after a term-long delay), the study does not satisfy the required follow-up interval for Criterion T. Criterion T is not met because outcomes were assessed immediately after a single 130-minute session rather than after term-long follow-up.
  • D

    Documented Control Group

    • The paper documents the control condition and provides baseline characteristics and denominators by group.
    • "As shown in Table 1, 78 participants were included, with 38 (49%) assigned to the F2F group and 40 (51%) to the WBVE group." (p. 5)
    • Relevant Quotes: 1) "Hence, 79 participants were enrolled in the study and allocated into 2 groups: the F2F group (n=38, 48%) and the WBVE group (n=41, 52%)." (p. 4) 2) "A total of 78 (99%) participants had complete data in the analysis (Figure 2)." (p. 4) 3) "As shown in Table 1, 78 participants were included, with 38 (49%) assigned to the F2F group and 40 (51%) to the WBVE group." (p. 5) 4) "The groups were comparable in gender, age, and grade point average." (p. 5) Detailed Analysis: Criterion D requires a well-documented control group, including clear identification of the comparator condition and sufficient descriptive information (eg, baseline characteristics and denominators) to support comparisons. The paper clearly identifies the control/comparator condition as the face-to-face (F2F) group and reports denominators at allocation and analysis. It also explicitly reports that Table 1 provides baseline characteristics (including gender, age, and grade point average) and describes comparability across groups. Criterion D is met because the control group is clearly identified and baseline characteristics and group sizes are documented by group.

• Level 2 Criteria

  • S

    School-level RCT

    • The study was conducted at a single medical school and randomized scheduled-session clusters, not schools.
    • "We used cluster randomization at the class level to minimize contamination between teaching groups." (p. 2)
    • Relevant Quotes: 1) "We conducted a randomized, controlled, assessor-blinded trial at a Thai medical school from August 2024 to January 2025." (p. 1) 2) "We used cluster randomization at the class level to minimize contamination between teaching groups." (p. 2) Detailed Analysis: Criterion S requires randomization at the school/site level (ie, entire schools or equivalent sites are assigned). The paper describes a trial at a single Thai medical school and explicitly states randomization was done "at the class level" using clusters based on scheduled learning sessions. There is no evidence of multiple schools participating or of school-level randomization; therefore, the study does not meet the school-level RCT requirement. Criterion S is not met because randomization was at the class/cluster level within one school rather than across schools.
  • I

    Independent Conduct

    • The paper does not document a third-party independent evaluation team; implementation occurred within the same instructional setting.
    • "In each group, students were organized into 4 TBL teams of 5 to 6 students, supervised by 1 instructor." (p. 3)
    • Relevant Quotes: 1) "Department of Anesthesiology, Faculty of Medicine, Khon Kaen University, Muang Khon Kaen, Khon Kaen, Thailand" (p. 1) 2) "In each group, students were organized into 4 TBL teams of 5 to 6 students, supervised by 1 instructor." (p. 3) 3) "The randomization sequence was generated and implemented by investigators who were not involved in teaching or assessment to reduce the risk of allocation bias." (p. 2) Detailed Analysis: Criterion I requires that the evaluation be conducted independently from the intervention designers/implementers (eg, an external evaluation team), to reduce bias in delivery, measurement, and analysis. The paper shows internal role separation for randomization (investigators not involved in teaching or assessment), but this is not the same as an independent, third-party evaluation. The teaching sessions were supervised by an instructor within the same departmental setting, and the paper does not state that data collection/analysis was performed by an external organization independent of the intervention context. Criterion I is not met because independent (third-party) conduct is not documented beyond internal role separation.
  • Y

    Year Duration

    • Outcomes were measured immediately post-session, so the study does not include at least 75% of an academic year of outcome tracking.
    • "Immediately after the teaching session, 15 minutes were allocated for outcome measurement." (p. 4)
    • Relevant Quotes: 1) "The study was conducted from August 26, 2024, to January 7, 2025." (p. 2) 2) "Each session lasted 130 minutes." (p. 3) 3) "Immediately after the teaching session, 15 minutes were allocated for outcome measurement." (p. 4) Detailed Analysis: Criterion Y requires learning outcomes to be measured at least 75% of an academic year after the intervention begins. Although the study has an overall conduct window spanning August 2024 to January 2025, the primary outcomes were measured immediately after a single 130-minute session. Since the outcome measurement does not track learning outcomes over an academic-year timescale (and the term-duration criterion is not met), it cannot satisfy the year-duration requirement. Criterion Y is not met because outcomes were assessed immediately after a single session rather than tracked across (most of) an academic year.
  • B

    Balanced Control Group

    • The study explicitly kept instructional time, content, and materials the same across groups, varying only delivery mode.
    • "Only the delivery mode differed (Spatial WBVE vs classroom F2F)." (p. 1)
    • Relevant Quotes: 1) "Both groups received identical 10-section content in a standardized TBL sequence lasting 130 minutes." (p. 1) 2) "Only the delivery mode differed (Spatial WBVE vs classroom F2F)." (p. 1) 3) "Both groups participated in a single, standardized 130-minute TBL session. The content was identical, organized into 10 sections on anesthesia techniques (listed in Multimedia Appendix 1)." (p. 3) 4) "Materials were identical to those in the other group (handouts mirroring the slides and identical question stems)." (p. 3) Detailed Analysis: Criterion B requires that time and resources be balanced across intervention and control conditions, unless the resource difference is explicitly the treatment being tested. The paper repeatedly emphasizes equivalence of instructional inputs: both groups had the same standardized 130-minute TBL session, identical content, and identical materials. The intended contrast is explicitly limited to modality ("Only the delivery mode differed"). Although the WBVE condition necessarily involves platform technology, this technology is integral to the delivery-mode contrast and the control condition provides a directly comparable substitute in instructional time, content coverage, and instructor-facilitated TBL structure. Criterion B is met because the study documents equivalent instructional time and materials, with delivery mode as the primary difference.

• Level 3 Criteria

  • R

    Reproduced

    • No independent replication study by a different research team could be identified.
    • Relevant Quotes: 1) (No statements in the paper indicate that this specific trial has been independently replicated by other researchers.) (p. 1-12) Detailed Analysis: Criterion R requires an independent replication by a different author team in a different context, published in a peer-reviewed outlet. The paper itself does not report that this study has been replicated. Additionally, an internet search (as of 2026-03-04) did not identify peer-reviewed publications by other author teams that explicitly replicate this specific trial (WBVE Spatial vs classroom F2F delivery of an otherwise identical TBL anesthesia session at Khon Kaen University). Criterion R is not met because no independent reproduction was found.
  • A

    All-subject Exams

    • Because the study does not use standardized exams (Criterion E is not met), it also cannot meet the all-subject standardized exam requirement.
    • "Individual knowledge was assessed before and after the session using a 15-item questionnaire containing multiple-choice questions via Google Forms." (p. 1)
    • Relevant Quotes: 1) "Individual knowledge was assessed before and after the session using a 15-item questionnaire containing multiple-choice questions via Google Forms." (p. 1) Detailed Analysis: Criterion A requires standardized exam-based assessments across all main subjects and depends on Criterion E being satisfied. Here, the study uses a study-specific 15-item MCQ focused on anesthesia techniques rather than standardized exams, and it does not assess all core subjects. Because Criterion E is not met, Criterion A cannot be met. Criterion A is not met because standardized exams were not used, so all-subject standardized testing is not possible.
  • G

    Graduation Tracking

    • The study does not track participants to graduation and does not meet the year-duration prerequisite.
    • "Immediately after the teaching session, 15 minutes were allocated for outcome measurement." (p. 4)
    • Relevant Quotes: 1) "Each session lasted 130 minutes." (p. 3) 2) "Immediately after the teaching session, 15 minutes were allocated for outcome measurement." (p. 4) Detailed Analysis: Criterion G requires tracking participants through graduation from the relevant educational stage. This study measures outcomes immediately after a single teaching session and does not describe any longer-term follow-up, cohort linkage, or graduation outcome collection. Per the ERCT dependency rule, because Criterion Y (year duration) is not met, Criterion G cannot be met. An internet search (as of 2026-03-04) did not identify follow-up papers by the same author team reporting graduation tracking for this cohort. Criterion G is not met because there is no long-term follow-up to graduation and the year-duration prerequisite is not satisfied.
  • P

    Pre-Registered

    • The trial reports registration in the Thai Clinical Trials Registry before participant enrollment, and an earlier preprint version states a registration date preceding study conduct.
    • "It was also registered with the Thai Clinical Trials Registry before participant enrollment (TCTR20240708012)." (p. 4)
    • Relevant Quotes: Main paper: 1) "Trial Registration: Thai Clinical Trials Registry TCTR20240708012; https://www.thaiclinicaltrials.org/show/TCTR20240708012" (p. 1) 2) "It was also registered with the Thai Clinical Trials Registry before participant enrollment (TCTR20240708012)." (p. 4) Other sources (earlier preprint version of the same study): 3) "Clinical Trial: TCTR20240708012 (Thai Clinical Trials Registry) registered July 8, 2024." (JMIR Preprints, version posted 2025) Detailed Analysis: Criterion P requires pre-registration of the study protocol before the study begins. The peer-reviewed paper provides a registry identifier and URL, and explicitly states that the study was registered "before participant enrollment," which directly addresses the required timing. During this ERCT check, the Thai Clinical Trials Registry landing page could not be reliably accessed in a way that exposed the full registry record text; therefore, the independent database verification is limited to the paper’s stated registry link plus the earlier preprint’s explicit registration date statement. The preprint version additionally provides a concrete registration date (July 8, 2024), and the main paper reports the study conduct window starting August 26, 2024, which is consistent with prospective registration. Criterion P is met because the paper states registration occurred before enrollment and the preprint indicates a registration date preceding the study period.