Blended Learning in Anesthesiology Training: Is Interactive E-Learning an Effective Method for Enhancing Anesthesiological Skills?—A Single-Center Randomized Trial

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomisation was at the individual student level by lottery within one cohort, not at the class or school level, and no tutoring exception applies.
    • After providing written informed consent, 80 students were randomly assigned to the intervention (exposure) or control group by lottery.
    • Relevant Quotes: 1) "A single-center, randomized, cross-over trial was conducted at the University Medical Center Mainz." (Abstract, p. 1) 2) "After providing written informed consent, 80 students were randomly assigned to the intervention (exposure) or control group by lottery." (p. 3) 3) "Seventy-eight final-year medical students were randomized into an exposure group (instructional video based on Peyton's four-step approach) and a control group." (Abstract, p. 1) 4) "T1: The exposure group viewed the instructional video once on a tablet screen without pause or rewind functions, while the control group familiarized themselves with the equipment..." (p. 3) Detailed Analysis: The unit of randomisation was the individual student: students were "randomly assigned... by lottery" to the exposure or control group. This is a student-level randomisation carried out within a single course cohort, not randomisation of entire classes or schools. The ERCT 'C' criterion requires class-level (or stronger school-level) randomisation unless the intervention is a personal one-to-one tutoring format. Here the intervention is a self-directed instructional video delivered to individuals in the same training program, not one-to-one tutoring, so the tutoring exception does not apply. Because both conditions coexisted within the same student group, contamination risk is exactly what class-level randomisation is meant to avoid. Criterion C is not met because randomisation was performed at the individual student level rather than at the class or school level, and no tutoring exception applies.
  • E

    Exam-based Assessment

    • Outcomes were measured with a custom 14-item DOPS checklist built from the study's own video content and institutional SOP, not a widely recognized standardized exam.
    • "Performance was assessed using a checklist based on the video content and SOP steps, like the scoring system of Oriot et al. for pediatric intraosseous access." (p. 3)
    • Relevant Quotes: 1) "Performance was assessed using a checklist based on the video content and SOP steps, like the scoring system of Oriot et al. for pediatric intraosseous access. Each correctly performed step was awarded one point ... The checklist contained 14 items, with a maximum score of 16 points." (p. 3) 2) "We use 'Direct Observation of Procedural Skills (DOPS)' for the assessment. DOPS is a structured method in medical education in which procedural skills are directly observed and assessed using standardized criteria." (p. 2) 3) "The educational video covered the following key steps for radial artery puncture, aligned with the Standard Operating Procedure (SOP) of the University Medical Center Mainz and AMBOSS guidelines." (p. 3) Detailed Analysis: Criterion E requires a standardized, widely recognized exam-based assessment that was not custom-built for the study. Although DOPS is a recognized assessment format and the raters were blinded, the actual instrument used here was a bespoke 14-item checklist created specifically for this study, explicitly "based on the video content and SOP steps." Because the assessment is directly aligned with the intervention's own video content, it is exactly the kind of self-designed, potentially content-aligned measure the criterion warns against, and it is not a national or state-wide standardized examination. Criterion E is not met because the outcome was measured with a study-specific checklist aligned to the intervention content rather than a standardized exam.
  • T

    Term Duration

    • Outcomes were measured only one week after the intervention, far shorter than one academic term.
    • "T2: After one week, the groups switched conditions." (p. 3)
    • Relevant Quotes: 1) "T2: After one week, the groups switched conditions." (p. 3) 2) "The sustainability of learning was examined by reassessing the exposure group one week after the initial session." (p. 8) 3) "Additionally, the follow-up interval of one week does not provide information about long-term retention of procedural competence." (p. 9) Detailed Analysis: Criterion T requires that outcomes be measured at least one full academic term (approximately 3-4 months) after the intervention begins. In this study the intervention (video viewing plus simulation) and the final measurement (T2) were separated by only one week, and the authors themselves acknowledge that the one-week interval provides no information about longer-term retention. The interval from intervention start to final measurement is therefore an order of magnitude shorter than a single term. Criterion T is not met because the follow-up interval was only one week, well short of a full academic term.
  • D

    Documented Control Group

    • The control group's population, size, and conditions (equipment familiarization at T1, delayed video at T2) are documented, along with sample demographics.
    • "the control group familiarized themselves with the equipment (puncture arm, needles, positioning aids, pressure measurement system, hygiene materials)." (p. 3)
    • Relevant Quotes: 1) "T1: The exposure group viewed the instructional video once on a tablet screen ..., while the control group familiarized themselves with the equipment (puncture arm, needles, positioning aids, pressure measurement system, hygiene materials)." (p. 3) 2) "A total of 78 final-year medical students participated in the study. 41 (52.6%) were female and 37 (47.4%) were male, with a median age of 28 years. Thirty-three students (42.3%) had completed a vocational qualification prior to medical school. Sixteen students (28.2%) reported prior simulation training in radial artery puncture, and 26 students (44.1%) had previously performed at least one radial artery puncture on a patient." (p. 4) 3) "In the control group, 30 of the initial 38 students participated at T2. At T1, 10 students (26.3%) failed..." (p. 6) 4) "A critical limitation is the absence of a pre-intervention assessment of procedural competence ... no evidence is provided that the groups were equivalent in terms of prior technical skill before the intervention." (p. 9) Detailed Analysis: Criterion D requires that the control group be documented well enough to judge comparability and conditions. The paper documents the control group's size (n = 38 at T1, 30 at T2), the population it is drawn from (final-year medical students in the MINERVA course), and precisely what the control condition consisted of (equipment familiarization at T1, then receiving the instructional video at T2 under the crossover design). Demographic characteristics are reported for the full study sample, and the CONSORT flow diagram documents allocation and attrition. A genuine weakness is the absence of a group-specific baseline performance assessment, which the authors flag as a limitation; however, the composition, size, and conditions of the control group are clearly described. Criterion D is met because the control group's population, size, and conditions are documented, notwithstanding the lack of a baseline procedural-skill measurement.

• Level 2 Criteria

  • S

    School-level RCT

    • This was a single-center trial with randomization at the individual student level, not at the school level.
    • "A single-center, randomized, cross-over trial was conducted at the University Medical Center Mainz." (p. 1)
    • Relevant Quotes: 1) "A single-center, randomized, cross-over trial was conducted at the University Medical Center Mainz." (p. 1) 2) "80 students were randomly assigned to the intervention (exposure) or control group by lottery." (p. 3) Detailed Analysis: Criterion S requires randomization at the level of whole schools (or equivalent implementing institutions/units). This study was explicitly single-center and randomized individual students within one institution. There is no randomization of schools, sites, or centers, so the school-level requirement cannot be satisfied. Criterion S is not met because randomization occurred at the individual student level within a single center, not across schools.
  • I

    Independent Conduct

    • The same anesthesiology team designed the instructional video and conducted, analyzed, and reported the study, with no independent third-party evaluator.
    • "An instructional video was produced, demonstrating the step-by-step radial artery puncture procedure ... in accordance with the Department of Anesthesiology's standard operating procedure (SOP)." (p. 3)
    • Relevant Quotes: 1) "An instructional video was produced, demonstrating the step-by-step radial artery puncture procedure on a skills trainer ... in accordance with the Department of Anesthesiology's standard operating procedure (SOP)." (p. 3) 2) "Author Contributions: Conceptualization, A.M., S.K. and K.E.; methodology, A.M.; software, M.M.; ... formal analysis, A.M.; investigation, A.M.; data curation, A.M. and M.M.; writing—original draft preparation, A.M. and S.K." (p. 10) 3) "All video-recorded performances were assessed by two independent raters who were blinded to group allocation." (p. 3) Detailed Analysis: Criterion I requires that the study be conducted independently of the people who designed the intervention. Here the intervention (the instructional video, built from the department's own SOP) was created by the authors, and the same authors conceived, ran, analyzed, and reported the study, as shown by the author-contributions section. The use of two blinded raters improves internal rigor of the scoring, but the raters are part of the authors' own assessment process, not an external evaluation team independent of the intervention designers. No third-party or external evaluator conducted the trial. Criterion I is not met because the intervention was designed and the study conducted and analyzed by the same team, with no independent external evaluation.
  • Y

    Year Duration

    • The tracking interval was one week, nowhere near a full academic year, and criterion T is also not met.
    • "the follow-up interval of one week does not provide information about long-term retention of procedural competence." (p. 9)
    • Relevant Quotes: 1) "T2: After one week, the groups switched conditions." (p. 3) 2) "the follow-up interval of one week does not provide information about long-term retention of procedural competence." (p. 9) Detailed Analysis: Criterion Y requires outcomes to be measured across at least ~75% of a full academic year (roughly 9-10 months). The study's total interval from intervention to final measurement was one week. Furthermore, per the ERCT dependency rule, if criterion T (Term Duration) is not met, criterion Y cannot be met; T is not met here. Criterion Y is not met because the tracking interval was only one week, far below a full academic year, and T is not met.
  • B

    Balanced Control Group

    • The instructional video is the explicit treatment variable being tested, and the control group received a time-comparable active alternative (equipment familiarization) during the same session.
    • "the control group familiarized themselves with the equipment ... T2: After one week, the groups switched conditions." (p. 3)
    • Relevant Quotes: 1) "T1: The exposure group viewed the instructional video once on a tablet screen without pause or rewind functions, while the control group familiarized themselves with the equipment (puncture arm, needles, positioning aids, pressure measurement system, hygiene materials)." (p. 3) 2) "T2: After one week, the groups switched conditions." (p. 3) 3) "This study investigates the efficacy of a short, structured instructional video as part of a blended learning curriculum for teaching radial artery puncture." (p. 1) Detailed Analysis: Applying the criterion B decision tree: the only additional resource the exposure group received was the short instructional video, which is precisely the treatment variable the study set out to evaluate ("investigates the efficacy of a short, structured instructional video"). Under the standard, when the extra resource is itself the explicit treatment being tested, a business-as-usual control is acceptable. Moreover, the control was not idle: during the same session it engaged in an active alternative activity (familiarizing with the puncture arm, needles, positioning aids, pressure measurement system, and hygiene materials), so instructional time was broadly balanced. The crossover design also ensured both groups ultimately received the video. There is therefore no confounding, unmatched resource advantage separable from the intervention itself. Criterion B is met because the instructional video is the explicit treatment variable and the control group received a time-comparable active alternative before crossing over.

• Level 3 Criteria

  • R

    Reproduced

    • No independent replication of this specific study by a different team in a peer-reviewed journal is reported or evident.
    • Relevant Quotes: 1) "Evidence from other disciplines supports this approach: in obstetrics, for example, a blended learning module on managing shoulder dystocia enabled over 95% of students to meet performance standards in simulation ... (Stieglitz et al., 2023)." (p. 2) 2) "Other studies have shown that flipped classroom formats, mobile learning, and multimedia-enhanced skills training can improve student preparation, engagement, and psychomotor performance (Lew, 2016; Surov et al., 2021)." (p. 2) Detailed Analysis: Criterion R requires independent replication of the study by a different research team in a different context, published in a peer-reviewed journal. The paper cites related work on blended learning in other procedures (e.g., Stieglitz et al., 2023, on shoulder dystocia), but these are by the same Mainz group and/or concern different skills; they are not independent replications of this specific radial-artery-puncture video trial. An internet check was attempted for this recently published (June 2026) single-center Mainz crossover trial; the session's web-search quota was exhausted, but no independent replication of this specific study is reported in the paper or otherwise apparent, and given its very recent publication none would be expected. Criterion R is not met because there is no evidence of independent replication of this specific study in a peer-reviewed journal.
  • A

    All-subject Exams

    • Only a single procedural skill was assessed, not all core subjects, and criterion E is not met, so A cannot be met.
    • "This study investigates the efficacy of a short, structured instructional video as part of a blended learning curriculum for teaching radial artery puncture to final-year medical students." (p. 1)
    • Relevant Quotes: 1) "Performance was assessed using a 14-item checklist (maximum 16 points; pass threshold: 60%)." (p. 1) 2) "This study investigates the efficacy of a short, structured instructional video ... for teaching radial artery puncture to final-year medical students." (p. 1) Detailed Analysis: Criterion A requires impact to be measured across all main subjects using standardized exams, and it is explicitly contingent on criterion E being met. Here only one narrow procedural skill (radial artery puncture) was assessed, using a custom checklist rather than standardized exams across subjects. Because criterion E is not met and only a single skill was measured, criterion A fails on both counts. Criterion A is not met because only one procedural skill was assessed with a non-standardized instrument, and E is not met.
  • G

    Graduation Tracking

    • Participants were followed for only one week with no tracking to graduation, and criterion Y is not met.
    • "the follow-up interval of one week does not provide information about long-term retention of procedural competence." (p. 9)
    • Relevant Quotes: 1) "T2: After one week, the groups switched conditions." (p. 3) 2) "the follow-up interval of one week does not provide information about long-term retention of procedural competence." (p. 9) Detailed Analysis: Criterion G requires follow-up tracking of participants through to graduation and is contingent on criterion Y being met. The study tracked students for only one week and reports no follow-up through the end of their medical studies or any graduation outcome. An internet check for subsequent follow-up publications by the same authors tracking this cohort to graduation was attempted; the session's web-search quota was exhausted, but no such publication is referenced in the paper and none is apparent. Since Y is not met and no graduation tracking exists, G fails. Criterion G is not met because tracking lasted only one week with no graduation follow-up, and Y is not met.
  • P

    Pre-Registered

    • The paper reports only an ethics reference, not a pre-registered trial protocol with hypotheses and analysis plan filed before data collection.
    • "An ethics application was submitted (reference 2020-15360)." (p. 2)
    • Relevant Quotes: 1) "An ethics application was submitted (reference 2020-15360). According to the Rhineland-Palatinate ethics committee, no formal professional ethics review was required as the project involved training on models only." (p. 2) 2) "The Ethics Committee of Rhineland-Palatinate (reference number 2020-15360, approved on 22 September 2020) determined that formal ethical approval was not required for this study." (p. 10) 3) "To transparently report the flow of participants through each stage of the study, we followed the Consolidated Standards of Reporting Trials (CONSORT) guidelines." (p. 4) Detailed Analysis: Criterion P requires a full study protocol (hypotheses, methods, planned analyses) pre-registered in a trial registry before data collection began. The paper references only an ethics-committee submission (reference 2020-15360) and states it followed CONSORT reporting guidelines, but it provides no trial-registry identifier (e.g., DRKS, ClinicalTrials.gov, ISRCTN), no link, and no pre-registration date tied to a public protocol. The ethics reference is not a study-registry entry, so there is no registry to check against a data-collection start date. Ethics approval and CONSORT-compliant reporting are not equivalent to prospective protocol pre-registration. Criterion P is not met because no pre-registered trial protocol is referenced; only an ethics reference and CONSORT reporting are mentioned.