Effects of game-based learning and telesimulation methods on the knowledge, skills, motivations, and self-efficacy levels of nursing students in postpartum hemorrhage management: a randomized controlled trial

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomization was performed at the individual student level within a single course, not at the class or school level, and no tutoring exception applies.
    • "The 177 nursing students who met the inclusion criteria were stratified according to cumulative grade point average and randomly allocated to one control group and three intervention groups using a block randomization method." (p. 7)
    • Relevant Quotes: 1) "Participants were classified according to their grade point average and randomly assigned to four groups of equal size." (Abstract, p. 3) 2) "The 177 nursing students who met the inclusion criteria were stratified according to cumulative grade point average and randomly allocated to one control group and three intervention groups using a block randomization method." (p. 7) 3) "Following randomization, participants were allocated to separate Microsoft Teams classrooms: Group I (n = 44), Group II (n = 44), Group III (n = 45), and Group IV (n = 44)" (p. 7) Detailed Analysis: The ERCT 'C' criterion requires randomization by entire class or school (unless the intervention is one-to-one tutoring). Here the unit of randomization is the individual student: 177 students enrolled in a single obstetrics and gynecology course at one nursing department were individually block-randomized (stratified by GPA) into four groups. Although participants were subsequently placed into separate Microsoft Teams "classrooms," these classrooms were created after student-level allocation and do not represent pre-existing classes that were randomized as intact units. The intervention (game-based learning and telesimulation delivered to groups) is not one-to-one personal tutoring, so the tutoring exception does not apply. Criterion C is not met because randomization was carried out at the individual student level within a single class/course rather than at the class or school level.
  • E

    Exam-based Assessment

    • The primary knowledge and skills outcomes were measured with instruments the researchers developed themselves, not with a widely recognized standardized exam.
    • "The 25-item original form, developed by the researchers, was created based on a review of the literature and in consideration of the learning outcomes of the obstetrics and gynecology nursing course" (p. 12)
    • Relevant Quotes: 1) "The 25-item original form, developed by the researchers, was created based on a review of the literature and in consideration of the learning outcomes of the obstetrics and gynecology nursing course (1-3,22). ... The revised form consisted of 20 items." (pp. 12-13) 2) "The PPH telesimulation skills evaluation form was developed by the researchers based on the literature (1-3)." (p. 13) 3) "The interview employed in the telesimulation was specifically developed for this study and has neither been used nor published elsewhere previously" (p. 11) 4) "PPH telesimulation skills evaluation form and scoring have been developed by researchers. Validity and reliability analyses were not conducted" (p. 21, Limitations) 5) "Motivation Scale Used in Teaching Materials was developed by Keller et al. in 2006 ... IMMS was translated into Turkish by Acar (2009)." (p. 14) 6) "The SECPS scale was developed by Cheraghi et al. (2009) and translated into Turkish by Pozam and Zaybak (2015)." (p. 14) Detailed Analysis: Criterion E requires a standardised, widely recognised exam-based assessment that was not custom-built for the study. The two primary academic outcome instruments here — the 20-item Knowledge of PPH Management form and the 30-item PPH Telesimulation Skills Evaluation form — were both developed by the researchers specifically for this study; the skills form was not even subjected to validity and reliability analysis. The IMMS and SECPS are previously validated scales, but they measure motivation and self-efficacy rather than being standardised exam-based measures of academic achievement. No national, state-wide, or otherwise externally standardised examination was used. Criterion E is not met because the knowledge and skills outcomes relied on researcher-designed, study-specific instruments rather than a recognised standardised exam.
  • T

    Term Duration

    • The entire intervention and outcome measurement spanned only about two weeks, far shorter than one full academic term.
    • "Following the completion of the theoretical course and Kahoot activities, a telesimulation intervention was conducted the following week for Groups II and IV." (p. 8)
    • Relevant Quotes: 1) "One week after the completion of the theoretical course, the Kahoot game-based learning intervention was initiated for Groups III and IV." (p. 8) 2) "Following the completion of the theoretical course and Kahoot activities, a telesimulation intervention was conducted the following week for Groups II and IV." (p. 8) 3) "Pretest measurements were obtained prior to the theoretical instruction, and posttest assessments were administered after completion of all interventions using a standardized time interval across all study groups." (p. 12) 4) Figure 2 shows the virtual lecture at Week 0, Kahoot at Week 1, telesimulation at Week 2, and the final knowledge/self-efficacy/motivation post-tests applied to all groups at Week 2. (Figure 2, p. 37) Detailed Analysis: Criterion T requires the interval from intervention start to primary outcome measurement to be at least one full academic term (~3-4 months). The timeline here is: virtual lecture in Week 0, Kahoot over five consecutive days in Week 1, telesimulation in Week 2, and the post-tests immediately after all interventions in Week 2. The total span from intervention start to outcome measurement is roughly two weeks, with no longer-term follow-up. This is far shorter than one academic term. Criterion T is not met because outcomes were measured only about two weeks after the intervention began, well short of a full term.
  • D

    Documented Control Group

    • The control group's size, demographic profile, baseline scores, and condition (virtual lecture only) are clearly documented and shown to be comparable to the intervention groups.
    • "There was no significant difference between the intervention and control groups regarding age, gender, economic status, desire to be a nurse, and place of residence (p>0.05)." (p. 16)
    • Relevant Quotes: 1) "The study groups were defined as follows: Group I (virtual lecture only) ... Group I (n = 44)" (p. 7) 2) "The participants' average age is 20.1±0.9 years, and the majority (87.8%) are women. There was no significant difference between the intervention and control groups regarding age, gender, economic status, desire to be a nurse, and place of residence (p>0.05)." (p. 16) 3) "No statistically significant differences were found between the groups in terms of participation in the simulation application, use of digital tools for course-related activities, playing digital games ... and prior awareness of Kahoot (p > 0.05)." (p. 16) 4) "No significant difference was found in the pre-test scores between all groups from the knowledge of PPH management form (p=0.282)." (p. 16) 5) CONSORT diagram: Group I "Allocated to Control (n=44) ... Received allocated intervention (n=44) PPH Virtual Lecture ... Lost to follow-up (n=2) ... Analysed (n=42)." (Figure 1, p. 38) Detailed Analysis: Criterion D requires that the control group be well-documented, including its size, demographic characteristics, baseline performance, and the conditions it received. The paper identifies the control group (Group I) precisely, reports its allocated and analysed sample sizes (n=44 allocated, n=42 analysed), documents that it received the same virtual lecture as all groups but no additional intervention, and demonstrates baseline comparability on demographics, technology-use characteristics, and pre-test knowledge scores. This is sufficient documentation of the control group. Criterion D is met because the control group's size, demographic and baseline characteristics, and condition (virtual lecture only) are clearly described and shown to be comparable to the intervention groups.

• Level 2 Criteria

  • S

    School-level RCT

    • Randomization occurred at the individual student level within one nursing department, not across schools or institutions.
    • "The 177 nursing students who met the inclusion criteria were ... randomly allocated to one control group and three intervention groups using a block randomization method." (p. 7)
    • Relevant Quotes: 1) "This randomized-controlled study comprised 177 nursing students who had attended an obstetrics and gynecology nursing course at a nursing department in Ankara, Türkiye." (Abstract, p. 3) 2) "The 177 nursing students who met the inclusion criteria were stratified according to cumulative grade point average and randomly allocated to one control group and three intervention groups using a block randomization method." (p. 7) Detailed Analysis: Criterion S requires randomization at the school level, i.e. entire educational institutions or implementing units randomly assigned to conditions. This study was conducted within a single nursing department at one university in Ankara, and the unit of randomization was the individual student. No schools, institutions, or sites were randomly assigned. Criterion S is not met because the study randomized individual students within one institution rather than randomizing at the school level.
  • I

    Independent Conduct

    • The same two researchers designed the interventions and the assessment instruments and also conducted the study and analysis, with no independent or third-party evaluator.
    • "The PPH telesimulation scenario was collaboratively developed by two researchers using the simulation design template proposed by Ünver and Başak (2016)" (p. 9)
    • Relevant Quotes: 1) "Two researchers developed five tests, which were based on the learning objectives of the PPH training program" (p. 9) 2) "The PPH telesimulation scenario was collaboratively developed by two researchers using the simulation design template proposed by Ünver and Başak (2016) (20)." (p. 9) 3) "The 25-item original form, developed by the researchers, was created based on a review of the literature" (p. 12) 4) "Authors' contributions: Conception and design: YÇŞ, GG; Data acquisition and interpretation YÇŞ, GG; Drafting of the manuscript ... YÇŞ, GG." (p. 24) Detailed Analysis: Criterion I requires that the study be conducted independently from those who designed the intervention, to reduce bias in implementation, measurement, and analysis. Here the same two authors developed the Kahoot tests, the telesimulation scenario, and the knowledge and skills assessment forms, and — per the author contributions statement — also carried out data acquisition, interpretation, and analysis. Additionally, the facilitator scored student skill performance during the telesimulation. There is no statement of any external evaluator, blinded assessor, or third-party body independent of the intervention designers. Criterion I is not met because the intervention designers also implemented, assessed, and analyzed the study with no independent oversight.
  • Y

    Year Duration

    • With a total intervention-to-measurement span of about two weeks (and term duration already unmet), the study falls far short of the year-duration requirement.
    • "posttest assessments were administered after completion of all interventions using a standardized time interval across all study groups." (p. 12)
    • Relevant Quotes: 1) "One week after the completion of the theoretical course, the Kahoot game-based learning intervention was initiated for Groups III and IV." (p. 8) 2) "Following the completion of the theoretical course and Kahoot activities, a telesimulation intervention was conducted the following week for Groups II and IV." (p. 8) 3) Figure 2 places the virtual lecture at Week 0 and the final post-tests at Week 2. (Figure 2, p. 37) Detailed Analysis: Criterion Y requires outcomes to be measured at least about 75% of a full academic year (~9-10 months) after the intervention begins. Per the prompt's rule, if Criterion T (Term Duration) is not met, Y cannot be met. Here T is not met, and the overall span from intervention start to post-test is roughly two weeks with no long-term follow-up. Criterion Y is not met because the tracking interval was only about two weeks, far short of a full academic year, and the weaker term-duration criterion was already not met.
  • B

    Balanced Control Group

    • The additional engagement time is inseparable from the Kahoot and telesimulation methods that are themselves the treatment being tested against a business-as-usual virtual-lecture control, though the authors acknowledge exposure time differed across groups.
    • "Within the scope of the research, all groups received the theoretical course on Postpartum Hemorrhage and Management on the same day, with the same instructor, the same content, and the same duration" (p. 8)
    • Relevant Quotes: 1) "Within the scope of the research, all groups received the theoretical course on Postpartum Hemorrhage and Management on the same day, with the same instructor, the same content, and the same duration, using a synchronous distance learning method ... This approach was chosen to ensure temporal and content equivalence in terms of initial knowledge levels among the groups." (p. 8) 2) "Students spent a total of 20 minutes playing each game and were allowed to play the game only once." (p. 9); "A PIN code was sent to Group III and Group IV students via WhatsApp every day for 5 days." (p. 9) 3) "The telesimulation program took 30 minutes in total." (p. 10) 4) "the amount of time students spent engaging with the educational materials differed across groups. Accordingly, the present study was designed to explore the potential effects of spaced repetition on learning rather than to isolate the individual effects of Kahoot and telesimulation on postpartum hemorrhage training." (p. 21) 5) "Control schools" analogue here: "Group I (virtual lecture only)" (p. 7). Detailed Analysis: Comparing resources across conditions: every group, including the control (Group I), received the identical virtual lecture (same instructor, content, and duration), so the shared baseline instruction is balanced. Beyond that shared lecture, the intervention groups received additional activities that carry additional engagement time — Kahoot games over five consecutive days (Groups III and IV) and/or a 30-minute telesimulation with debriefing (Groups II and IV). The control group received no time-matched substitute activity. This is a real difference in educational time and is not negligible. Applying the ERCT decision logic: the additional time is not a separable, optional add-on layered on top of a common method — it is intrinsic to the very interventions being tested. Game-based learning simply is the Kahoot sessions, and telesimulation simply is the telesimulation session; these methods cannot be delivered without their associated engagement time. The study explicitly tests these methods against a business-as-usual virtual-lecture condition, which parallels the ERCT exception examples (e.g., DPL devices/support and GBG activities that are integral to the intervention tested against business-as-usual). Under that integral-resource rule, the unmatched time does not by itself violate Criterion B. It must be flagged, however, that the authors themselves note that engagement time differed across groups and that they could not isolate the methods' effects from spaced repetition/exposure, which is precisely the confound Criterion B guards against. Criterion B is met because the extra engagement time is an integral, inseparable component of the game-based learning and telesimulation methods being tested against a business-as-usual virtual-lecture control, while noting the authors' acknowledgment that exposure time was not equated across groups.

• Level 3 Criteria

  • R

    Reproduced

    • This specific combined game-based learning plus telesimulation intervention has not been independently replicated; the authors state no comparable study exists.
    • "no studies using Kahoot, telesimulation, or these methods together were found in the literature." (p. 20)
    • Relevant Quotes: 1) "To the best of our knowledge, the combined use of telesimulation and gaming/web-based learning, including serious games, has not previously been examined in nursing education." (p. 5) 2) "Although no study in the literature uses telesimulation and game-based learning methods in PPH management training" (p. 18) 3) "The study revealed that Group IV, which utilized a combination of Kahoot and telesimulation, demonstrated significantly higher PPH knowledge, skill, IMMS, and SECPS scores than all other groups. However, no studies using Kahoot, telesimulation, or these methods together were found in the literature." (p. 20) Detailed Analysis: Criterion R requires independent replication of the study by a different research team in a different context, published in a peer-reviewed journal. The authors repeatedly state that the combined use of telesimulation and game-based learning in nursing (and specifically in PPH management) has not previously been examined, and that no comparable studies exist. This is a very recent study (BMC Nursing, article in press 2026) of a novel combined intervention, so an independent reproduction could not plausibly exist yet; an internet check found no replication of this specific combined intervention and design. While individual components (Kahoot, telesimulation) have been studied separately by others, those are not replications of this specific combined intervention. No independent reproduction of this trial was identified. Criterion R is not met because there is no independent replication of this specific combined intervention.
  • A

    All-subject Exams

    • Only the single topic of postpartum hemorrhage was assessed, and the underlying exam-based criterion (E) is not met.
    • "the study only examined one subject of the gynecology and obstetrics nursing course." (p. 21)
    • Relevant Quotes: 1) "In this study, we aimed to evaluate the effects of game-based learning and telesimulation methods on the knowledge, skills, motivation, and self-efficacy levels of nursing students in postpartum hemorrhage management." (Abstract, p. 3) 2) "Second, the study only examined one subject of the gynecology and obstetrics nursing course. To further enhance the understanding of nursing skills, additional studies should be conducted on other topics." (p. 21) Detailed Analysis: Criterion A requires that impact be measured across all main subjects using standardised exam-based assessments, and it is explicitly conditional on Criterion E: if E is not met, A cannot be met. Here E is not met (the outcomes used researcher-designed instruments). Furthermore, the study measured outcomes only for a single narrow topic — postpartum hemorrhage management — and the authors themselves note the study examined only one subject of the course. No broader curriculum subjects were assessed. Criterion A is not met because only one topic was assessed and the prerequisite standardised exam-based criterion (E) is not satisfied.
  • G

    Graduation Tracking

    • Outcomes were collected immediately after the intervention with no follow-up to graduation, and the prerequisite year-duration criterion (Y) is not met.
    • "posttest assessments were administered after completion of all interventions using a standardized time interval across all study groups." (p. 12)
    • Relevant Quotes: 1) "After these interventions, participants in all groups were asked to complete the post-test, which consisted of the knowledge of PPH management form and the self-efficacy in clinical performance scale." (p. 12) 2) "posttest assessments were administered after completion of all interventions using a standardized time interval across all study groups." (p. 12) Detailed Analysis: Criterion G requires tracking participants through to graduation from their educational stage, and it is explicitly conditional on Criterion Y: if Y is not met, G cannot be met. Here Y is not met. In addition, outcomes were measured immediately after the roughly two-week intervention, with no follow-up phase, and there is no mention of any planned or subsequent tracking of the cohort to graduation. An internet check found no follow-up publication by the same authors tracking this cohort to graduation. Criterion G is not met because there was no follow-up to graduation and the prerequisite year-duration criterion (Y) is not satisfied.
  • P

    Pre-Registered

    • The trial was registered retrospectively, after data collection, so it does not meet the pre-registration timing requirement.
    • "Trial registration: ClinicalTrials.gov ID, NCT05323903, Retrospectively registered. Registered on April 5, 2022" (p. 4)
    • Relevant Quotes: 1) "Trial registration: ClinicalTrials.gov ID, NCT05323903, Retrospectively registered. Registered on April 5, 2022" (p. 4) 2) "This was a randomized controlled trial registered on ClinicalTrials.gov (NCT05323903/2022-04-05)." (p. 6) 3) "The study population comprised 180 nursing students who took an obstetrics and gynecology nursing course ... during the 2020-2021 academic year." (p. 6) 4) "we revised the form and conducted a pilot study on 52 nursing students on April 3, 2021." (p. 13); ethics approval "(25.02.2021-2021/04-2021-109)" (p. 23) Detailed Analysis: Criterion P requires the full protocol to be pre-registered before data collection begins. The paper states plainly that the trial was "Retrospectively registered" on April 5, 2022, whereas the study was conducted during the 2020-2021 academic year, with ethics approval in February 2021 and the pilot in April 2021. An internet check of the ClinicalTrials.gov record for NCT05323903 confirms this: Study First Submitted April 5, 2022 and First Posted April 12, 2022 — after the reported data collection. Registration therefore occurred after data collection, not before it. Criterion P is not met because the trial was registered retrospectively, after data collection had been completed.