Challenges in educating nursing students on mechanical ventilation: the case for interactive mobile learning tools – a randomized controlled trial

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomization was at the individual student level within one cohort (not class- or school-level), and the study is not a one-to-one tutoring exception.
    • "Participants were randomly allocated to either the intervention or control group." (p. 7)
    • Relevant Quotes: 1) "In this randomized controlled clinical trial, all final-year students (7th and 8th semester) from the School of Nursing at Mashhad University of Medical Sciences (MUMS) were considered eligible (n=104). Participants were randomly allocated to either the intervention or control group." (p. 7) 2) "Since all participants were drawn from the same nursing class, informal interactions and information sharing between the intervention and control groups outside the clinical setting could have diluted the observed effect size." (p. 30-31) Detailed Analysis: ERCT criterion C requires randomisation at the class level (or stronger) to reduce contamination across participants who share the same learning environment. The paper explicitly states that "participants" (students) were allocated to groups, and it also acknowledges contamination risk because participants were from the same nursing class. The tutoring/personal-teaching exception does not apply here because the intervention is app-based training (not one-to-one tutoring delivered to isolated pairs). Final summary sentence: Criterion C is not met because the unit of randomisation is individual students in one cohort, and contamination risk is explicitly acknowledged.
  • E

    Exam-based Assessment

    • Outcomes relied on researcher-developed knowledge and OSCE-based performance instruments rather than widely recognized standardized exams.
    • "Knowledge was assessed using a 15‑item researcher‑developed multiple‑choice test (score range 0–100)..." (p. 14-15)
    • Relevant Quotes: 1) "The primary outcome was clinical performance, measured using a researcher‑developed 52‑item OSCE‑based Performance Assessment Tool." (p. 14) 2) "Knowledge was assessed using a 15‑item researcher‑developed multiple‑choice test (score range 0–100) covering ventilator weaning, extubation, and re‑intubation." (p. 14-15) Detailed Analysis: ERCT criterion E requires standardized, widely recognized exam-based assessments (i.e., external exams not created for the study). This paper reports researcher-developed outcome instruments: a researcher-developed OSCE-based performance tool and a researcher-developed multiple-choice knowledge test. While the paper reports psychometric properties for these tools, they are still custom instruments rather than standardized exams. Final summary sentence: Criterion E is not met because the primary educational outcomes were measured with researcher-developed assessments rather than standardized external exams.
  • T

    Term Duration

    • The intervention lasted one week and the longest reported follow-up was one month, which is shorter than a full academic term.
    • "The intervention consisted of one week of app-based training." (p. 2)
    • Relevant Quotes: 1) "The intervention consisted of one week of app-based training." (p. 2) 2) "It was assessed at three time points: a baseline OSCE, a post‑intervention OSCE one week later, and direct observation of real ICU performance one month afterward during routine clinical rotations." (p. 14) Detailed Analysis: ERCT criterion T requires outcomes to be measured at least one full academic term (typically ~3-4 months) after the intervention begins. In this study, the described follow-up includes a post-intervention OSCE one week later and a clinical observation one month afterward. Even using the longest follow-up ("one month afterward"), the tracking duration is substantially shorter than a term. Final summary sentence: Criterion T is not met because the longest start-to-outcome interval described is about one month, not a full term.
  • D

    Documented Control Group

    • The control group is clearly described (non-interactive app), with sample sizes and baseline demographics reported for comparison.
    • "Seventy-one students were randomly assigned to the interactive i-BreathGuard group (n=36) or the non-interactive control group (n=35)." (p. 2)
    • Relevant Quotes: 1) "Seventy-one students were randomly assigned to the interactive i-BreathGuard group (n=36) or the non-interactive control group (n=35)." (p. 2) 2) "Students in the control group accessed the same content in the non interactive version (n BreathGuard)." (p. 12) 3) "According to the results of the Mann-Whitney U and Chi-Square tests, there was no significant difference between the two educational application groups in terms of age and gender [Table 1]." (p. 15) Detailed Analysis: ERCT criterion D requires that the control group be well documented, including (at minimum) what the control group received and baseline comparability information. The paper clearly defines the control as the non-interactive app version, provides group sizes (n=35 control), and reports baseline comparisons for age and gender, including a referenced table. Final summary sentence: Criterion D is met because the control condition and baseline characteristics are described with group sizes and baseline demographic comparisons.

• Level 2 Criteria

  • S

    School-level RCT

    • Participants were randomized as individual students in a single university program, not by school/site.
    • "The study was conducted in the nursing skills laboratory and several affiliated ICUs at Mashhad University of Medical Sciences." (p. 2)
    • Relevant Quotes: 1) "The study was conducted in the nursing skills laboratory and several affiliated ICUs at Mashhad University of Medical Sciences." (p. 2) 2) "In this randomized controlled clinical trial, all final-year students (7th and 8th semester) from the School of Nursing at Mashhad University of Medical Sciences (MUMS) were considered eligible (n=104). Participants were randomly allocated to either the intervention or control group." (p. 7) Detailed Analysis: ERCT criterion S requires randomisation at the school (or site) level. The paper describes a single-institution study (Mashhad University of Medical Sciences) and allocation of individual students to groups. There is no evidence that multiple schools/sites were randomized. Final summary sentence: Criterion S is not met because randomisation was among students within one institution rather than among schools/sites.
  • I

    Independent Conduct

    • The authors’ team developed the intervention and ran the study; an independent statistician and blinding help internal validity but do not demonstrate an independent, third-party evaluation.
    • "the design for this interactive educational application was developed by a team of intensive care specialists, nursing faculty members, medical informatics expert, and a software engineering team." (p. 10)
    • Relevant Quotes: 1) "Therefore, based on several brainstorming sessions, the design for this interactive educational application was developed by a team of intensive care specialists, nursing faculty members, medical informatics expert, and a software engineering team." (p. 10) 2) "Randomization was performed using simple randomization. An independent statistician generated the allocation sequence using a computerized random-number list (www.randomization.com)." (p. 7) 3) "Clinical assessors and the statistical analyst were blinded to group allocation." (p. 2) Detailed Analysis: ERCT criterion I requires that the study be conducted independently from the designers of the intervention. The paper indicates the application was developed by a team aligned with the project (specialists, faculty, informatics, engineering). While the trial includes safeguards (an independent statistician for allocation generation and blinding of assessors/analyst), these do not constitute an external, independent evaluation team that is clearly separate from the intervention designers/developers. Final summary sentence: Criterion I is not met because the intervention was developed by a team associated with the study and no independent third-party evaluation is clearly documented.
  • Y

    Year Duration

    • The reported follow-up is only up to one month, so the study does not approach 75% of an academic year.
    • "It was assessed at three time points... and direct observation of real ICU performance one month afterward..." (p. 14)
    • Relevant Quotes: 1) "The intervention consisted of one week of app-based training." (p. 2) 2) "It was assessed at three time points: a baseline OSCE, a post‑intervention OSCE one week later, and direct observation of real ICU performance one month afterward during routine clinical rotations." (p. 14) Detailed Analysis: ERCT criterion Y requires outcomes to be measured at least 75% of an academic year after the intervention begins. The paper’s longest follow-up time point is "one month afterward," which is far below the year-duration threshold. Final summary sentence: Criterion Y is not met because the study tracks outcomes for about one month, not most of an academic year.
  • B

    Balanced Control Group

    • Both groups received the same content and schedule; differences in instructor feedback and interaction are integral to the interactivity treatment being tested.
    • "Both groups received identical notifications, access schedules, and ranking views, ensuring comparability apart from the interactivity component." (p. 12)
    • Relevant Quotes: 1) "Students in the intervention group learned the educational content via the interactive version (i BreathGuard) for one week." (p. 11) 2) "Students in the control group accessed the same content in the non interactive version (n BreathGuard). The structure and visual interface were identical to the interactive version, but engagement elements (commenting, feedback, participation in interactive discussions, and real time instructor interaction) were removed." (p. 12) 3) "Both groups received identical notifications, access schedules, and ranking views, ensuring comparability apart from the interactivity component." (p. 12) Detailed Analysis: Criterion B examines whether the nature, quantity, and quality of resources (time, materials, staff attention) are balanced across groups, unless the extra resources are the treatment variable under test. Here, both groups had the same app content, the same schedule/duration ("for one week"), and the same interface structure. The primary difference is interactivity, including instructor feedback and real-time interaction. This could imply additional instructor time in the intervention condition; however, the paper frames these interactive elements as the defining treatment contrast (interactive vs non-interactive), and it explicitly states that other inputs were held comparable. Final summary sentence: Criterion B is met because the only meaningful resource differences are integral to the interactivity treatment being tested, while content/time/schedule are otherwise matched.

• Level 3 Criteria

  • R

    Reproduced

    • No independent replication of this specific i-BreathGuard vs n-BreathGuard RCT was found in available sources as of the ERCT check date.
    • "Larger, longitudinal studies are recommended to validate and extend these findings." (p. 3)
    • Relevant Quotes: 1) "Larger, longitudinal studies are recommended to validate and extend these findings." (p. 3) Detailed Analysis: Criterion R requires an independent reproduction of this specific study (or its central experimental claim) by a different research team, in a peer-reviewed outlet. Internet searching (by DOI, intervention name "i-BreathGuard" / "n-BreathGuard", and author names) did not identify a peer-reviewed replication study that explicitly repeats this specific intervention and comparison. Final summary sentence: Criterion R is not met because no independent, peer-reviewed replication of this specific RCT was found as of 2026-03-13.
  • A

    All-subject Exams

    • Because criterion E is not met (no standardized exams), criterion A cannot be met; additionally, outcomes are specific clinical competence measures rather than all core subjects.
    • Relevant Quotes: 1) "The primary outcome was clinical performance, measured using a researcher‑developed 52‑item OSCE‑based Performance Assessment Tool." (p. 14) 2) "Knowledge was assessed using a 15‑item researcher‑developed multiple‑choice test (score range 0–100)..." (p. 14-15) Detailed Analysis: ERCT criterion A requires all-subject standardized exams, and it depends on criterion E being met. Because this study uses researcher-developed assessments rather than standardized exams, criterion E is not met, so criterion A cannot be met. Final summary sentence: Criterion A is not met because criterion E is not met and the study does not use standardized exams across subjects.
  • G

    Graduation Tracking

    • The study’s follow-up ends at one month post-intervention, far short of tracking participants through graduation, and criterion Y is not met.
    • "It was assessed at three time points... and direct observation of real ICU performance one month afterward..." (p. 14)
    • Relevant Quotes: 1) "It was assessed at three time points: a baseline OSCE, a post‑intervention OSCE one week later, and direct observation of real ICU performance one month afterward during routine clinical rotations." (p. 14) 2) "Future studies should explore the long‑term effects of interactive educational apps on clinical skills and patient care outcomes to determine their sustained impact." (p. 31) Detailed Analysis: ERCT criterion G requires tracking participants until graduation. This paper reports follow-up only up to "one month afterward" and does not report tracking through graduation. Per the ERCT dependency rule, criterion G cannot be met if criterion Y is not met; this study clearly does not meet criterion Y. Internet searching for follow-up publications by the same author team on the i-BreathGuard cohort did not identify a paper reporting graduation tracking outcomes. Final summary sentence: Criterion G is not met because outcomes are measured only up to one month and there is no graduation tracking (and criterion Y is not met).
  • P

    Pre-Registered

    • The paper reports an IRCT registration and registration date, but the IRCT registry entry could not be accessed to verify registration timing relative to the actual start of data collection.
    • "Also this RCT was registered in the Iran Registry of Clinical Trials (IRCT code: IRCT20231001059571N1, https://irct.behdasht.gov.ir/trial/72883, Registration date: 2023-11-27)" (p. 33)
    • Relevant Quotes: 1) "Trial registration: this research was registered (27/11/2023) in the https://irct.behdasht.gov.ir with registration number: IRCT20231001059571N1." (p. 3) 2) "Also this RCT was registered in the Iran Registry of Clinical Trials (IRCT code: IRCT20231001059571N1, https://irct.behdasht.gov.ir/trial/72883, Registration date: 2023-11-27)" (p. 33) Detailed Analysis: ERCT criterion P requires that the full study protocol be pre-registered before data collection begins, and that this timing can be verified via the registry record (or equivalent documentation showing recruitment/data collection start date). The paper provides an IRCT ID and a registration date (2023-11-27). To verify pre-registration timing, the IRCT record needs to show a study start date / first enrollment date / recruitment start date that is after 2023-11-27. During this ERCT check, attempts to access the referenced IRCT registry page timed out, so the registry timing details could not be confirmed from the database. Final summary sentence: Criterion P is not met because registration is reported, but registry verification of timing relative to data collection start could not be completed.