Evaluation of an educational program for nurses on the care coordination of pleural mesothelioma patients: A pilot randomized controlled trial

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomization was at the individual nurse (participant) level, not at a class/site level, and no tutoring-style exception applies.
    • "Participants were randomly assigned in a 1:1 ratio to either the intervention or control group using random permuted block sizes of 4." (p. 6 of 12)
    • Relevant Quotes: 1) "This study was an open-labeled randomized controlled trial (pre-test and post-test design) with two parallel groups at 1:1 ratio." (p. 2 of 12) 2) "Sixty participants were randomly assigned to the intervention group (n = 30; with program) and control group (n = 30; without program)." (p. 1 of 12) 3) "Participants were randomly assigned in a 1:1 ratio to either the intervention or control group using random permuted block sizes of 4." (p. 6 of 12) 4) "The randomization sequence was generated by the principal investigator (corresponding author) using the cloud service Mujinwari (https://autoassign.mujinwari.biz/)." (p. 6 of 12) Detailed Analysis: Criterion C (Class-level RCT) requires random assignment at the level of intact classes (or stronger, such as whole schools) to reduce contamination, with a narrow exception for one-to-one tutoring/personal teaching interventions. Here, participants are nurses who were randomized as individuals. There is no indication that intact hospital units, sites, or workshop cohorts were randomized as groups. The intervention is also not a one-to-one tutoring/personal teaching intervention, so the tutoring exception does not apply. Criterion C is not met because the unit of randomization is the individual nurse, not an intact class/site (or stronger unit), and no tutoring exception is stated.
  • E

    Exam-based Assessment

    • Outcomes were measured using researcher-developed and self-reported instruments rather than standardized exams.
    • "In this randomized controlled study relative to the CCOM program, we measured the self-reported total scores of knowledge, attitude, and confidence of nurses before (pre-test) and after (post-test) the CCOM program." (p. 1 of 12)
    • Relevant Quotes: 1) "In this randomized controlled study relative to the CCOM program, we measured the self-reported total scores of knowledge, attitude, and confidence of nurses before (pre-test) and after (post-test) the CCOM program." (p. 1 of 12) 2) "The three outcome measures [(1) Knowledge of disease and treatment of PM, (2) Attitude toward care coordination of PM patients, (3) Confidence in the care coordination of PM patients] were originally developed by the research team (Nagamatsu et al., 2025) based on the theoretical model of self-efficacy by Bandura and Walters (1977) which was used as the framework for the present study to implement the educational program to improve nurses' care coordination skills." (p. 3 of 12) 3) "(2) Attitude toward care coordination of PM patients (Attitude): Original 10 items; 5-point Likert scale (1 = strongly agree, 5 = never agree)." (p. 4 of 12) 4) "(3) Confidence in the care coordination of PM patients (Confidence): Original 10 items; 5-point Likert scale (1 = not confident, 5 = confident)." (p. 4 of 12) Detailed Analysis: Criterion E requires standardized, widely recognized exam-based assessments rather than instruments created for the study or self-reported Likert scales. The paper explicitly describes the outcomes as "self-reported" and states that the three measures were "originally developed by the research team." The attitude and confidence measures are explicitly described as original Likert-scale instruments, not standardized exams used broadly outside the study. Criterion E is not met because outcomes rely on researcher- developed, self-reported measures rather than standardized exams.
  • T

    Term Duration

    • Outcomes were measured immediately at program end (or after about two weeks for controls), not at least one academic term after intervention start.
    • "The intervention group answered at the post-test form at the end of the education program, and the control group answered the post-test 2 weeks after the pre-test." (p. 4 of 12)
    • Relevant Quotes: 1) "This study was conducted from November 2023 to December of 2023." (p. 4 of 12) 2) "The CCOM program for the intervention group was conducted three times in a university in Tokyo and once in a private hospital in Osaka from November 25 to December 16, 2023." (p. 4 of 12) 3) "The intervention group answered at the post-test form at the end of the education program, and the control group answered the post-test 2 weeks after the pre-test." (p. 4 of 12) 4) "The effectiveness of the CCOM program was assessed only at two measurement points: pre-test and post-test." (p. 10 of 12) Detailed Analysis: Criterion T requires measuring outcomes at least one academic term (about 3-4 months) after the intervention begins. The paper states the study was conducted from November to December 2023. It further states that the intervention group's post-test occurred "at the end of the education program," while the control group's post-test was "2 weeks after the pre-test." This is far shorter than a full academic term, and the authors confirm only two measurement points (pre and post), indicating there was no term-long follow-up. Criterion T is not met because measurement occurred at program end (or ~2 weeks later), far short of a full academic term.
  • D

    Documented Control Group

    • The control group is clearly described (no education), with group sizes and baseline characteristics reported in tables.
    • "The control group received no education." (p. 3 of 12)
    • Relevant Quotes: 1) "Sixty participants were randomly assigned to the intervention group (n = 30; with program) and control group (n = 30; without program)." (p. 1 of 12) 2) "The control group received no education." (p. 3 of 12) 3) "Fifty-eight participants completed the study (intervention group, 28; control group, 30)." (p. 1 of 12) 4) "The characteristics of the participants in the intervention and control groups are shown in Table 5." (p. 7 of 12) 5) "Mean scores on measurements (pre-test) Knowledge ... Attitude ... Confidence ..." (Table 5, p. 7 of 12) Detailed Analysis: Criterion D requires a well-documented control group, including what they received and baseline characteristics. The paper states the control group was "without program" and explicitly that it "received no education." It also provides sample sizes (assigned and analyzed) and presents baseline demographic and baseline outcome information in Table 5, including pre-test knowledge, attitude, and confidence. Criterion D is met because the control condition and baseline characteristics are explicitly reported (including tables).

• Level 2 Criteria

  • S

    School-level RCT

    • Randomization was not at the school/site level; individual nurses were randomized.
    • "Participants were randomly assigned in a 1:1 ratio to either the intervention or control group using random permuted block sizes of 4." (p. 6 of 12)
    • Relevant Quotes: 1) "Sixty participants were randomly assigned to the intervention group (n = 30; with program) and control group (n = 30; without program)." (p. 1 of 12) 2) "Participants were randomly assigned in a 1:1 ratio to either the intervention or control group using random permuted block sizes of 4." (p. 6 of 12) 3) "The CCOM program for the intervention group was conducted three times in a university in Tokyo and once in a private hospital in Osaka from November 25 to December 16, 2023." (p. 4 of 12) Detailed Analysis: Criterion S requires that schools (or equivalent implementation units/sites) be the unit of randomization. Although the program was delivered at multiple venues, the randomization described is for "participants" (nurses) rather than venues or institutions. The paper provides no statement that universities/hospitals (or departments) were randomized; instead, individuals were assigned. Criterion S is not met because allocation occurred at the level of individual nurses rather than institutions/sites.
  • I

    Independent Conduct

    • The research team developed and implemented the program and the PI generated the randomization sequence, with no external evaluator documented.
    • "The intervention group received the CCOM program developed and previously pilot-tested by the research team (Nagamatsu et al., 2025)." (p. 2 of 12)
    • Relevant Quotes: 1) "The intervention group received the CCOM program developed and previously pilot-tested by the research team (Nagamatsu et al., 2025)." (p. 2 of 12) 2) "The randomization sequence was generated by the principal investigator (corresponding author)..." (p. 6 of 12) 3) "Yasuko Nagamatsu had primary responsibility for conducting the study." (p. 10 of 12) 4) "Nagamatsu contributed to the conceptualization and study design, development and implementation of the educational program, data collection and analysis..." (p. 10 of 12) 5) "All assessments were self-reported and data analysis was blinded to the group allocation." (p. 6 of 12) Detailed Analysis: Criterion I requires independent conduct to reduce bias, such as third-party implementation/evaluation or a clear separation between intervention developers and evaluators. The paper states the program was developed by the research team, and that the principal investigator generated the randomization sequence. The author contributions state the same team developed and implemented the program and also conducted data collection and analysis. While analysis was "blinded to the group allocation," there is no explicit third-party evaluator or external oversight documented. Criterion I is not met because the intervention developers also conducted key trial activities, and no external evaluator is documented.
  • Y

    Year Duration

    • The study ran only from November to December 2023, far less than 75% of an academic year; per ERCT rules, Y is also blocked because T is not met.
    • "This study was conducted from November 2023 to December of 2023." (p. 4 of 12)
    • Relevant Quotes: 1) "This study was conducted from November 2023 to December of 2023." (p. 4 of 12) 2) "The intervention group answered at the post-test form at the end of the education program, and the control group answered the post-test 2 weeks after the pre-test." (p. 4 of 12) 3) "The effectiveness of the CCOM program was assessed only at two measurement points: pre-test and post-test." (p. 10 of 12) Detailed Analysis: Criterion Y requires outcomes to be measured at least ~75% of an academic year after the intervention begins. The paper indicates the study occurred within November-December 2023, with post-testing at program end (or two weeks later), which is far short of a year-length tracking period. Additionally, per the ERCT instruction, if criterion T is not met then criterion Y is not met; the quoted timing confirms T is not met. Criterion Y is not met because follow-up was weeks, not most of a year, and T is not satisfied.
  • B

    Balanced Control Group

    • The intervention added substantial instructional time/materials, but these resources are integral to the treatment being tested (receiving the CCOM program vs no program).
    • "The control group received no education." (p. 3 of 12)
    • Relevant Quotes: 1) "The CCOM program consisted of a care guide and nine study videos featuring various aspects of PM (2 h 15 min) and a face-to-face workshop (3 h 30 min)." (p. 1 of 12) 2) "The control group received no education." (p. 3 of 12) 3) "This study aimed to evaluate the effects of the care coordination of pleural mesothelioma program (CCOM program), an educational program that we developed for nurses..." (p. 1 of 12) 4) "The contents of the videos (2 h and 15 min) and face-to-face workshop (3 h and 30 min) are shown in Table 1." (p. 3 of 12) Detailed Analysis: Criterion B compares the nature, quantity, and quality of resources (time, materials, support) provided to intervention and control conditions, and asks whether the control provides a comparable substitute for the intervention's inputs, unless the additional resources are explicitly the treatment variable. The intervention clearly provides extra educational resources: a guide, 2 h 15 min of videos, and a 3 h 30 min workshop. The control group "received no education," so the inputs are not matched. However, the study explicitly tests the effect of providing the CCOM educational program versus not providing it. The additional time/materials are not an accidental add-on; they define the treatment package being evaluated. Criterion B is met because the extra instructional resources are integral to (and define) the intervention being tested (program vs no program).

• Level 3 Criteria

  • R

    Reproduced

    • No independent replication of this specific CCOM program trial was found.
    • Relevant Quotes: 1) "The intervention group received the CCOM program developed and previously pilot-tested by the research team (Nagamatsu et al., 2025)." (p. 2 of 12) Detailed Analysis: Criterion R requires an independent replication by a different research team in a different context, published in a peer- reviewed scientific journal. The paper references a prior pilot test by the same research team and reports this single pilot RCT. It does not report any independent replication, and a targeted literature search did not identify a peer-reviewed independent replication of this specific CCOM program RCT. Criterion R is not met because independent replication of this specific trial was not found.
  • A

    All-subject Exams

    • Criterion E is not met (no standardized exams), so A is not met, and outcomes were not broad core-subject exams.
    • Relevant Quotes: 1) "In this randomized controlled study relative to the CCOM program, we measured the self-reported total scores of knowledge, attitude, and confidence of nurses..." (p. 1 of 12) 2) "The three outcome measures [(1) Knowledge of disease and treatment of PM, (2) Attitude toward care coordination of PM patients, (3) Confidence in the care coordination of PM patients] were originally developed by the research team (Nagamatsu et al., 2025)..." (p. 3 of 12) Detailed Analysis: Criterion A requires all-subject standardized exams and is explicitly dependent on Criterion E being met. Here, outcomes are self-reported and researcher-developed, not standardized exams. In addition, the outcomes are specific to PM care coordination knowledge/attitudes/confidence rather than broad curriculum-wide exam performance across subjects. Criterion A is not met because standardized exam-based assessment is absent (E fails), and no all-subject exam framework exists.
  • G

    Graduation Tracking

    • There is no tracking to any graduation milestone, and per ERCT rules G is also blocked because Y is not met.
    • "The effectiveness of the CCOM program was assessed only at two measurement points: pre-test and post-test." (p. 10 of 12)
    • Relevant Quotes: 1) "The effectiveness of the CCOM program was assessed only at two measurement points: pre-test and post-test." (p. 10 of 12) 2) "Future studies should include follow-up assessments after a certain period to examine the sustained effects of the program." (p. 10 of 12) Detailed Analysis: Criterion G requires follow-up tracking until graduation (for the relevant educational stage). This paper evaluates short-term pre/post changes and explicitly notes only two measurement points (pre-test and post-test), while recommending future follow-up. Additionally, per ERCT rules, if criterion Y is not met then criterion G is not met; this study is far shorter than a year, confirming Y is not met. Criterion G is not met because the study ends at post-test with no long-term tracking to a graduation milestone, and Y is not met.
  • P

    Pre-Registered

    • The protocol was registered in UMIN-CTR, and the registry shows a registered date before the trial's anticipated start and the paper's data-collection period.
    • "Registered date" (UMIN-CTR entry for UMIN000052220) and "2023 Year 09 Month 15 Day."
    • Relevant Quotes: 1) "The trial protocol was registered with the University Hospital Medical Information Network Clinical Trials Registry (UMIN-CTR). The Trial registration number is ID: UMIN000052220." (p. 2 of 12) 2) "Registered date" (UMIN-CTR entry for UMIN000052220). 3) "2023 Year 09 Month 15 Day" (UMIN-CTR entry for UMIN000052220). 4) "Anticipated trial start date" (UMIN-CTR entry for UMIN000052220). 5) "2023 Year 11 Month 02 Day" (UMIN-CTR entry for UMIN000052220). 6) "This study was conducted from November 2023 to December of 2023." (p. 4 of 12) Detailed Analysis: Criterion P requires a publicly accessible pre-registered protocol, with evidence that registration occurred before data collection began. The paper states the trial protocol was registered with UMIN-CTR and provides the trial ID (UMIN000052220). The corresponding UMIN-CTR record lists a "Registered date" of "2023 Year 09 Month 15 Day" and an "Anticipated trial start date" of "2023 Year 11 Month 02 Day." The paper states data collection occurred from November to December 2023. Because the registry's registered date (2023-09-15) precedes the anticipated start date (2023-11-02) and the paper's stated conduct period (Nov-Dec 2023), the protocol appears to have been registered before the study began. Criterion P is met because the trial is publicly registered and the registry date precedes the study period.