Effect of gamified flipped classroom on improving nursing students' skills competency and learning motivation: a randomized controlled trial

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • The study randomized individual students, not entire classes or schools.
    • This study randomly divided the students into the study and control groups.
    • Relevant Quotes: 1) "This study adopted a randomized controlled study design with a pre-test and post-test and involved 128 nursing students at Mansoura University." (p. 1) 2) "This study randomly divided the students into the study and control groups." (p. 1) 3) "This study required an a priori sample size of 128 students, who were randomly divided into two groups (study and control) using block randomization with a block size of 4." (p. 3) Detailed Analysis: The ERCT standard requires randomization at the class or school level to minimize contamination, unless it's a personal tutoring intervention. The quotes clearly state that individual students were randomly assigned to the study or control group. The intervention (gamified flipped classroom) was not personal tutoring. Therefore, the randomization unit does not meet the criterion. Criterion C (Class-level RCT) is not met because randomization was performed at the student level.
  • E

    Exam-based Assessment

    • Assessments used researcher-developed quizzes and checklists, not standardized exams.
    • Three formative quizzes for each timepoint were developed by the researchers. Each consisted of 20 questions... Eight checklists were developed after reviewing the literature, nursing textbooks, and logbook of the Faculty of Nursing...
    • Relevant Quotes: 1) "Tool V: Fundamentals of nursing II knowledge tests Three formative quizzes for each timepoint were developed by the researchers." (p. 4) 2) "Each consisted of 20 questions: quiz 1 covered oral medication administration and intramuscular injection; quiz 2 covered range of motion exercises and subcutaneous and intradermal injections. However, quiz 3 covered glucocheck and heat and cold applications." (p. 4) 3) "Tool VI: Fundamentals of nursing II skill observation checklists Eight checklists were developed after reviewing the literature, nursing textbooks, and logbook of the Faculty of Nursing, Mansoura University..." (p. 4) Detailed Analysis: Criterion E requires the use of standardized, widely recognized exams, not assessments specially designed for the study. The paper explicitly states that the knowledge quizzes (Tool V) and skill observation checklists (Tool VI) were developed by the researchers. There is no mention of using any external, standardized exams for nursing skills or knowledge. Criterion E (Exam-based Assessment) is not met because the assessments were custom-made by the researchers.
  • T

    Term Duration

    • The intervention period was 6 weeks, shorter than a full academic term.
    • Then, after collecting baseline data, Moodle was gamified over the remaining 6 weeks for the students in the intervention group...
    • Relevant Quotes: 1) "In the intervention group, Moodle was gamified for 6 weeks." (p. 1) 2) "The assistant staff gathered data for 11 weeks at three timepoints in the second academic semester (2020–2021)... Timepoint 0 (T0) covers the first 2 weeks, and timepoints 1 (T1) and 2 (T2) cover the subsequent 3 weeks." (p. 5) 3) "Then, after collecting baseline data, Moodle was gamified over the remaining 6 weeks for the students in the intervention group..." (p. 5) Detailed Analysis: Criterion T requires outcomes to be measured at least one full academic term (approx. 3-4 months) after the intervention begins. The study collected baseline data (T0) over 2 weeks, then implemented the gamified intervention for the next 6 weeks (covering T1 and T2, each spanning 3 weeks). The total intervention duration was 6 weeks, which is shorter than a typical academic term. Data collection spanned 11 weeks in total, but the core intervention period where gamification occurred was only 6 weeks. Criterion T (Term Duration) is not met because the intervention duration was only 6 weeks.
  • D

    Documented Control Group

    • The control group's demographics, baseline characteristics, and treatment (routine FC) are documented.
    • No substantial differences in sex, secondary school location, economic status, interest in the Fundamentals of Nursing course, interest in the nursing profession, degree of participation... were observed between the two groups.
    • Relevant Quotes: 1) "Table 1 Frequency and percentage distribution of the students' demographic characteristics (n=64 in each group)" (p. 4) 2) "No substantial differences in sex, secondary school location, economic status, interest in the Fundamentals of Nursing course, interest in the nursing profession, degree of participation... were observed between the two groups." (p. 7) 3) "However, the control group and students who didn't agree to participate in the program did not receive any intervention, except for their routine FC education." (p. 5) 4) "The two groups were taught once a week (120 min) in the laboratory skills with the same six instructors and content." (p. 5) 5) "In the first 2 weeks, both groups received routine FC instructions, which included a 2-h skill video and one multiple choice quiz related to pre-class materials..." (p. 5) Detailed Analysis: Criterion D requires clear documentation of the control group's characteristics and conditions. Table 1 provides demographic data for both groups, showing their baseline comparability. The paper states the control group received only the routine Flipped Classroom (FC) instruction, the same as the intervention group during the baseline phase and throughout for the control group. Both groups had the same instructors and content. Criterion D (Documented Control Group) is met because the control group's characteristics and treatment are described.

• Level 2 Criteria

  • S

    School-level RCT

    • Randomization was at the student level within a single university, not at the school level.
    • This study required an a priori sample size of 128 students, who were randomly divided into two groups (study and control) using block randomization... Participants were recruited from the Faculty of Nursing, Mansoura University.
    • Relevant Quotes: 1) "This study required an a priori sample size of 128 students, who were randomly divided into two groups (study and control) using block randomization with a block size of 4." (p. 3) 2) "Participants were recruited from the Faculty of Nursing, Mansoura University." (p. 3) 3) "This study randomly divided the students into the study and control groups." (p. 1) Detailed Analysis: Criterion S requires randomization at the school level. The paper states randomization occurred at the student level within a single faculty (Faculty of Nursing) at one university (Mansoura University). This does not meet the requirement for school-level randomization. Criterion S (School-level RCT) is not met because randomization was done at the student level within one institution.
  • I

    Independent Conduct

    • The same authors appear to have designed the intervention, conducted the study, and analyzed the data, with no mention of independent conduct.
    • Authors' contributions SME: conceptualization and data curation; WGM: administration, validation, and visualization; MEH & HME: original draft preparation and writing, review- ing, and editing; MEH: resources, software, and data analysis. All authors have read and approved the final version of the manuscript to be published.
    • Relevant Quotes: 1) "Authors' contributions SME: conceptualization and data curation; WGM: administration, validation, and visualization; MEH & HME: original draft preparation and writing, review- ing, and editing; MEH: resources, software, and data analysis. All authors have read and approved the final version of the manuscript to be published." (p. 11) 2) "The instructor used the Active Presenter and Hot Potatoes software and game module in designing the gamified quizzes." (p. 5) (Note: MEH is listed as handling resources, software, and data analysis). 3) "Three formative quizzes for each timepoint were developed by the researchers." (p. 4) 4) "Eight checklists were developed after reviewing the literature, nursing textbooks, and logbook of the Faculty of Nursing, Mansoura University..." (p. 4) Detailed Analysis: Criterion I requires the study to be conducted independently from the intervention designers. The author contributions section indicates the authors were involved in conceptualization, design (software, resources), data curation, administration, validation, writing, and analysis. The researchers developed the assessment tools and designed the gamified quizzes. There is no mention of an independent third party conducting the study or evaluation. Although data collectors were blinded , this doesn't equate to independent conduct of the overall study design and analysis. Criterion I (Independent Conduct) is not met because the same team designed, implemented, and analyzed the study.
  • Y

    Year Duration

    • The study duration, including data collection, was 11 weeks, which is less than a full academic year. Also, Criterion Y was not met.
    • The assistant staff gathered data for 11 weeks at three timepoints in the second academic semester (2020–2021).
    • Relevant Quotes: 1) "The assistant staff gathered data for 11 weeks at three timepoints in the second academic semester (2020–2021)." (p. 5) 2) "Timepoint 0 (T0) covers the first 2 weeks, and timepoints 1 (T1) and 2 (T2) cover the subsequent 3 weeks." (p. 5) Detailed Analysis: Criterion Y requires outcomes to be measured at least one full academic year (approx. 9-10 months) after the intervention begins. The paper states data collection spanned 11 weeks in total during one semester. This duration is significantly shorter than a full academic year. Criterion Y (Year Duration) is not met because the study's duration was only 11 weeks. (Also, Criterion T was not met).
  • B

    Balanced Resources

    • The intervention group received gamified activities (extra quizzes, points, badges) which constitute additional resources/ engagement time compared to the control group's routine FC, and this difference was not explicitly tested as the treatment variable nor balanced.
    • Then, after collecting baseline data, Moodle was gamified over the remaining 6 weeks for the students in the intervention group, and game elements included three gamified quizzes on each skill... badges, leaderboards, ranks, levels/unlocks, and points... However, the control group... did not receive any intervention, except for their routine FC education.
    • Relevant Quotes: 1) "Then, after collecting baseline data, Moodle was gamified over the remaining 6 weeks for the students in the intervention group, and game elements included three gamified quizzes on each skill (i.e., easy, moderate, and high difficulty), badges, leaderboards, ranks, levels/unlocks, and points..." (p. 5) 2) "However, the control group and students who didn't agree to participate in the program did not receive any intervention, except for their routine FC education." (p. 5) 3) "The students could compete and make maximum points and badges through this online motivation." (p. 5) 4) "Easy game quizzes contain 5 questions; a moderate quiz consists of 10 questions in crossword form; and a difficult quiz contains 15 questions in millionaire quiz form." (p. 5) Detailed Analysis: Criterion B requires balanced time/budget between groups unless the difference is the treatment variable. The intervention group received gamified elements (multiple extra quizzes per skill, badges, leaderboards, points, levels) on top of the routine FC activities. This represents additional content and engagement time compared to the control group, which only received routine FC. The study aimed to assess the effects of *gamified* FC versus routine FC , implying the gamification elements themselves were the focus. However, the standard requires that if extra resources (like additional quiz time/engagement) are provided, they must either be explicitly framed as the variable under test OR balanced in the control group. The paper frames the *gamified FC approach* as the intervention, but doesn't explicitly state it's testing the effect of *additional engagement time* provided by these gamified elements versus a control group with equivalent non-gamified engagement time. The control group did not receive any comparable extra activities or time. The difference in resources/ engagement (gamified activities vs. none) was not balanced and not explicitly framed as *testing the effect of additional resources/time*. Following the pseudocode: EXTRA_RESOURCES_PRESENT is true (gamified quizzes/activities take time/effort). RESOURCES_ARE_TREATMENT is arguably false (they test 'gamified FC' effect, not explicitly 'extra time via gamification' effect). CONTROL_MATCHES_RESOURCES is false. WITHIN_SUBJECTS is false. Therefore, the criterion is not met. Criterion B (Balanced Control Group) is not met because the intervention group received additional gamified activities (implying extra engagement time/resources) not matched in the control group, and this imbalance wasn't explicitly the treatment variable being tested.

• Level 3 Criteria

  • R

    Reproduced Results

    • The paper does not mention any independent replication of this specific study by another research team.
    • Relevant Quotes: 1) "However, this study has many limitations. First, there are few similar randomized controlled trials with which to compare our results." (p. 10) 2) "Most randomized controlled trials were not comparable because they either used game-based learning or the control group used conventional teaching and not FC learning and the subjects were not usually higher education nursing students." (p. 10) Detailed Analysis: Criterion R requires the study to be independently replicated by a different team in a different context. The authors explicitly state there are few similar RCTs to compare their results with and that most are not comparable. There is no mention within the paper of any external team replicating this specific intervention (Gamified FC using these specific elements vs routine FC for nursing skills). A quick check for external replications would be needed, but based solely on the paper, none is reported. Additionally, a literature search did not find any external replication of this study by independent researchers as of April 2025. Criterion R (Reproduced) is not met as there is no evidence within the paper of independent replication.
  • A

    All Exams

    • The study measured only nursing skills competency and related factors, not performance across all core academic subjects. Also, Criterion E was not met.
    • This study was designed to assess the effects of using gamified FCs on the Fundamentals of Nursing students' skills competency and learning motivation.
    • Relevant Quotes: 1) "This study was designed to assess the effects of using gamified FCs on the Fundamentals of Nursing students' skills competency and learning motivation." (p. 2) 2) "The primary outcomes were improvement in nursing students' motivation, skill knowledge, skill performance and skill confidence level." (p. 3) 3) "The secondary outcome includes improvement in nursing students' intensity of preparation." (p. 3) Detailed Analysis: Criterion A requires measuring impact on all main subjects using standardized exams. The study focused exclusively on outcomes related to a specific course, "Fundamentals of Nursing II," specifically measuring skills competency (knowledge, performance, confidence), motivation, and preparation intensity. There is no mention of assessing performance in other general academic subjects. Furthermore, Criterion E (Exam-based Assessment) was not met because non-standardized assessments were used. Criterion A (All-subject Exams) is not met because only nursing-specific outcomes were measured, and standardized exams were not used (Criterion E failed).
  • G

    Graduation Tracking

    • The study tracked students for 11 weeks, not until graduation. Also, Criterion Y was not met.
    • The assistant staff gathered data for 11 weeks at three timepoints in the second academic semester (2020–2021).
    • Relevant Quotes: 1) "The assistant staff gathered data for 11 weeks at three timepoints in the second academic semester (2020–2021)." (p. 5) 2) "The long-term effects of gamification are important to be studied, to evaluate the long-term effects of game designs on user behavior." (p. 11) Detailed Analysis: Criterion G requires tracking participants until graduation from their current educational stage. The study's data collection period was explicitly stated as 11 weeks within one semester. The authors acknowledge the need to study long-term effects, indicating such tracking was not part of this study. No follow-up publication by the authors tracking this cohort to graduation has been found (as of April 2025). Furthermore, Criterion Y (Year Duration) was not met. Criterion G (Graduation Tracking) is not met because the follow-up period was only 11 weeks.
  • P

    Pre-Registered Protocol

    • The study was prospectively registered on ClinicalTrials.gov before the intervention likely started based on the semester timing.
    • Prospectively registered with ClinicalTrials.gov on 26/04/2021; registration number NCT04859192.
    • Relevant Quotes: 1) "Trial registration. Prospectively registered with ClinicalTrials.gov on 26/04/2021; registration number NCT04859192." (p. 1) 2) "The assistant staff gathered data for 11 weeks at three timepoints in the second academic semester (2020–2021)..." (p. 5) (Assuming the second semester runs roughly Feb-June). Detailed Analysis: Criterion P requires pre-registration before the study begins . The paper explicitly states it was prospectively registered on ClinicalTrials.gov on April 26, 2021. Data collection occurred during the second academic semester of 2020-2021. While the exact start date of data collection isn't given, the second semester typically starts earlier in the year (e.g., January/February). However, the term "prospectively registered" implies registration before participant enrollment or intervention initiation. Given the explicit statement of prospective registration and the date, it meets the requirement. According to the trial registry, the study's start date was April 22, 2021​:contentReference[oaicite:0]{index=0}, which shows that registration occurred essentially at the study's onset. Criterion P (Pre-registered) is met as the study states it was prospectively registered on ClinicalTrials.gov.