Effects of Metaverse-Based Education on Motivation and Academic Achievement in Undergraduate Physiotherapy Students: A Randomized Three-Arm Study

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomization was carried out at the individual student level within a single course cohort, not at the class or school level, and the tutoring exception does not apply.
    • "After completion of the baseline assessments, participants were randomly assigned in a 1:1:1 ratio to one of three groups... A computer-generated randomization sequence was created using Random.org."
    • Relevant Quotes: 1) "This study was designed as a three-arm randomized controlled trial with a pre-test/post-test repeated-measures structure..." (Methods, Study Design) 2) "Participants were second-year undergraduate physiotherapy students enrolled in the relevant course during the study period." (Methods, Eligibility criteria) 3) "After completion of the baseline assessments, participants were randomly assigned in a 1:1:1 ratio to one of three groups: metaverse-based education, synchronous online education, or face-to-face education. A computer-generated randomization sequence was created using Random.org." (Methods, Randomization and Allocation) 4) "All 97 students enrolled in the course were invited; 90 provided consent and were randomized equally (30/30/30)." (Methods, Sample Size) Detailed Analysis: Criterion C requires that randomisation be conducted at the class level (or the stronger school level), so that entire classes rather than individual students within a single class are allocated to conditions, thereby preventing contamination. In this study, all participants were second-year students enrolled in a single course at one institution, and randomisation was performed on individual students ("participants were randomly assigned in a 1:1:1 ratio", using a Random.org sequence). This is student-level randomisation within a single cohort, not class-level or school-level allocation. The exception for one-to-one tutoring or personal teaching does not apply, because the intervention was group-based classroom instruction delivered to whole groups by an instructor. Because individual students in the same course were split across the three delivery-mode conditions, there is a clear risk of cross-group contamination. Criterion C is not met because randomisation occurred at the individual student level within a single course, rather than at the class or school level, and no valid tutoring exception applies.
  • E

    Exam-based Assessment

    • Academic achievement was measured with a researcher- developed, non-validated 15-item multiple-choice quiz rather than a standardised, widely recognised exam.
    • "Academic achievement was evaluated using a 15-item multiple-choice quiz developed by the research team to reflect the learning objectives of the course."
    • Relevant Quotes: 1) "Academic achievement was evaluated using a 15-item multiple-choice quiz developed by the research team to reflect the learning objectives of the course." (Methods, Academic Achievement Quiz) 2) "The quiz items were generated by the researchers through a consensus-based process and were designed to represent the theoretical content covered during the educational sessions in accordance with the predefined instructional objectives." (Methods) 3) "In addition, although the quiz was developed to align with the predefined instructional objectives, it was researcher-developed and was not subjected to a formal psychometric validation process." (Discussion, limitations) 4) "Learning motivation was assessed using the IMMS, which is grounded in Keller's ARCS model... The Turkish adaptation and psychometric properties of the IMMS were reported by Kutu and Sozbilir." (Methods, IMMS) Detailed Analysis: Criterion E requires that outcomes be measured using a standardised, widely recognised exam-based assessment that was not specially created for the study. The academic achievement outcome here was a 15-item multiple-choice quiz "developed by the research team" and "generated by the researchers through a consensus-based process," which the authors themselves acknowledge "was researcher-developed and was not subjected to a formal psychometric validation process." This is precisely the kind of custom-made instrument that criterion E excludes. The IMMS is a validated (Turkish-adapted) instrument, but it measures learning motivation, not academic achievement via a standardised exam, so it does not satisfy the exam-based assessment requirement. Criterion E is not met because academic achievement was assessed with a custom, researcher-developed, non-validated quiz rather than a standardised exam.
  • T

    Term Duration

    • The intervention consisted of brief instructional sessions with an immediate post-test, far shorter than one full academic term.
    • "After completion of the educational sessions, the IMMS was re-administered as the post-test measure of learning motivation. Academic achievement was assessed at post-test using the 15-item multiple-choice quiz."
    • Relevant Quotes: 1) "All groups received identical learning objectives, theoretical content, instructional materials, and planned instructional sequence... all sessions were delivered by the same instructor with the same planned duration." (Methods, Educational Interventions) 2) "At baseline, participants completed the Personal Information Form and the IMMS before randomization. After completion of the educational sessions, the IMMS was re-administered as the post-test measure of learning motivation. Academic achievement was assessed at post-test using the 15-item multiple-choice quiz." (Methods, Assessment Procedure and Timeline) 3) "Second, quiz performance was assessed only post-intervention, limiting our ability to quantify within-person learning gains and retention over time." (Discussion, limitations) 4) "Future studies should incorporate pre-post testing and longer follow-up to better quantify learning gains and retention over time." (Discussion) Detailed Analysis: Criterion T requires that primary outcomes be measured at least one full academic term (roughly 3-4 months) after the intervention begins, even for short interventions, via term-long follow-up tracking. Here the intervention was a set of short instructional sessions on palpation-related theoretical content delivered on the same planned duration, with the post-test motivation survey and the achievement quiz administered immediately "after completion of the educational sessions." There is no indication of any follow-up window approaching one term; the authors explicitly note the lack of longer follow-up and call for it in future work. The interval from intervention start to outcome measurement is therefore far shorter than a full academic term. Criterion T is not met because outcomes were measured immediately after brief sessions, with no term-long follow-up from intervention start.
  • D

    Documented Control Group

    • Baseline demographics, sample sizes, and pre-instruction scores for the comparison (face-to-face and online) groups are documented in detail in Table 1.
    • "Baseline participant characteristics and pre-instruction IMMS scores are presented in Table 1 for descriptive comparison only. Age and sex distributions appeared broadly similar across groups..."
    • Relevant Quotes: 1) "Ninety undergraduate physiotherapy students were randomized to metaverse-based, synchronous online, or face-to-face education; 83 completed the study (metaverse n=28, online n=26, face-to-face n=29)." (Abstract) 2) "Table 1. Baseline participant characteristics and pre-instruction IMMS scores by group" listing, for the face-to-face (n=29) and online (n=26) comparison groups, age (20.10 +/- 1.42; 20.54 +/- 1.39), sex, study-supporting technology, prior metaverse/VR experience, and baseline IMMS totals (83.69 +/- 14.57; 79.12 +/- 6.53). (Table 1) 3) "Participants in the face-to-face group attended the sessions in a conventional classroom setting. Sessions were delivered by the same instructor and followed the same instructional plan, schedule, and duration as the metaverse-based and synchronous online groups." (Methods) 4) "In contrast to the metaverse group, this format did not include avatar-based participation, spatial navigation, or a shared virtual classroom environment..." (Methods, Synchronous Online Education) Detailed Analysis: Criterion D requires that the control/comparison group be well documented, including demographics, baseline performance, size, and the conditions it received. In this three-arm trial, the conventional face-to-face and synchronous online arms serve as comparators to the metaverse arm. Table 1 provides detailed baseline data for each group, including age, sex, study-supporting technology, social media use, prior metaverse/VR experience, and pre-instruction IMMS total and subscale scores, along with exact sample sizes. The nature of what each comparison group received (same content and duration, differing only in delivery mode) is clearly described. This level of documentation allows readers to judge baseline comparability. Criterion D is met because the comparison groups' demographics, sizes, baseline scores, and received conditions are clearly documented in Table 1 and the methods.

• Level 2 Criteria

  • S

    School-level RCT

    • The trial was conducted in a single course at one university with student-level randomization, not randomization of whole schools or institutions.
    • "The study was conducted at the Department of Physiotherapy and Rehabilitation, Faculty of Health Sciences, Süleyman Demirel University, Türkiye..."
    • Relevant Quotes: 1) "The study was conducted at the Department of Physiotherapy and Rehabilitation, Faculty of Health Sciences, Süleyman Demirel University, Türkiye, during the fall semester of the 2025-2026 academic year." (Methods, Setting and Study Period) 2) "All 97 students enrolled in the course were invited; 90 provided consent and were randomized equally (30/30/30)." (Methods, Sample Size) 3) "After completion of the baseline assessments, participants were randomly assigned in a 1:1:1 ratio to one of three groups..." (Methods, Randomization and Allocation) Detailed Analysis: Criterion S requires randomisation at the school level, where 'school' means the institution or unit implementing the intervention (e.g., centres, sites, K-12 schools). In this study, there was only a single institution and a single course cohort, and randomisation was performed on individual students within that cohort. No multiple schools, centres, or sites were randomised. The design is therefore a single-site, student-level RCT, which does not satisfy the school-level requirement. Criterion S is not met because the study involved one institution and randomised individual students rather than whole schools or sites.
  • I

    Independent Conduct

    • The same research team designed the metaverse intervention and conducted and analysed the study, with no independent third-party evaluation of the trial.
    • "ME: conceptualization, methodology, investigation, data curation, writing—original draft."
    • Relevant Quotes: 1) "Group allocation was implemented by an independent researcher who was not involved in outcome assessment or data analysis, and assignment was carried out only after completion of the baseline assessments." (Methods, Randomization and Allocation) 2) "All educational sessions were delivered by the same instructor in order to standardize content delivery across groups." (Methods, Setting and Study Period) 3) "ME: conceptualization, methodology, investigation, data curation, writing—original draft. TK: formal analysis... FB: conceptualization, methodology, supervision... FD: resources, software..." (Authors' contributions) 4) "Although the use of a single instructor helped standardize content delivery across groups, it may also have introduced instructor-related bias, including the possibility of unconscious differences in emphasis or interaction between educational modalities." (Discussion, limitations) Detailed Analysis: Criterion I requires that the study be conducted independently from those who designed the intervention, to reduce bias in implementation, measurement, analysis, and reporting. Here the authors conceptualised and designed the intervention (including software/resources for the metaverse environment) and also carried out the investigation, data curation, formal analysis, and reporting. While an "independent researcher" implemented group allocation and was not involved in outcome assessment, this is limited to the randomisation step; the overall trial (design, delivery, assessment, and analysis) was run by the same team that created the intervention. There is no external evaluation agency or third-party oversight of data collection and analysis, and the authors acknowledge possible instructor-related bias. Criterion I is not met because the intervention designers also conducted and analysed the study, with only the allocation step (not the overall evaluation) handled independently.
  • Y

    Year Duration

    • Because the term-duration criterion (T) is not met and the intervention lasted only brief sessions, the year-duration requirement is also not met.
    • "Future studies should incorporate pre-post testing and longer follow-up to better quantify learning gains and retention over time."
    • Relevant Quotes: 1) "After completion of the educational sessions, the IMMS was re-administered as the post-test measure of learning motivation. Academic achievement was assessed at post-test using the 15-item multiple-choice quiz." (Methods, Assessment Procedure and Timeline) 2) "Second, quiz performance was assessed only post-intervention, limiting our ability to quantify within-person learning gains and retention over time." (Discussion, limitations) 3) "Future studies should examine whether the observed motivational advantages translate into objective skill performance and should include pre-post knowledge testing with longer follow-up to evaluate retention." (Discussion) Detailed Analysis: Criterion Y requires that outcomes be measured at least 75% of a full academic year (approximately 9-10 months) after the intervention begins. Per the criterion-specific rule, if T (Term Duration) is not met, Y cannot be met. In this study, the intervention consisted of brief instructional sessions with immediate post-testing and no extended follow-up; the interval from start to measurement is far below one term, let alone a full academic year. The authors themselves call for "longer follow-up" in future work. Criterion Y is not met because the tracking interval was only brief, well under a year, and because criterion T was not met.
  • B

    Balanced Control Group

    • All three groups received the same instructor, content, materials, and planned duration, differing only in delivery mode, so instructional time and resources were balanced across groups.
    • "All groups received identical learning objectives, theoretical content, instructional materials, and planned instructional sequence... all sessions were delivered by the same instructor with the same planned duration."
    • Relevant Quotes: 1) "All groups received identical learning objectives, theoretical content, instructional materials, and planned instructional sequence. The same visual resources, videos demonstrating how relevant structures are palpated, and discussion points were used across groups, and all sessions were delivered by the same instructor with the same planned duration." (Methods, Educational Interventions) 2) "The interventions differed primarily in the mode of delivery and in the interactive and spatial features of the learning environment, including the use of embedded three-dimensional instructional supports within the metaverse environment." (Methods, Educational Interventions) 3) "Sessions were delivered by the same instructor and followed the same instructional plan, schedule, and duration as the metaverse-based and synchronous online groups." (Methods, Face-to-Face Education) 4) "Although the core educational content was kept consistent across groups, the metaverse group experienced this content in a more interactive and spatially organised learning environment." (Methods, Metaverse Environment) Detailed Analysis: Criterion B compares the nature, quantity, and quality of resources (time, budget, materials, adult support) provided to the conditions and asks whether the comparison condition offers a comparable substitute, unless additional resources are the explicit treatment variable. Applying the decision tree: the study repeatedly states that all three arms received identical learning objectives, the same theoretical content, the same instructional materials and sequence, the same single instructor, and the same planned instructional duration. Thus instructional time and human support are equalised across groups (EXTRA_RESOURCES_PRESENT is effectively false with respect to time/budget). The only difference is the delivery environment (a shared metaverse virtual classroom with embedded 3D supports versus a videoconference versus a physical classroom), which is itself the treatment variable being tested. The added metaverse features (Spatial.io platform, 3D elements) are integral to the metaverse-based education being evaluated rather than a separable, confounding resource add-on, and the time on task is matched. Therefore the groups are balanced. Criterion B is met because all groups received equal instructional time, content, materials, and instructor, with the metaverse environment features being the integral treatment variable rather than an unmatched extra resource.

• Level 3 Criteria

  • R

    Reproduced

    • No independent replication of this specific trial by a separate research team is reported or identifiable.
    • "Consistent with this rationale, prior randomized work in nursing education reported that instructional-materials motivation (IMMS) was comparable between groups at baseline but was higher in the metaverse group after the intervention..."
    • Relevant Quotes: 1) "Consistent with this rationale, prior randomized work in nursing education reported that instructional-materials motivation (IMMS) was comparable between groups at baseline but was higher in the metaverse group after the intervention, and that IMMS scores also increased from pre to post within the metaverse group [21]." (Discussion) 2) "Çetinkaya Uslusoy E, Aydinli A, Durna F. Enhancing learning motivation and academic achievement in nursing students through metaverse-based learning: A randomized controlled study. Japan J Nurs Sci. 2024;21(3):1–11." (Reference 21) 3) "In particular, direct comparisons between metaverse-based education, conventional synchronous online education, and face-to-face instruction are still scarce in physiotherapy education when teaching palpation-related theoretical content." (Background) Detailed Analysis: Criterion R requires that this specific study (or its central experimental claim, in the same context and design) be independently replicated by a different research team, published in a peer-reviewed outlet. This is a novel three-arm physiotherapy trial that the authors describe as addressing a scarce/limited evidence gap. The cited prior nursing-education RCT [21] shares a co-author (F. Durna) and is in a different population and design, so it is related background work rather than an independent replication of this study. No subsequent independent replication of this palpation-focused physiotherapy trial was identified in available sources. Criterion R is not met because no independent replication of this specific trial by a separate team has been reported or found.
  • A

    All-subject Exams

    • Criterion E is not met, and outcomes were limited to a single palpation-related topic, so the all-subject exam requirement is not satisfied.
    • "The quiz included multiple-choice items addressing anatomical landmarks, vertebral level identification, recognition of structures relevant to palpation, and interpretation of basic positional and directional principles used in palpation..."
    • Relevant Quotes: 1) "The educational content focused on palpation-related theoretical content, including anatomical landmarks and conceptual understanding that support palpation practice, without hands-on practical training." (Methods, Study Design) 2) "The quiz included multiple-choice items addressing anatomical landmarks, vertebral level identification, recognition of structures relevant to palpation, and interpretation of basic positional and directional principles used in palpation, including visually based identification tasks." (Methods, Academic Achievement Quiz) 3) "it was researcher-developed and was not subjected to a formal psychometric validation process." (Discussion, limitations) Detailed Analysis: Criterion A requires that impact be measured across all main subjects using standardised exam-based assessments, and it explicitly depends on criterion E being met. Here, criterion E is not met (the achievement measure was a custom, non-validated quiz), which alone means A cannot be met. Additionally, the outcome measurement was confined to a single narrow topic (palpation-related theoretical content, anatomical landmarks), with no assessment of other core subjects. Criterion A is not met because criterion E is not met and only one narrow subject area was assessed with a non-standardised instrument.
  • G

    Graduation Tracking

    • Criterion Y is not met and there was no follow-up beyond the immediate post-test, so graduation tracking is not satisfied.
    • "Second, quiz performance was assessed only post-intervention, limiting our ability to quantify within-person learning gains and retention over time."
    • Relevant Quotes: 1) "After completion of the educational sessions, the IMMS was re-administered as the post-test measure of learning motivation. Academic achievement was assessed at post-test using the 15-item multiple-choice quiz." (Methods, Assessment Procedure and Timeline) 2) "Future studies should examine whether the observed motivational advantages translate into objective skill performance and should include pre-post knowledge testing with longer follow-up to evaluate retention." (Discussion) Detailed Analysis: Criterion G requires that participants be tracked through to graduation from their educational stage, and per the criterion-specific rule it depends on criterion Y being met. Criterion Y is not met, so G cannot be met. Moreover, the study measured outcomes only immediately after the brief intervention, with no follow-up cohort tracking, and no follow-up publications by these authors tracking this cohort to graduation were found in available sources. Criterion G is not met because there was no long-term or graduation follow-up and because criterion Y was not met.
  • P

    Pre-Registered

    • Registry verification confirms ClinicalTrials.gov NCT07166705 was first submitted 2 Sept 2025, before the recorded study start of 5 Sept 2025, indicating prospective registration before data collection.
    • "The study was registered at ClinicalTrials.gov (NCT07166705)."
    • Relevant Quotes: 1) "Ethical approval was obtained from the Süleyman Demirel University Health Sciences Ethics Committee, Türkiye (Meeting No: 96; Decision No: 7; Date: 12 May 2025)." (Methods, Ethical Approval and Trial Registration) 2) "The study was registered at ClinicalTrials.gov (NCT07166705)." (Methods, Ethical Approval and Trial Registration) 3) "The study was conducted at the Department of Physiotherapy and Rehabilitation... during the fall semester of the 2025-2026 academic year." (Methods, Setting and Study Period) Registry verification (ClinicalTrials.gov NCT07166705, checked 23 July 2026): - Registered title: "Metaverse-Based Education for Physiotherapy and Rehabilitation Students" - First Submitted: 2 September 2025 - First Posted: 10 September 2025 - Study Start Date (enrolment): 5 September 2025 (estimated) - Overall status: Not yet recruiting - Planned enrolment: 48 participants (estimated) Detailed Analysis: Criterion P requires a pre-registered protocol whose registration date precedes the start of data collection. The paper cites ClinicalTrials.gov NCT07166705, and the registry record confirms the entry was first submitted on 2 September 2025 and first posted on 10 September 2025, while the study start (enrolment) is recorded as 5 September 2025 and the status remains "Not yet recruiting." The submission date thus precedes the recorded enrolment start, and the study's data collection took place during the 2025-2026 fall semester, which in Türkiye begins in late September/October 2025, after registration. This establishes prospective registration ahead of data collection. Minor record inconsistencies exist (registry planned enrolment of 48 versus 90 randomised in the paper, and an unrefreshed "Not yet recruiting" status), but these do not negate the pre-registration timing. Criterion P is met because the ClinicalTrials.gov record (NCT07166705) was first submitted on 2 September 2025, before the recorded study start of 5 September 2025, confirming prospective registration prior to data collection.