Enhancing motor skill acquisition and autonomous motivation in university physical education: a randomized controlled trial of a STEAM-integrated approach

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomisation was at the individual student level within intact sections rather than at the class or school level, and the intervention is not personal tutoring.
    • "Within each section, students were individually randomized (1:1) to either the STEAM-RB curriculum or the traditional Roliball syllabus, stratified by section to preserve existing social and timetable structures."
    • Relevant Quotes: 1) "This single-site, section-stratified, individual-level randomized trial evaluated a STEAM-integrated Roliball (STEAM-RB) curriculum in university physical education." (p. 2, Section 2.1) 2) "Within each section, students were individually randomized (1:1) to either the STEAM-RB curriculum or the traditional Roliball syllabus, stratified by section to preserve existing social and timetable structures." (p. 3, Section 2.3) 3) "This produced two mixed-composition classes in each section (STEAM-RB and traditional), with instruction delivered in parallel." (p. 3, Section 2.3) 4) "Two intact first-year Roliball PE sections at a public university in central China were recruited at the beginning of the semester (N = 70)." (p. 2, Section 2.1) 5) "Time-on-task was matched across sections; contamination was minimized by tool-use restrictions in the traditional condition." (Figure 2 note, p. 5) Detailed Analysis: Criterion C requires that the unit of randomisation be the whole class (or a stronger unit such as the school), unless the intervention is personal tutoring or one-to-one teaching, in which case student-level randomisation is acceptable. The paper is explicit and repeated that randomisation was at the individual student level: it self-describes as an "individual-level randomized trial" and states that "students were individually randomized (1:1)". Section was used only as a stratification variable, not as the randomised unit. Students drawn from the same intact section were split between the two arms, and the resulting instructional groups were "mixed-composition classes" formed after randomisation from within the same sections. This is precisely the design pattern that criterion C is intended to guard against: students in the same social and timetable structure are allocated to different conditions, creating contamination risk. The authors themselves recognise this exposure, noting that "contamination was minimized by tool-use restrictions in the traditional condition" - an administrative mitigation, not a design-level isolation of the groups. The tutoring exception does not apply. The STEAM-RB curriculum is a group-delivered university PE syllabus taught in standard indoor PE facilities in two 90-min classes per week; it is not one-to-one or personal tutoring. Criterion C is not met because randomisation was performed at the individual student level within intact sections rather than at the class or school level, and no tutoring exception applies.
  • E

    Exam-based Assessment

    • All outcomes used bespoke, curriculum-aligned instruments (custom STEAM test, study-specific rubric and kinematic proxy) rather than any recognised standardised exam.
    • "Task-specific STEAM knowledge was assessed through a 15-item criterion-referenced test aligned with the core STEAM concepts embedded in the curriculum..."
    • Relevant Quotes: 1) "Kinematic performance was indexed by peak racket-head angular velocity (deg/s) during a standardized Roliball forehand drive task at pre- and post-test." (p. 4, Section 2.5.1) 2) "Technical quality of the forehand drive was also assessed using blinded expert ratings from standardized video clips ... Two experienced Roliball instructors not involved in day-to-day teaching independently rated each clip on a 10-point holistic scale, using a brief rubric emphasizing swing path continuity, segment coordination, timing, and overall control." (p. 4, Section 2.5.2) 3) "Task-specific STEAM knowledge was assessed through a 15-item criterion-referenced test aligned with the core STEAM concepts embedded in the curriculum ... Items included multiple-choice, short-answer, and diagram-based questions, each scored on a 0-3 scale (0 = incorrect, 3 = fully correct), yielding total scores from 0 to 45." (p. 4, Section 2.5.3) 4) "Content validity was evaluated by an expert panel of eight PE and STEAM educators using content validity index (CVI) procedures. Item-level CVIs ranged from 0.88 to 1.00, and the scale-level CVI was 0.94 ... In the present sample, internal consistency was acceptable (Cronbach's alpha = 0.85)." (p. 4, Section 2.5.3) 5) "The STEAM knowledge test showed how curricular tasks can be mapped onto a focused assessment instrument with acceptable content validity, rather than relying on generic science or mathematics tests." (p. 8, Section 4.4) 6) "Finally, the STEAM knowledge test was tightly aligned to RB-specific content, so scores should not be interpreted as reflecting general STEAM competence..." (p. 9, Section 4.6) 7) "Motivation for physical activity in the PE context was assessed with the Chinese University Students' Physical Activity Motivation Scale (CUSPAMS), developed and validated under contemporary PE policy constraints." (p. 4, Section 2.6.1) Detailed Analysis: Criterion E requires outcomes to be measured with standardised, widely recognised exam-based assessments rather than instruments designed for the study. None of the outcome measures in this trial qualifies. The STEAM knowledge test is a bespoke 15-item criterion-referenced instrument built specifically for this curriculum; the author states plainly that it was designed "rather than relying on generic science or mathematics tests" and that it was "tightly aligned to RB-specific content". A locally constructed test whose content mirrors the intervention's own inquiry tasks is exactly the alignment bias criterion E exists to exclude, and the reported CVI and Cronbach's alpha figures speak to internal quality, not to external standardisation. The kinematic performance measure (peak racket-head angular velocity from smartphone video) is a study-specific instrumented performance proxy, not an exam. The blinded expert ratings use "a brief rubric" devised for the trial and rated on a 10-point holistic scale by two instructors; the excellent inter-rater ICC (0.94) establishes rater agreement but not exam standardisation. CUSPAMS is an externally developed and validated psychometric scale, but it is a self-report motivation questionnaire, not a standardised exam-based assessment of educational attainment. The same applies to the revised Creative Personality Scale. There is no national, state-wide, or otherwise widely recognised standardised exam anywhere in the outcome battery, and no university course grade or official examination result is reported. Criterion E is not met because all attainment outcomes relied on custom-built, curriculum-aligned instruments rather than recognised standardised exams.
  • T

    Term Duration

    • Outcomes were measured at the end of an 8-week intervention with no later follow-up, which is shorter than one full academic term.
    • "Both groups completed two 90-min Roliball classes per week for 8 weeks (16 sessions) in standard indoor PE facilities."
    • Relevant Quotes: 1) "Participants were allocated to either an 8-week STEAM-RB intervention or a traditional syllabus." (p. 1, Abstract) 2) "Both groups completed two 90-min Roliball classes per week for 8 weeks (16 sessions) in standard indoor PE facilities." (p. 3, Section 2.4) 3) "The following outcomes were assessed at pretest and posttest: kinematic performance, blinded expert ratings, STEAM knowledge, physical activity motivation, and creative disposition." (p. 4, Section 2.5) 4) "Thus, there was no attrition over the 8-week intervention, and all randomized participants were included in the primary analyses." (p. 5, Section 3.1) 5) "The 8-week duration precluded examination of long-term maintenance and may have been too short to affect more stable constructs such as creative disposition." (p. 9, Section 4.6) 6) "Future research could extend this work through multi-site trials across different types of universities and regions, longer interventions spanning a semester or year..." (p. 9, Section 4.6) Detailed Analysis: Criterion T requires at least one full academic term (approximately 3-4 months) to elapse between the start of the intervention and the measurement of primary outcomes. Short interventions are permitted, but term-long follow-up tracking is not optional. Here the intervention ran for 8 weeks (16 sessions across weeks 1-8, as also depicted in Figure 2), and the posttest was administered at the end of that 8-week block. There is no delayed or maintenance follow-up: the paper reports only pretest and posttest, and explicitly acknowledges that "the 8-week duration precluded examination of long-term maintenance". The author's own future-directions statement contrasts this trial with "longer interventions spanning a semester or year", confirming that 8 weeks falls short of a semester in this institutional context. Eight weeks is roughly two months, well under the 3-4 month minimum, and since no measurement occurred after the intervention ended, the interval from intervention start to primary outcome measurement is itself only 8 weeks. Criterion T is not met because the interval from intervention start to outcome measurement was only 8 weeks, shorter than one full academic term, with no later follow-up.
  • D

    Documented Control Group

    • The control group's size, baseline demographics and pretest scores (Table 1), and instructional conditions are documented in full, with fidelity checks confirming no STEAM elements leaked in.
    • "The traditional curriculum followed the existing university syllabus, emphasizing teacher-directed demonstration, part-whole practice of core techniques (e.g., basic swings, directional control), and summative skill testing."
    • Relevant Quotes: 1) "All 70 students who consented were randomized to the STEAM-RB (n = 35) or traditional Roliball group (n = 35) and completed both pretest and posttest assessments (Figure 2)." (p. 5, Section 3.1) 2) "The traditional curriculum followed the existing university syllabus, emphasizing teacher-directed demonstration, part-whole practice of core techniques (e.g., basic swings, directional control), and summative skill testing." (p. 3, Section 2.4) 3) "TABLE 1 Baseline characteristics with ASMD, t/chi-square, and p (N = 70)." (p. 6) 4) "The two groups were similar in age (STEAM-RB: 18.5 +/- 0.6 years; traditional: 18.4 +/- 0.7 years; ASMD = 0.15; t approx. 0.64, p = 0.523) and body mass index (21.6 +/- 2.8 vs. 21.2 +/- 2.5 kg/m2; ASMD = 0.15; p = 0.531)." (p. 5, Section 3.2) 5) "Groups were also balanced on prior sport exposure (5.8 +/- 2.1 vs. 5.5 +/- 2.3 years; ASMD = 0.13; p = 0.571), weekly physical activity (3.4 +/- 1.5 vs. 3.6 +/- 1.6 h; ASMD = 0.13; p = 0.591), and autonomous motivation at pretest (3.30 +/- 0.58 vs. 3.38 +/- 0.54; ASMD = 0.14; p = 0.550)." (p. 5, Section 3.2) 6) "Pretest STEAM knowledge scores were similarly low in both groups (15.2 +/- 4.8 vs. 14.7 +/- 5.2; ASMD = 0.10; p = 0.677). Sex distribution was comparable (21/35 [60.0%] vs. 19/35 [54.3%] male; ASMD = 0.11; chi-square(1) = 0.23, p = 0.629)." (p. 5, Section 3.2) 7) "In week 4, the first author observed one session per group using a structured observation form to verify that STEAM-specific elements were implemented only in the STEAM-RB group. No major protocol deviations were identified." (p. 4, Section 2.6) Detailed Analysis: Criterion D requires that the control group be well documented: size, demographic composition, baseline performance, and the conditions or treatment it received. All four elements are present. The control arm size is stated exactly (n = 35 of 70, with zero attrition). Table 1 provides a full column of baseline data for the traditional group covering age, sex, BMI, prior sport exposure, weekly physical activity, pretest peak racket-head angular velocity, pretest blinded expert rating, pretest autonomous motivation, and pretest STEAM knowledge, each with means, SDs, ASMDs, test statistics, and p values. The condition the control group received is described substantively rather than merely labelled "business as usual": it followed the existing university Roliball syllabus with teacher-directed demonstration, part-whole technique practice, and summative skill testing, delivered in the same facilities on the same two 90-min-per-week schedule. Fidelity monitoring further documents that adherence to the traditional syllabus was checked after each class and that STEAM-specific elements were verified as absent from the control condition, confirming no unintended interventions leaked in. Criterion D is met because the control group's size, baseline demographic and performance characteristics, and instructional conditions are all documented in detail and verified by fidelity checks.

• Level 2 Criteria

  • S

    School-level RCT

    • The trial was single-site with individual-level randomisation, so no randomisation among schools or implementing institutions took place.
    • "The trial was conducted at a single public university with a modest sample (N = 70) and two instructors, limiting generalizability to other institutions and populations."
    • Relevant Quotes: 1) "This single-site, section-stratified, individual-level randomized trial evaluated a STEAM-integrated Roliball (STEAM-RB) curriculum in university physical education." (p. 2, Section 2.1) 2) "Two intact first-year Roliball PE sections at a public university in central China were recruited at the beginning of the semester (N = 70)." (p. 2, Section 2.1) 3) "Within each section, students were individually randomized (1:1) to either the STEAM-RB curriculum or the traditional Roliball syllabus..." (p. 3, Section 2.3) 4) "The trial was conducted at a single public university with a modest sample (N = 70) and two instructors, limiting generalizability to other institutions and populations." (p. 9, Section 4.6) 5) "Future research could extend this work through multi-site trials across different types of universities and regions..." (p. 9, Section 4.6) Detailed Analysis: Criterion S requires randomisation among schools, i.e. among the educational institutions or implementing units, with a described school selection process and number of institutions involved. This trial involved exactly one institution. It is explicitly "single-site", conducted "at a single public university", and the author lists single-site conduct as a limitation and calls for "multi-site trials" in future work. With only one institution there is no institution-level randomisation possible, and in fact the randomised unit was the individual student. Even the intermediate unit available here (the PE section) was used only for stratification, not for allocation. Criterion S is not met because the study was conducted at a single university with individual-level randomisation, so no school- or institution-level randomisation occurred.
  • I

    Independent Conduct

    • The single author both designed the STEAM-RB curriculum and carried out the investigation, analysis, and reporting, with only partial safeguards rather than third-party evaluation.
    • "DZ: Conceptualization, Validation, Writing - review & editing, Project administration, Funding acquisition, Supervision, Resources, Formal analysis, Software, Writing - original draft, Methodology, Data curation, Visualization, Investigation."
    • Relevant Quotes: 1) "Daoqing Zhang. School of Physical Education, Hubei University of Arts and Science, Xiangyang, China" (p. 1) 2) "DZ: Conceptualization, Validation, Writing - review & editing, Project administration, Funding acquisition, Supervision, Resources, Formal analysis, Software, Writing - original draft, Methodology, Data curation, Visualization, Investigation." (p. 9, Author contributions) 3) "The present trial designs and evaluates a theory-grounded STEAM-Roliball (STEAM-RB) curriculum model for university PE, with a specific focus on measurement." (p. 2, Introduction) 4) "This work was supported by the Hubei University of Arts and Science Teaching Research Project (Grant No. JY2025091): 'AI-Enabled Innovation and Practice in University Roliball Teaching.'" (pp. 9-10, Funding) 5) "In week 4, the first author observed one session per group using a structured observation form to verify that STEAM-specific elements were implemented only in the STEAM-RB group." (p. 4, Section 2.6) 6) "An independent researcher generated the random sequence and concealed allocation until assignment." (p. 3, Section 2.3) 7) "Two experienced Roliball instructors not involved in day-to-day teaching independently rated each clip on a 10-point holistic scale ... Raters were blind to group, time point, and study hypotheses." (p. 4, Section 2.5.2) 8) "The protocol was reviewed and approved by the Departmental Research Committee of Hubei University of Arts and Science, which confirmed that the study met local institutional requirements for educational research." (p. 3, Section 2.2) Detailed Analysis: Criterion I requires that the evaluation be conducted independently of the party that designed the intervention, with an explicit statement identifying who conducted the study and their relationship to the designers. This is a single-author paper in which the sole author both designed the intervention and ran the evaluation. The CRediT statement assigns Conceptualization, Methodology, Software, Investigation, Formal analysis, Data curation, Supervision, and Project administration all to DZ. The paper states that "the present trial designs and evaluates" the curriculum, confirming that design and evaluation are the same undertaking by the same person. The author also personally performed the week-4 fidelity observations, so even implementation monitoring was not externalised. The funding grant is an internal teaching-research project at the author's own institution, held by the author. There are genuine partial safeguards: an independent researcher generated and concealed the allocation sequence, and the expert raters were external to day-to-day teaching and blinded to group, time point, and hypotheses. These reduce specific biases in allocation and in one outcome measure. However, they fall well short of the standard's requirement. Unlike the specification's worked examples, where an external agency or a government body led the trial and the developer was excluded from data collection, analysis, and conclusions, here the designer retained control of data collection, analysis, and conclusions. The blinded raters contributed one of five outcomes; the custom STEAM knowledge test, the kinematic analysis, and all statistical modelling remained with the designer. Departmental committee approval is ethics oversight, not independent evaluative conduct. Criterion I is not met because the sole author designed the STEAM-RB curriculum and also conducted the data collection, analysis, and reporting, with only partial safeguards (independent randomisation and blinded raters) rather than independent conduct of the evaluation.
  • Y

    Year Duration

    • The tracking interval was only 8 weeks, far below 75% of an academic year, and criterion T was not met.
    • "The 8-week duration precluded examination of long-term maintenance and may have been too short to affect more stable constructs such as creative disposition."
    • Relevant Quotes: 1) "Participants were allocated to either an 8-week STEAM-RB intervention or a traditional syllabus." (p. 1, Abstract) 2) "Both groups completed two 90-min Roliball classes per week for 8 weeks (16 sessions) in standard indoor PE facilities." (p. 3, Section 2.4) 3) "Thus, there was no attrition over the 8-week intervention, and all randomized participants were included in the primary analyses." (p. 5, Section 3.1) 4) "The 8-week duration precluded examination of long-term maintenance and may have been too short to affect more stable constructs such as creative disposition." (p. 9, Section 4.6) 5) "Future research could extend this work through ... longer interventions spanning a semester or year..." (p. 9, Section 4.6) Detailed Analysis: Criterion Y requires outcomes to be measured at least 75% of a full academic year (roughly 7 months or more of a 9-10 month year) after the intervention begins. Per the criterion-specific instructions, Y cannot be met when T is not met. The tracking interval here is 8 weeks from intervention start to posttest, approximately 20% of a standard academic year. There is no follow-up beyond the immediate posttest; the author explicitly notes that the 8-week duration "precluded examination of long-term maintenance" and identifies year-length implementations as future work. Since criterion T (one term) was not met, criterion Y is automatically not met as well. Criterion Y is not met because the study tracked outcomes for only 8 weeks, far below 75% of an academic year, and criterion T was not met.
  • B

    Balanced Control Group

    • Class time, instructor experience, facilities, and content coverage were matched across arms, and the only extra resource (low-cost smartphone video feedback) is integral to the intervention being tested.
    • "Class time and overall content coverage (e.g., repertoire of Roliball techniques) were matched across groups."
    • Relevant Quotes: 1) "Both groups completed two 90-min Roliball classes per week for 8 weeks (16 sessions) in standard indoor PE facilities." (p. 3, Section 2.4) 2) "Class time and overall content coverage (e.g., repertoire of Roliball techniques) were matched across groups. Students in the STEAM-RB group primarily used their own smartphones for video capture, with shared tablets or instructor devices as backups when needed." (p. 4, Section 2.4) 3) "The traditional curriculum followed the existing university syllabus, emphasizing teacher-directed demonstration, part-whole practice of core techniques (e.g., basic swings, directional control), and summative skill testing." (p. 3, Section 2.4) 4) "The STEAM-RB curriculum adapted this syllabus within Kolb's experiential learning cycle (concrete experience, reflective observation, abstract conceptualization, active experimentation), embedding smartphone-based kinematic feedback and STEAM-focused inquiry tasks into each unit." (p. 4, Section 2.4) 5) "Time-on-task was matched across sections; contamination was minimized by tool-use restrictions in the traditional condition." (Figure 2 note, p. 5) 6) "To reduce instructor effects, an instructor crossover was implemented at week 4: the two experienced Roliball instructors exchanged groups mid-intervention while keeping curriculum condition (STEAM-RB vs. traditional) unchanged. Both instructors had at least 5 years of university PE teaching experience and received joint training on the study protocol before implementation." (p. 3, Section 2.3) 7) "From a curricular standpoint, the STEAM-RB model shows that integrating smartphone-based kinematic feedback and STEAM inquiry into an existing university PE syllabus is feasible without specialized laboratories or additional contact time." (p. 8, Section 4.5) 8) "Low-cost tools (240 fps video, smartphone sensors, GeoGebra) provide data-driven feedback that supports motor skill acquisition and concept formation." (Figure 1 caption, p. 3) Detailed Analysis: Applying the criterion B decision tree: Step 1 - are extra time or budget present? Instructional time is identical: both arms received two 90-min classes per week for 8 weeks in the same facilities, and the paper states directly that "class time and overall content coverage ... were matched across groups" and that the model works "without ... additional contact time". Instructor quality is also balanced, and deliberately so: both instructors had 5+ years of experience, received the same joint protocol training, and were crossed over at week 4 so that each taught each condition. Content coverage of the Roliball technique repertoire was matched, so the control arm was not a thinner version of the same course. The only asymmetric input is material: the STEAM-RB group used smartphones, high-frame-rate video capture, free motion-analysis software and GeoGebra, plus occasional shared tablets or instructor devices as backups. This is a real but very small resource difference - students "primarily used their own smartphones", the tools are explicitly framed as "low-cost", and no specialised laboratory or purchased equipment was required. Under step 1a this is arguably a negligible budget difference. Step 2 - even setting negligibility aside, the additional resource is integral rather than supplementary. The smartphone-based kinematic feedback is not an add-on layered onto a curriculum being tested for something else; it is constitutive of the treatment. The STEAM-RB condition is defined as the Kolb cycle with "smartphone-based kinematic feedback and STEAM-focused inquiry tasks ... embedded into each unit", and the trial's stated aim is to evaluate exactly this technology-and-inquiry package against a business-as-usual syllabus. Withholding the smartphone tooling from the control arm is what defines the contrast; indeed "tool-use restrictions in the traditional condition" were used specifically to preserve it. Balancing the tooling into the control group would dissolve the treatment variable. The control condition is a genuine active comparator, not an empty one: control students spent the same hours doing structured Roliball instruction under an equally experienced instructor covering the same technique repertoire. The core educational inputs of time, staffing, facilities, and content are equal across arms. Criterion B is met because instructional time, instructor quality, facilities, and content coverage were explicitly matched across groups, and the only additional resource (low-cost smartphone video and analysis tooling) is integral to the STEAM-RB package being tested rather than a separable confounding add-on.

• Level 3 Criteria

  • R

    Reproduced

    • Internet searching found no independent replication of this STEAM-Roliball trial; the paper presents itself as filling a gap and lists replication in other sports and sites as future work.
    • "In higher education, however, STEAM-integrated PE units remain rare, and few have been examined using randomized designs or multi-domain outcome measures."
    • Relevant Quotes: 1) "However, rigorous evidence regarding the effectiveness of interdisciplinary approaches, such as STEAM (Science, Technology, Engineering, Arts, and Mathematics), remains limited in higher education settings." (p. 1, Abstract) 2) "In higher education, however, STEAM-integrated PE units remain rare, and few have been examined using randomized designs or multi-domain outcome measures." (p. 2, Introduction) 3) "While previous research has explored STEAM in general education, its application to complex motor skill acquisition in university PE remains understudied." (p. 2, Introduction) 4) "Although Roliball is a niche, culturally specific sport, the design principles tested here are not unique to RB." (p. 9, Section 4.5) 5) "Future adaptations could test analogous modules in badminton, tennis, or pickleball, preserving the data-informed inquiry structure while drawing on locally meaningful cultural narratives (Steyn and Rix, 2025)." (p. 9, Section 4.5) 6) "Future research could extend this work through multi-site trials across different types of universities and regions, longer interventions spanning a semester or year..." (p. 9, Section 4.6) Detailed Analysis: Criterion R requires that this specific study or its central experimental claim have been independently replicated by a different research team in a different context, published in a peer-reviewed journal. Internet verification (July 2026) was carried out across general web search, Frontiers, PubMed-indexed sources and Google-Scholar-style queries, using combinations of "Roliball", "STEAM", "university physical education", "randomized controlled trial", "smartphone kinematic feedback" and the author name "Daoqing Zhang". No independent replication of this trial was found. The only other Roliball RCT located is "Efficacy of Tai Chi and Roliball Exercise on Balance, Mobility, and Cognitive Function in Community-Dwelling Older Adults: A Randomized Controlled Trial" (Clinical Interventions in Aging, 2025), which studies older adults' balance, mobility and cognition rather than a STEAM-integrated university PE curriculum, uses different authors, a different population and different outcomes, and therefore is not a replication of this study. No quotes from any replication study can be provided because no such study exists. The paper itself reports no replication and positions itself as filling a gap: STEAM-integrated PE units in higher education "remain rare, and few have been examined using randomized designs". The cited literature (Yuan et al., 2022 on STEAM and primary-school physical exercise; Tran et al., 2021 on STEAM-based courses; Moedinger et al., 2022 and Meng, 2023 on video feedback and digital technology in PE) provides background support for the general mechanisms but does not replicate this STEAM-RB curriculum, this outcome battery, or this design. The author explicitly frames replication in other sports and other sites as future work not yet done. Given that the paper was published on 1 May 2026 and describes a niche, culturally specific sport (Roliball) with a bespoke curriculum and bespoke instruments, an independent replication of this particular trial by another team could not plausibly exist yet, and none is identified. Criterion R is not met because internet searching found no independent replication of this specific STEAM-Roliball trial; the paper itself positions replication and multi-site extension as future work.
  • A

    All-subject Exams

    • Criterion E was not met, and all outcomes were confined to the intervention's own Roliball/STEAM domain with no assessment of other school subjects.
    • "Finally, the STEAM knowledge test was tightly aligned to RB-specific content, so scores should not be interpreted as reflecting general STEAM competence..."
    • Relevant Quotes: 1) "Primary outcomes included kinematic performance (peak racket-head angular velocity via video analysis), blinded expert ratings of technique, and STEAM-specific knowledge." (p. 1, Abstract) 2) "The following outcomes were assessed at pretest and posttest: kinematic performance, blinded expert ratings, STEAM knowledge, physical activity motivation, and creative disposition." (p. 4, Section 2.5) 3) "Task-specific STEAM knowledge was assessed through a 15-item criterion-referenced test aligned with the core STEAM concepts embedded in the curriculum..." (p. 4, Section 2.5.3) 4) "Finally, the STEAM knowledge test was tightly aligned to RB-specific content, so scores should not be interpreted as reflecting general STEAM competence..." (p. 9, Section 4.6) 5) "The STEAM knowledge test showed how curricular tasks can be mapped onto a focused assessment instrument with acceptable content validity, rather than relying on generic science or mathematics tests." (p. 8, Section 4.4) Detailed Analysis: Criterion A requires that impact be measured across all main subjects of the curriculum using standardised exam-based assessments, and the criterion-specific instruction states that if criterion E is not met, criterion A cannot be met either. Criterion E was not met: every outcome instrument was custom-built for this trial or is a self-report psychometric scale rather than a standardised exam. That alone is decisive. Independently, the subject coverage also fails. All outcomes lie within the single PE unit being taught: forehand-drive kinematics, expert-rated technique of the same stroke, RB-specific STEAM knowledge, PE motivation, and creative disposition. The author states explicitly that the knowledge test should not be read as measuring general STEAM competence. No other university subject the students were studying - mathematics, sciences, languages, or any other course in their first-year programme - was assessed, so any spillover effect (positive or negative) on the wider curriculum is unmeasured. No exception rationale is offered for restricting measurement to the intervention's own domain. Criterion A is not met because criterion E was not met, and because all outcomes were confined to the intervention's own Roliball/STEAM domain with no assessment of other subjects.
  • G

    Graduation Tracking

    • Measurement stopped at the 8-week posttest with no follow-up toward graduation of these first-year undergraduates, no follow-up publication was found, and criterion Y was not met.
    • "Thus, there was no attrition over the 8-week intervention, and all randomized participants were included in the primary analyses."
    • Relevant Quotes: 1) "The following outcomes were assessed at pretest and posttest: kinematic performance, blinded expert ratings, STEAM knowledge, physical activity motivation, and creative disposition." (p. 4, Section 2.5) 2) "Thus, there was no attrition over the 8-week intervention, and all randomized participants were included in the primary analyses." (p. 5, Section 3.1) 3) "The 8-week duration precluded examination of long-term maintenance and may have been too short to affect more stable constructs such as creative disposition." (p. 9, Section 4.6) 4) "All students were first-year undergraduates enrolled to meet a mandatory PE requirement and had no prior formal Roliball training." (pp. 2-3, Section 2.1) 5) "Future research could extend this work through multi-site trials across different types of universities and regions, longer interventions spanning a semester or year..." (p. 9, Section 4.6) Detailed Analysis: Criterion G requires that participants be followed until they graduate from the relevant educational stage, and the criterion-specific instruction states that if criterion Y is not met, criterion G cannot be met. Criterion Y was not met (8-week tracking), which settles the matter. Internet verification (July 2026) searched for subsequent or follow-up publications by the same author tracking this cohort, using the author name "Daoqing Zhang", the affiliation "Hubei University of Arts and Science", and the terms "Roliball", "STEAM", "follow-up" and "graduation". No follow-up paper, cohort-tracking study, or longer-term report on these participants was located in any available source, and the author appears to have no other indexed Roliball trial publications. No quotes from a follow-up publication can be provided because no such publication was found. On the substance, the participants were first-year undergraduates, so graduation would fall roughly three years after the trial. Measurement stopped at the posttest at the end of week 8; the author explicitly states that the design "precluded examination of long-term maintenance". No follow-up study, planned follow-up, or graduation-outcome data collection is described or referenced anywhere in the paper. Criterion G is not met because measurement stopped at the 8-week posttest with no tracking toward graduation, no follow-up publication was found by internet search, and criterion Y was not met.
  • P

    Pre-Registered

    • The paper explicitly states the trial was not prospectively registered, and no registry record was found by internet search.
    • "This study does not meet the World Health Organization definition of a clinical trial because the primary outcomes concerned educational and kinematic performance within a routine curriculum context rather than health-related clinical endpoints. The trial was therefore not prospectively registered."
    • Relevant Quotes: 1) "This study does not meet the World Health Organization definition of a clinical trial because the primary outcomes concerned educational and kinematic performance within a routine curriculum context rather than health-related clinical endpoints. The trial was therefore not prospectively registered." (p. 4, Section 2.6) 2) "The protocol was reviewed and approved by the Departmental Research Committee of Hubei University of Arts and Science, which confirmed that the study met local institutional requirements for educational research." (p. 3, Section 2.2) 3) "Analyses were pre-specified around primary, secondary, and exploratory outcomes." (p. 5, Section 2.7) 4) "In line with meta-analytic evidence that such traits are relatively stable and slow to change, CPS scores were designated a priori as exploratory outcomes." (p. 5, Section 2.6.2) 5) "An a priori power analysis (G*Power 3.1) (Faul et al., 2007) indicated that a total of 68 participants (34 per arm) would be sufficient to detect a medium effect size (f = 0.25) on a primary outcome using ANCOVA with one covariate (alpha = 0.05, power = 0.80)." (p. 3, Section 2.3) Detailed Analysis: Criterion P requires a publicly pre-registered full protocol - hypotheses, methods, and planned analyses - deposited in a registry before data collection began, with a verifiable link and date. The paper states outright that "the trial was therefore not prospectively registered". There is no registry name, no registration identifier, and no registration date anywhere in the article. Internet verification (July 2026) checked for a registry record in ChiCTR, ClinicalTrials.gov, OSF and ISRCTN using the trial's identifying terms (Roliball, STEAM, Hubei University of Arts and Science, university physical education); no registration record for this trial was found, consistent with the author's own statement. The author's justification (that the study falls outside the WHO clinical-trial definition) explains why registration was skipped but does not satisfy the criterion, which under the ERCT standard applies to educational RCTs regardless of clinical-trial status. The paper does contain internal markers of planning: an a priori power analysis, outcomes described as "pre-specified" into primary/secondary/exploratory tiers, creativity designated "a priori" as exploratory, and departmental committee approval of a protocol. These are good practice and reduce, but do not eliminate, selective-reporting concerns. None of them is a public, timestamped, verifiable pre-registration, so none can be independently checked against what was ultimately reported. Internal ethics approval is not a registry entry. Criterion P is not met because the paper explicitly states the trial was not prospectively registered, no registry record was found by internet search, and no registry link, identifier, or date is provided.