Enhancing physics education through flipped problem-based learning (PBL): improving comprehension and problem-solving skills in undergraduate students

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • The study explicitly used a quasi-experimental design with no randomisation, so no class-level RCT exists.
    • the groups were not randomly assigned, and unmeasured factors (e.g., prior preparation, study habits) could have contributed to the observed gap.
    • Relevant Quotes: 1) "A total of 200 first-year students were assigned to either a lecture-based control group or a flipped-PBL group (n=100 per group)." (p. 1801, Abstract) 2) "All participants were first-year students enrolled in a compulsory physics course and were assigned in equal numbers to a flipped PBL group and a lecture-based control group (n=100 per group)." (p. 1805) 3) "Although both sections followed the same syllabus and assessment blueprint, the groups were not randomly assigned, and unmeasured factors (e.g., prior preparation, study habits) could have contributed to the observed gap." (p. 1809) 4) "First, the study employed a quasi-experimental design and was conducted within a single institutional context." (p. 1812) Detailed Analysis: Criterion C requires a properly implemented randomised controlled trial in which the unit of randomisation is at least the class (or a stronger school-level unit), unless a one-to-one tutoring exception applies. The paper does not describe any randomisation procedure. Students were simply "assigned" to two intact sections, and the authors explicitly acknowledge that "the groups were not randomly assigned" and that "the study employed a quasi-experimental design." There is no random allocation at the student, class, or school level, and no tutoring exception applies. Criterion C is not met because the study is an explicitly quasi-experimental design with no randomisation at any level.
  • E

    Exam-based Assessment

    • The outcome was a locally developed department final exam, not a widely recognised standardised assessment.
    • both groups completed the same department-level final examination, which consisted of 20 items
    • Relevant Quotes: 1) "To ensure fairness and validity, both groups completed the same department-level final examination, which consisted of 20 items" (p. 1807) 2) "The test blueprint covered the domains of knowledge, comprehension, and application in alignment with the course learning outcomes. Two faculty members reviewed the examination for content validity, and all responses were graded blindly to reduce potential scoring bias. Internal consistency was acceptable, with Cronbach's alpha =.82." (p. 1807) 3) "In this study, we deliberately mapped the problem scenarios and in-class tasks to the same content domains and cognitive levels covered in the final exam blueprint." (p. 1805) Detailed Analysis: Criterion E requires a standardised, widely recognised exam-based assessment (e.g., a national or state-wide standardised test) rather than an instrument built by the researchers for the study. The outcome measure here is a "department-level final examination" of 20 items authored and reviewed internally by two faculty members and deliberately aligned with the study's own problem scenarios. Although content validity was checked and Cronbach's alpha was reported, this is a locally constructed course exam, not a standardised, externally recognised test. The deliberate alignment of the exam with the intervention activities further undermines its status as an independent standardised measure. Criterion E is not met because the assessment was a locally developed department final exam rather than a widely recognised standardised exam.
  • T

    Term Duration

    • The intervention spanned only 12 class hours with no term-long interval or follow-up to the measurement.
    • The total instructional duration was 12 class hours (6 sessions x 2 class hours).
    • Relevant Quotes: 1) "The intervention was implemented during the second semester of the first year and focused on the topic of electromagnetic induction, in alignment with the national introductory undergraduate physics curriculum in China." (p. 1806) 2) "The total instructional duration was 12 class hours (6 sessions x 2 class hours)." (p. 1806) 3) "Post-class consolidation ... Complete homework problems (Calculation & Design reflection)." (p. 1816, Appendix D) Detailed Analysis: Criterion T requires that outcomes be measured at least one full academic term (approximately 3-4 months) after the intervention begins, either through a term-long intervention or term-long follow-up tracking from the start. This intervention consisted of only 12 class hours across 6 sessions on a single topic (electromagnetic induction). The final examination was the outcome measure, administered around the end of this short module. The paper provides no dates establishing that the interval from intervention start to outcome measurement spanned a full term, and there is no term-long follow-up tracking. Criterion T is not met because the intervention lasted only 12 class hours with no evidence of at least a full academic term between intervention start and outcome measurement.
  • D

    Documented Control Group

    • The control group's size, demographics, baseline scores, and lecture-based condition are clearly documented.
    • Baseline comparability was verified using independent-samples t-tests on prior physics course grades, which indicated no significant between-group differences (p>.10).
    • Relevant Quotes: 1) "All participants were first-year students enrolled in a compulsory physics course and were assigned in equal numbers to a flipped PBL group and a lecture-based control group (n=100 per group). The groups were comparable in gender (65 males, 135 females) and age (18-22 years). Baseline comparability was verified using independent-samples t-tests on prior physics course grades, which indicated no significant between-group differences (p>.10)." (p. 1805) 2) "The control group received traditional lecture-based learning (LBL), in which the instructor explained core concepts and demonstrated derivations on the board." (p. 1806) 3) "Compared to the lecture-based group (M=22.9, SD=2.1), the flipped PBL group scored higher (M=26.3, SD=1.8)" (p. 1810); "Lecture-based group 100 81.03 +/- 6.15" (Table 1, p. 1807) Detailed Analysis: Criterion D requires clear documentation of the control group's characteristics, size, baseline performance, and conditions. The paper documents the control group's size (n=100), demographics (gender and age reported for the full comparable sample), baseline performance (prior physics course grades tested for comparability), and its condition (traditional lecture-based learning by the same instructors following the same syllabus). Baseline and outcome descriptive statistics for the control group are reported in Table 1 and the results text. Criterion D is met because the control group's size, demographics, baseline comparability, and instructional condition are clearly documented.

• Level 2 Criteria

  • S

    School-level RCT

    • A single-site quasi-experimental study with no randomisation cannot meet school-level RCT requirements.
    • the study employed a quasi-experimental design and was conducted within a single institutional context.
    • Relevant Quotes: 1) "A total of 200 undergraduate physics majors at a public university in northern China participated in this study." (p. 1805) 2) "First, the study employed a quasi-experimental design and was conducted within a single institutional context." (p. 1812) 3) "the groups were not randomly assigned" (p. 1809) Detailed Analysis: Criterion S requires randomisation at the school level (the educational institution or implementing unit). This study was conducted at a single university with two intact sections and no randomisation of any kind, let alone at the school level. Only one institution was involved, so school-level random assignment is impossible here. Criterion S is not met because the study was conducted at a single institution with no school-level (or any) randomisation.
  • I

    Independent Conduct

    • The same team designed, delivered, and analysed the study, with no independent external evaluation.
    • Both groups were taught by the same instructors, followed the same syllabus, and were evaluated under the same grading framework.
    • Relevant Quotes: 1) "Both groups were taught by the same instructors, followed the same syllabus, and were evaluated under the same grading framework." (p. 1806) 2) "The experimental group participated in the flipped-PBL intervention, in which students completed pre-class microlectures and engaged in problem-based classroom activities designed to align with specific course outcomes." (p. 1806) 3) "Two faculty members reviewed the examination for content validity, and all responses were graded blindly to reduce potential scoring bias." (p. 1807) Detailed Analysis: Criterion I requires that the study be conducted independently from those who designed the intervention, typically via an external evaluation team. Here the same authors designed the flipped-PBL model, delivered it through the same instructors who taught the control group, and analysed the outcomes. Although exam responses were graded blindly, blind grading is not the same as independent conduct of the trial. There is no external or third-party evaluator overseeing design, delivery, collection, or analysis. Criterion I is not met because the same team designed, implemented, and evaluated the intervention with no independent external evaluation.
  • Y

    Year Duration

    • A 12-class-hour intervention is far short of a year, and Term Duration (T) was not met.
    • The total instructional duration was 12 class hours (6 sessions x 2 class hours).
    • Relevant Quotes: 1) "The total instructional duration was 12 class hours (6 sessions x 2 class hours)." (p. 1806) 2) "The intervention was implemented during the second semester of the first year and focused on the topic of electromagnetic induction" (p. 1806) Detailed Analysis: Criterion Y requires outcomes to be measured at least 75% of a full academic year (roughly 9-10 months) after the intervention begins. Per the criterion-specific rule, if Criterion T (Term Duration) is not met, then Y cannot be met. Here the intervention was only 12 class hours on a single topic, and T is not met; the study spans a small fraction of an academic year with no year-long follow-up. Criterion Y is not met because the intervention and follow-up were far shorter than an academic year, and Criterion T was not met.
  • B

    Balanced Control Group

    • Instructional time and resources were matched across groups, and the short pre-class microlectures are integral to the flipped design rather than a confounding add-on.
    • Both groups were taught by the same instructors, followed the same syllabus, and were evaluated under the same grading framework. The total instructional duration was 12 class hours (6 sessions x 2 class hours).
    • Relevant Quotes: 1) "While both groups used the same basic resources to study electromagnetic induction, the flipped-PBL group engaged in pre-class microlectures and in-class group problem-solving." (p. 1801, Abstract) 2) "Both groups were taught by the same instructors, followed the same syllabus, and were evaluated under the same grading framework. The total instructional duration was 12 class hours (6 sessions x 2 class hours)." (p. 1806) 3) "These videos were concise and focused, typically lasting no more than a few minutes, and were designed to introduce the core concepts needed for the upcoming session." (p. 1806) 4) "In addition to the final examination, in-class performance accounted for 30% of the overall course grade in both the flipped-PBL and lecture-based sections. To maintain comparability across conditions, we used identical rubric dimensions and scoring criteria for the two groups." (p. 1806) Detailed Analysis: Criterion B compares the time, budget, and materials given to intervention and control groups, unless additional resources are the explicit treatment variable. Here both groups shared the same instructors, the same syllabus, the same total instructional duration (12 class hours), the same basic resources, and the same grading framework. The manipulated difference is the instructional method (flipped-PBL versus lecture), not extra time or budget. The flipped group's pre-class microlectures are short videos ("no more than a few minutes") that shift foundational content delivery outside class; moving content out of class is the defining, integral feature of a flipped design rather than a separable add-on of extra resources. The additional pre-class viewing is therefore integral to the intervention being tested, and in-class time and assessment were matched across groups. Criterion B is met because instructional time, resources, and assessment were matched across groups, and the only additional element (short pre-class microlectures) is integral to the flipped design being tested rather than a confounding extra resource.

• Level 3 Criteria

  • R

    Reproduced

    • No independent replication of this specific study exists; the authors only call for future verification.
    • the developmental artifacts provided here (Appendices B-D; Dataset S1; Fig. 1) are intended to support adaptation and future verification in other settings.
    • Relevant Quotes: 1) "Future research should extend this work across multiple institutional contexts and further examine how implementation fidelity and learner variability shape the outcomes of flipped-PBL instruction." (p. 1813) 2) "the developmental artifacts provided here (Appendices B-D; Dataset S1; Fig. 1) are intended to support adaptation and future verification in other settings." (p. 1812) Detailed Analysis: Criterion R requires that this specific study be independently replicated by a different research team in a different context and published in a peer-reviewed journal. The paper itself calls for future verification and multisite replication, indicating no such replication yet exists. The cited flipped-learning and PBL literature provides general background but does not constitute an independent replication of this particular single-site quasi-experiment at Harbin Normal University. An internet check was attempted (WebSearch quota was exhausted; the DOI landing page was confirmed to resolve), and no external replication of this specific study was identified; the study was only published in June 2026. Criterion R is not met because no independent replication of this specific study has been published; the authors themselves call for future verification.
  • A

    All-subject Exams

    • Only physics was assessed and Criterion E was not met, so the all-subject requirement fails.
    • both groups completed the same department-level final examination, which consisted of 20 items
    • Relevant Quotes: 1) "The intervention was implemented during the second semester of the first year and focused on the topic of electromagnetic induction" (p. 1806) 2) "both groups completed the same department-level final examination, which consisted of 20 items" (p. 1807) 3) "Appendix C. Final exam blueprint and sample items topic: electromagnetic induction" (p. 1815) Detailed Analysis: Criterion A requires that impact be measured across all main subjects using standardised exam-based assessments, and it is conditional on Criterion E: if E is not met, A cannot be met. Criterion E is not met (the outcome was a locally developed department exam, not a standardised test). Additionally, the study measured only physics (electromagnetic induction) and no other core subjects. Criterion A is not met because Criterion E was not met and only a single subject (physics) was assessed.
  • G

    Graduation Tracking

    • No graduation tracking is reported and Criterion Y was not met.
    • Relevant Quotes: 1) "The total instructional duration was 12 class hours (6 sessions x 2 class hours)." (p. 1806) 2) "Future research should extend this work across multiple institutional contexts and further examine how implementation fidelity and learner variability shape the outcomes of flipped-PBL instruction." (p. 1813) Detailed Analysis: Criterion G requires follow-up tracking of participants until their graduation, and it is conditional on Criterion Y: if Y is not met, G cannot be met. Criterion Y is not met. Moreover, the study measured outcomes at the end of a short 12-class-hour module with no long-term follow-up and no graduation tracking of the first-year cohort. An internet check for subsequent graduation-tracking papers by the same authors was attempted (WebSearch quota was exhausted); none could be identified, and none would be expected for a module published in June 2026. Criterion G is not met because there is no graduation tracking and Criterion Y was not met.
  • P

    Pre-Registered

    • Only a vague mention of pre-specified analysis plans is given, with no registry link, ID, or date before data collection.
    • As was noted in the pre-specified analyses plans, those students who were not present at the class meetings and the final exams were omitted in the comparisons for statistics
    • Relevant Quotes: 1) "As was noted in the pre-specified analyses plans, those students who were not present at the class meetings and the final exams were omitted in the comparisons for statistics, but were included in the completion rate for transparency purposes." (p. 1805) 2) "Descriptive statistics are reported as means +/- standard deviations (M +/- SD). Between-group differences were analyzed using two-tailed independent-samples Student's t tests" (p. 1808) Detailed Analysis: Criterion P requires a full study protocol pre-registered on a recognised registry before data collection began, including a registry link/ID and a registration date that precedes data collection. The paper mentions "pre-specified analyses plans" but provides no registry platform, no registration ID, no link, and no date, and gives no evidence that any registration preceded data collection. There is no registry reference to check against an external database. A passing reference to a pre-specified analysis plan does not satisfy the pre-registration requirement. Criterion P is not met because no registry link, ID, or pre-registration date is provided and there is no evidence of registration before data collection.