Combining Problem-Based Learning Methods With the WeChat Platform in Teaching Ophthalmology: Randomized Controlled Trial

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomization was done by intact student groups (cluster-style) rather than mixing treatment and control within the same instructional group.
    • "Participants were randomly assigned into 6 groups of 18 students each using Microsoft Excel to generate random numbers to ensure unbiased distribution." (p. 2)
    • Relevant Quotes: 1) "Participants were randomly assigned into 6 groups of 18 students each using Microsoft Excel to generate random numbers to ensure unbiased distribution." (p. 2) 2) "Three groups (18 students for each group) were randomly selected to take part in the WeChat-based PBL program." (p. 3) 3) "Three groups were randomly selected to receive PBL using WeChat as the platform, while the remaining 3 groups received conventional teaching." (p. 1) Detailed Analysis: Criterion C requires a classroom-level (or stronger) unit of randomization to reduce contamination across conditions. The paper describes that students were first assigned into groups of 18, and then entire groups were randomly selected into WeChat-based PBL versus conventional teaching. This is consistent with cluster assignment by intact learning groups rather than assigning different students within a single group/class to different interventions. The intervention is not described as one-to-one tutoring, so the tutoring exception does not apply. Final Summary: Criterion C is met because the randomization and assignment were performed at an intact group (cluster) level.
  • E

    Exam-based Assessment

    • Outcomes were measured using an instructor-created exam derived from a textbook, not a widely recognized standardized exam.
    • "The examination questions were formulated based on the 9th edition of Ophthalmology, an ophthalmology textbook published by the People’s Medical Publishing House [23]." (p. 4)
    • Relevant Quotes: 1) "Both groups then undertook an identical closed-book examination comprising essay questions, true or false questions, judgment questions, and case analyses to assess students’ memory, comprehension, and clinical application abilities." (p. 4) 2) "The examination questions were formulated based on the 9th edition of Ophthalmology, an ophthalmology textbook published by the People’s Medical Publishing House [23]." (p. 4) 3) "Notably, both the examination format and evaluation criteria were standardized across both groups." (p. 4) Detailed Analysis: Criterion E requires standardized exam-based assessments (eg, national or state standardized tests), not assessments created for the specific study or course. The paper describes a closed-book exam covering multiple question types, with questions formulated from a specific ophthalmology textbook. Although the exam was the same for both arms (standardized within the trial), it is not presented as an external, widely recognized standardized exam. Final Summary: Criterion E is not met because the main outcome assessment appears to be an instructor-created, course-aligned exam rather than a widely recognized standardized exam.
  • T

    Term Duration

    • The intervention and primary outcome measurement occurred over weeks, with the exam administered one week after course completion, which is shorter than one academic term.
    • "The closed-book examination was administered 1 week after course completion to allow students sufficient time for review while minimizing long-term memory effects." (p. 6)
    • Relevant Quotes: 1) "Table 1. Timeline of the WeChat-based problem-based learning activities." (p. 2) 2) "The closed-book examination was administered 1 week after course completion to allow students sufficient time for review while minimizing long-term memory effects." (p. 6) Detailed Analysis: Criterion T requires outcomes to be measured at least one full academic term after the intervention begins (typically about 3-4 months), even if the instructional activities are shorter. The paper presents a week-by-week timeline (Table 1) for the instructional activities and states the exam occurred one week after course completion. This indicates a short start-to-measurement window measured in weeks rather than an academic term. Final Summary: Criterion T is not met because the outcome measurement was conducted within weeks rather than at least one term after the intervention began.
  • D

    Documented Control Group

    • The paper describes the traditional-teaching control condition and reports baseline demographics for both groups.
    • "There was no significant difference in gender and average age between the two groups (Table 3)." (p. 5)
    • Relevant Quotes: 1) "Among them, 54 (50%) were randomly selected to participate in the WeChat-based PBL, while the other 54 (50%) received traditional teaching." (p. 1) 2) "The traditional teaching model involved the instructor predominantly delivering comprehensive explanations of foundational knowledge, adhering to a prescribed multimedia teaching outline." (p. 4) 3) "There was no significant difference in gender and average age between the two groups (Table 3)." (p. 5) Detailed Analysis: Criterion D requires clear documentation of the control group (what it received) and who was in it (eg, demographics and/or baseline performance). The paper specifies the control condition as "traditional teaching" and provides a concrete description of how it was delivered. It also reports sample sizes (n=54 per arm) and baseline demographic comparisons (age and sex), stating there was no significant difference between groups. Final Summary: Criterion D is met because the control condition and baseline participant characteristics are explicitly documented.

• Level 2 Criteria

  • S

    School-level RCT

    • Randomization occurred within one institution among student groups, not across multiple schools or institutional sites.
    • "A total of 108 fifth-year undergraduate students from Xiangya Medical College of Central South University, all of whom had passed the Chinese National Entrance Examination, participated in the study." (p. 2)
    • Relevant Quotes: 1) "A total of 108 fifth-year undergraduate students from Xiangya Medical College of Central South University, all of whom had passed the Chinese National Entrance Examination, participated in the study." (p. 2) 2) "Participants were randomly assigned into 6 groups of 18 students each using Microsoft Excel to generate random numbers to ensure unbiased distribution." (p. 2) Detailed Analysis: Criterion S requires randomization at the school/site level (ie, among multiple schools or comparable institutional sites). All participants were from a single medical college within one university, and randomization was among student groups within that one institution. There is no indication that multiple schools, campuses, or distinct sites were the unit of randomization. Final Summary: Criterion S is not met because randomization was within a single institution rather than across multiple schools/sites.
  • I

    Independent Conduct

    • The authors designed the intervention and also collected and analyzed the data; the paper does not document independent external conduct of the study.
    • "WJ and FF designed the experiments and drafted the manuscript. FF and BB collected the data and performed the statistical analysis." (p. 8)
    • Relevant Quotes: 1) "WJ and FF designed the experiments and drafted the manuscript. FF and BB collected the data and performed the statistical analysis." (p. 8) 2) "To maintain objectivity in scoring the theoretical examination, 3 instructors performed a blinded evaluation of the students’ responses." (p. 4) Detailed Analysis: Criterion I requires that evaluation conduct be independent from the designers of the intervention to reduce bias in implementation, measurement, and analysis. The authors explicitly report that they designed the experiments, collected the data, and performed the analysis. Although exam scoring was blinded and done by 3 instructors, the paper does not establish that the overall trial conduct and analysis were performed by an independent third-party evaluator separate from the intervention designers. Final Summary: Criterion I is not met because the study’s design, data collection, and analysis were performed by the author team with no documented independent evaluator.
  • Y

    Year Duration

    • Outcomes were measured within weeks rather than over at least 75% of an academic year, and Y is also not met because T is not met.
    • "The closed-book examination was administered 1 week after course completion to allow students sufficient time for review while minimizing long-term memory effects." (p. 6)
    • Relevant Quotes: 1) "Table 1. Timeline of the WeChat-based problem-based learning activities." (p. 2) 2) "The closed-book examination was administered 1 week after course completion to allow students sufficient time for review while minimizing long-term memory effects." (p. 6) Detailed Analysis: Criterion Y requires outcomes to be measured at least 75% of an academic year after the intervention begins. The intervention is presented in a week-by-week timeline and the main exam outcome is measured one week after course completion, indicating a short duration measured in weeks. Additionally, per the ERCT rule provided, if criterion T is not met then criterion Y is not met; this paper does not meet T. Final Summary: Criterion Y is not met because the study duration is far shorter than an academic year and because criterion T is not met.
  • B

    Balanced Control Group

    • The WeChat-based PBL condition required extra out-of-class effort and instructor monitoring without clearly documented time- and effort-matched inputs in the control condition.
    • "The new teaching methods were implemented outside their regular class schedules, primarily during evenings and weekends." (p. 2)
    • Relevant Quotes: 1) "The new teaching methods were implemented outside their regular class schedules, primarily during evenings and weekends." (p. 2) 2) "This approach presented unique challenges, including an increased workload for both students and instructors, as it required students to allocate additional time beyond their standard academic hours to participate in the study." (p. 2) 3) "Both intervention and control groups received instruction in dedicated classrooms at the hospital, which facilitated a conducive learning environment (Table 1)." (p. 2) 4) "Second, questionnaire results indicated that students in the WeChat-based PBL group spent more extracurricular time preparing course materials." (p. 8) Detailed Analysis: Criterion B evaluates whether time/resources are balanced across conditions, unless the study explicitly frames added resources as the treatment variable. The paper indicates the study activities occurred outside regular schedules and required additional time beyond standard academic hours. It also explicitly notes that WeChat-based PBL students spent more extracurricular time preparing course materials, implying additional time-on-task for the intervention condition. While both arms received instruction in dedicated classrooms, the paper does not document an equal-time, time-equivalent active control to match the WeChat-based discussion, preparation, and monitoring demands. The study is framed as testing "WeChat-based PBL" versus "traditional teaching," not as explicitly testing the effect of providing extra time/resources as the treatment variable. Therefore, unbalanced added time/effort is a confound rather than an intended manipulated input. Final Summary: Criterion B is not met because the intervention appears to add meaningful extra time/effort resources without clearly matched control-group inputs.

• Level 3 Criteria

  • R

    Reproduced

    • No independent replication of this specific trial was found in the paper or via a targeted literature search.
    • Relevant Quotes: 1) (No statements describing an independent replication of this study were identified in the paper.) (not applicable) Detailed Analysis: Criterion R requires that the study be independently replicated by a different team in a different context and published in a peer-reviewed outlet. The paper itself does not report any replication. A targeted web search for citations/replications of this specific JMIR Med Educ 2026 study (doi: 10.2196/65279) did not identify a published independent replication study. Final Summary: Criterion R is not met because no independent replication of this specific study was identified.
  • A

    All-subject Exams

    • The study does not use standardized exams (E not met) and assesses only an ophthalmology topic rather than all main subjects.
    • "The examination questions were formulated based on the 9th edition of Ophthalmology, an ophthalmology textbook published by the People’s Medical Publishing House [23]." (p. 4)
    • Relevant Quotes: 1) "The examination questions were formulated based on the 9th edition of Ophthalmology, an ophthalmology textbook published by the People’s Medical Publishing House [23]." (p. 4) 2) "We selected primary angle-closure glaucoma (PACG) as the topic for applying the WeChat-based PBL approach in this study, as the diagnosis and treatment of PACG represent crucial skills that students must acquire." (p. 3) Detailed Analysis: Criterion A requires exam-based assessment across all main subjects, and it also depends on criterion E being met. Criterion E is not met because the primary assessment is a textbook-derived exam rather than a widely recognized standardized exam; therefore, criterion A cannot be met. Substantively, the content assessed is ophthalmology (PACG), not all main subjects in the curriculum. Final Summary: Criterion A is not met because criterion E is not met and the study does not assess outcomes across all main subjects.
  • G

    Graduation Tracking

    • The study did not track outcomes through graduation, and G is also not met because Y is not met.
    • "However, as the students involved in this study have already graduated, it was not possible to collect further feedback from them." (p. 8)
    • Relevant Quotes: 1) "However, as the students involved in this study have already graduated, it was not possible to collect further feedback from them." (p. 8) 2) "The closed-book examination was administered 1 week after course completion to allow students sufficient time for review while minimizing long-term memory effects." (p. 6) Detailed Analysis: Criterion G requires outcome tracking through to graduation. The paper reports only an end-of-course questionnaire and a short-term exam one week after course completion. The limitations section notes that the students had already graduated and further feedback could not be collected, which supports that the study itself did not include graduation- based longitudinal outcome tracking. Additionally, per the ERCT rule provided, if criterion Y is not met then criterion G is not met; this paper does not meet Y. A targeted search for follow-up publications by the same authors (Fang Fang, Bing Bu, Wenmin Jiang) reporting longer- term tracking of the same cohort through graduation did not identify any such follow-up paper. Final Summary: Criterion G is not met because the study does not report graduation tracking and criterion Y is not met.
  • P

    Pre-Registered

    • No evidence of a pre-registered protocol (with an ID and a registration date before data collection) was found.
    • Relevant Quotes: 1) (No trial registry ID, registry link, or pre-registration statement was identified in the paper.) (not applicable) Detailed Analysis: Criterion P requires a publicly accessible pre-registered protocol and evidence that registration occurred before data collection started. The paper includes ethics approval and standard methods reporting, but it does not provide a trial registration section, registry name, registration number, or registration date. Searches for a registry entry linked to this study did not identify a public pre-registration record that can be matched to the paper and verified as pre-data-collection. Final Summary: Criterion P is not met because no verifiable pre-registration information (registry and pre-study date) is provided.