Effectiveness of a Blended Learning Program for Enhancing Mental Health Literacy Among University Students: A Randomized Educational Study

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomisation was performed at the individual student level, not at the class or school level, and the intervention is a group curriculum rather than personal tutoring, so the exception does not apply.
    • "Participants were randomly assigned to either the blended learning educational program group or the waitlist control group at a 1:1 ratio."
    • Relevant Quotes: 1) "Randomization followed a computer-generated sequence created by an independent statistician not otherwise involved in the study. We employed block randomization with varying block sizes of four and six to ensure balanced allocation while maintaining allocation concealment." (Section 3.1) 2) "Participants were randomly assigned to either the blended learning educational program group or the waitlist control group at a 1:1 ratio." (Section 3.1) 3) "We conducted a randomized educational evaluation study with 300 university students randomly assigned to either a blended learning program (n=150) or waitlist control group (n=150)." (Abstract) Detailed Analysis: The quotes make clear that the unit of randomisation was the individual student: "participants were randomly assigned," with block randomisation applied to individual enrollees, not to intact classes or schools. There is no mention anywhere of classes, cohorts, or institutions being the randomised unit. The intervention is a group-based blended curriculum (online modules plus weekly face-to-face group sessions delivered by facilitators to enrolled volunteers drawn from across the university), not a one-to-one personal tutoring arrangement, so the tutoring exception to this criterion does not apply. Because participants recruited from the same institution (and potentially the same courses) were individually randomised to program or waitlist, contamination between treatment and control participants (e.g. through shared classes, dorms, or social networks) cannot be ruled out. Criterion C is not met because randomisation was conducted at the individual student level, with no class- or school-level allocation and no qualifying personal-tutoring exception.
  • E

    Exam-based Assessment

    • The study relies on self-report psychometric scales (MHLS, DSS, GHSQ, MAKS) rather than a standardised, widely recognised academic exam.
    • "The primary outcome measure was the Mental Health Literacy Scale (MHLS), a 35-item instrument assessing knowledge of mental disorders, recognition ability, help-seeking information, and attitudes."
    • Relevant Quotes: 1) "The primary outcome measure was the Mental Health Literacy Scale (MHLS), a 35-item instrument assessing knowledge of mental disorders, recognition ability, help-seeking information, and attitudes facilitating recognition or appropriate help-seeking. We employed a culturally adapted Chinese version that demonstrated satisfactory reliability and validity in previous studies with university populations." (Section 3.3) 2) "Secondary measures captured complementary dimensions that the MHLS addresses less comprehensively" -- Table 3 lists the Depression Stigma Scale (DSS), General Help-Seeking Questionnaire (GHSQ), and Mental Health Knowledge Schedule (MAKS), all Likert-type self-report psychometric scales. (Section 3.3, Table 3) Detailed Analysis: All outcome instruments used in this study (MHLS, DSS, GHSQ, MAKS) are self-report psychometric scales designed to measure psychological constructs -- literacy about mental disorders, personal/perceived stigma, help-seeking intentions, and mental-health knowledge -- via Likert-type items. None of these is a standardised, widely recognised academic exam of the kind envisioned by the ERCT criterion (e.g. a national curriculum test or state-wide achievement exam). While the instruments themselves have published reliability coefficients, psychometric reliability of a self-report attitude/knowledge scale is not the same as use of a standardised exam-based academic assessment. Criterion E is not met because the assessments used are self-report psychological/psychometric scales, not standardised academic exams.
  • T

    Term Duration

    • The interval from intervention start (week 4) to the three-month follow-up (week 24) spans roughly four to five months, meeting the one-term minimum.
    • "Follow-up Assessment (Week 24) August 2024 ... Three-month follow-up measurement of primary and secondary outcomes."
    • Relevant Quotes: 1) "Educational program Delivery (Weeks 4-11) April-May 2024 ... Eight-week blended learning program for educational program group; waitlist monitoring for control group." (Table 1) 2) "Follow-up Assessment (Week 24) August 2024 ... Three-month follow-up measurement of primary and secondary outcomes." (Table 1) 3) "Assessments occurred at baseline, immediate post-educational program, and three-month follow-up." (Abstract, Section 3.3) Detailed Analysis: The intervention began around week 4 of the study timeline (early-to-mid April 2024), and the final outcome measurement (three-month follow-up) occurred at week 24 (August 2024). The interval from intervention start to the final measurement point therefore spans roughly four to five months, which exceeds the approximately one-term (3-4 month) minimum this criterion requires, even though the active instructional phase itself lasted only eight weeks. Criterion T is met because outcomes were tracked to a point roughly four to five months after the intervention began, satisfying the one-term minimum interval.
  • D

    Documented Control Group

    • Detailed demographic and baseline outcome data for the control group are reported in Table 4 and confirmed to be equivalent to the intervention group.
    • "Table 4. Comparison of Baseline Characteristics Between Two Groups" showing no significant differences between groups (all p > 0.50).
    • Relevant Quotes: 1) "Table 4. Comparison of Baseline Characteristics Between Two Groups" reporting age, gender, academic year, prior mental health education, personal/family mental health history, and baseline MHLS/DSS/GHSQ/MAKS scores for both the program group (n=150) and control group (n=150), with test statistics and effect sizes for each. (Table 4) 2) "The two groups demonstrated excellent baseline equivalence across all examined variables. No statistically significant differences emerged for any demographic characteristic or outcome measure, with all p-values exceeding 0.50 and effect sizes remaining negligible (all d < 0.10, phi < 0.05)." (Section 4.1) 3) "The control group received no educational program during the active phase but were offered access to all program materials following completion of the three-month follow-up assessment -- an arrangement designed to maintain ethical standards while preserving internal validity." (Section 3.1) Detailed Analysis: The paper provides a detailed, table-based comparison of the control group's demographic and baseline characteristics (age, gender, academic year, prior mental-health education exposure, personal/family mental-health history) alongside baseline scores on every outcome measure, confirming close equivalence with the intervention group. The conditions experienced by the control group (waitlist, no program until after the study concludes) are also explicitly described. Criterion D is met because the control group's composition, baseline characteristics, and conditions are documented in detail via Table 4 and the accompanying narrative.

• Level 2 Criteria

  • S

    School-level RCT

    • The trial was conducted at a single institution with student-level (not school-level) randomisation.
    • "Our sample consisted of volunteer university students at a single institution, which limits generalizability..."
    • Relevant Quotes: 1) "We conducted a randomized educational evaluation study with 300 university students randomly assigned to either a blended learning program (n=150) or waitlist control group (n=150)." (Abstract) 2) "First, our sample consisted of volunteer university students at a single institution, which limits generalizability..." (Discussion, Limitations) Detailed Analysis: The study was conducted entirely within a single institution (The Catholic University of Korea), with individual student volunteers randomised to condition. There is no school-level or multi-site randomisation; the authors themselves list the single-institution, individually randomised design as a limitation to generalisability. Criterion S is not met because the trial involved individual-level randomisation within one institution, not randomisation of multiple schools/institutions.
  • I

    Independent Conduct

    • The same authors who designed the intervention also supervised data collection and analysis; only the randomisation sequence was generated independently.
    • "YL conceptualized the study, designed the research methodology, supervised data collection and analysis, and drafted the manuscript."
    • Relevant Quotes: 1) "Randomization followed a computer-generated sequence created by an independent statistician not otherwise involved in the study." (Section 3.1) 2) "YL conceptualized the study, designed the research methodology, supervised data collection and analysis, and drafted the manuscript. MZ contributed to educational program design, participated in data collection, assisted with statistical analysis, and critically revised the manuscript." (Authors' Contributions) Detailed Analysis: Only the generation of the randomisation sequence was outsourced to an independent statistician; this is a narrow, procedural form of independence and does not extend to the conduct of the study as a whole. The Authors' Contributions section shows that the same two authors who conceptualised and designed the blended-learning program ("educational program design") also supervised or directly performed data collection and statistical analysis. There is no external evaluation team, agency, or third-party research group independent of the program designers responsible for running the trial, collecting data, or analysing results, as is required for this criterion (contrast with the exemplar cases where an external agency conducted data collection/analysis independent of the intervention developers). Criterion I is not met because the same authors who designed the intervention also supervised and conducted data collection and analysis, with only the randomisation-sequence generation delegated to an independent statistician.
  • Y

    Year Duration

    • The intervention-to-final-measurement interval is only about four to five months, well below 75% of an academic year.
    • "The experimental timeline spanned six months from recruitment through final follow-up."
    • Relevant Quotes: 1) "Recruitment & Baseline ... March 2024 (Weeks 1-2)" through "Follow-up Assessment ... August 2024 (Week 24)." (Table 1) 2) "The experimental timeline spanned six months from recruitment through final follow-up." (Section 3.1) 3) "Fourth, the three-month follow-up, though longer than in many comparable studies, is insufficient to determine whether literacy gains translate into lasting help-seeking behavior or mental health improvements." (Discussion, Limitations) Detailed Analysis: The entire study, from recruitment to final follow-up, spans about six months, and the interval specifically from intervention start (week 4) to final outcome measurement (week 24) is roughly four to five months. This falls well short of 75% of an academic year (roughly 7-9 months), and the authors themselves acknowledge the follow-up period is too short to draw longer-term conclusions. Criterion Y is not met because the total tracking interval (about six months from recruitment, roughly four to five months from intervention start) is far short of 75% of a full academic year.
  • B

    Balanced Control Group

    • The intervention group received substantial extra instructional time and facilitator support unmatched by the waitlist control, and the authors themselves acknowledge this imbalance rather than framing the extra resources as the explicit treatment variable.
    • "The use of a waitlist control group... means we cannot disentangle the effects of the blended format from those of program content or total contact time; conclusions about format superiority are therefore premature."
    • Relevant Quotes: 1) "The educational program group received an eight-week blended learning program combining self-paced online modules with weekly face-to-face group sessions. Online components, hosted on a secure learning management system, included video lectures, interactive case studies, and self-reflection exercises totaling approximately 90 minutes per week. Weekly 90-minute in-person sessions facilitated by trained mental health educators incorporated group discussions, experiential activities, and question-and-answer periods." (Section 3.1) 2) "The control group received no educational program during the active phase but were offered access to all program materials following completion of the three-month follow-up assessment." (Section 3.1) 3) "The results in Table 2 indicate a carefully balanced allocation averaging approximately 65 minutes of online work and 90 minutes of face-to-face engagement weekly, totaling roughly 155 minutes per week across both modalities." (Section 3.2) 4) "Second, the use of a waitlist control group, rather than an active comparison receiving purely digital or purely face-to-face instruction, means we cannot disentangle the effects of the blended format from those of program content or total contact time; conclusions about format superiority are therefore premature." (Discussion, Limitations) Detailed Analysis: The intervention group received substantial additional resources not offered to controls during the active phase: roughly 155 minutes per week of combined online modules and facilitated face-to-face group sessions, delivered by trained facilitators over eight weeks, plus platform access, discussion forums, and reminders. The control condition was a pure waitlist, receiving no matched activity, materials, or facilitator contact during this period ("business as usual" with literally no alternative program). Applying the decision procedure: extra resources are clearly present and are not negligible; the study is not explicitly framed as testing "the impact of providing additional time/resources" as its treatment variable -- rather, it aims to test the pedagogical effectiveness of a specific blended curriculum design (content plus delivery format) versus no intervention at all. The control group did not receive matched time/materials, and the design is not within-subjects. The authors' own limitations section confirms this reading: they explicitly state the waitlist design prevents them from disentangling the effect of the blended format from the effect of simply receiving more instructional contact time. This is precisely the imbalance the criterion is designed to flag -- extra time and facilitator resources given only to one arm, with no matched "business as usual" educational input for the control group beyond ordinary university life. Criterion B is not met because the intervention group received substantial additional instructional time, materials, and facilitator support that the waitlist control group did not receive during the active phase, and the authors do not frame the provision of these extra resources itself (as opposed to program content) as the explicit treatment variable being tested.

• Level 3 Criteria

  • R

    Reproduced

    • The paper is a newly accepted article with no independent replication reported in the text or found via internet search, and its recency precludes any replication having yet occurred.
    • "Accepted: 29 May 2026 ... Article in Press."
    • Relevant Quotes: 1) "Received: 15 December 2025 ... Accepted: 29 May 2026 ... Article in Press." (Front matter) 2) "We are providing an unedited version of this manuscript to give early access to its findings." (Front matter) Detailed Analysis: This is a newly accepted "Article in Press," received in December 2025 and accepted in May 2026, describing a single-institution study with no prior publications of the same intervention. No independent replication is mentioned anywhere in the paper. An internet search (web search and DOI/publisher lookups) for independent replications of this specific study -- by title, by authors Yang Luo and Min Zhang, and by close paraphrases of the intervention -- returned no evidence of any replication by a different research team. The search surfaced other, clearly distinct mental-health-literacy RCTs (e.g. one evaluating a blended program for adolescents' coping with school bullying, and other general Chinese-university mental-health-literacy trials), none of which reproduce this specific single-institution, university-student blended curriculum and study design; they are different studies with different populations, and none cites or attempts to replicate this paper. Given the manuscript's very recent acceptance date, there has not been sufficient time for an independent research team to replicate this specific study and publish results in a peer-reviewed journal. Criterion R is not met because there is no evidence, in the paper, via internet search, or plausible given its recency, of independent replication by a different research team.
  • A

    All-subject Exams

    • Criterion E is not met, and the study assesses only mental-health-literacy-related constructs rather than all core academic subjects.
    • "Primary outcomes included educational achievements in mental health literacy measured by the Mental Health Literacy Scale (MHLS)."
    • Relevant Quotes: 1) "Primary outcomes included educational achievements in mental health literacy measured by the Mental Health Literacy Scale (MHLS), with secondary educational measures assessing stigma awareness, help-seeking knowledge, and factual understanding." (Abstract) Detailed Analysis: Per the criterion-specific instruction, if criterion E (Exam-based Assessment) is not met, criterion A cannot be met either. As established under E, the assessments used (MHLS, DSS, GHSQ, MAKS) are self-report psychometric scales, not standardised academic exams, so the exam-based assessment prerequisite is not satisfied. Independently, the study measures only mental-health-literacy-related constructs and does not assess any other core school subjects (mathematics, science, language arts, etc.). Criterion A is not met, both because criterion E is not met and because the study measures only one narrow construct rather than all core subjects.
  • G

    Graduation Tracking

    • Criterion Y is not met, tracking stopped at the three-month follow-up with no tracking through graduation, and no follow-up publication by the same authors tracking this cohort was found.
    • "Future studies should prioritize longer follow-up periods with behavioral outcomes..."
    • Relevant Quotes: 1) "Fourth, the three-month follow-up, though longer than in many comparable studies, is insufficient to determine whether literacy gains translate into lasting help-seeking behavior or mental health improvements." (Discussion, Limitations) 2) "Future studies should prioritize longer follow-up periods with behavioral outcomes..." (Conclusion) Detailed Analysis: Per the criterion-specific instruction, if criterion Y (Year Duration) is not met, criterion G cannot be met either; Y was established as not met above. Independently, tracking stopped at the three-month follow-up, with the authors explicitly calling for longer follow-up in future work and giving no indication of any plan or execution of tracking through to participants' graduation. An internet search for subsequent publications by Yang Luo and/or Min Zhang tracking this same cohort toward graduation returned no results; no follow-up paper on this cohort could be found. Criterion G is not met, both because criterion Y is not met and because follow-up stopped at three months with no graduation tracking, and no subsequent follow-up publication by the same authors was found via internet search.
  • P

    Pre-Registered

    • The authors explicitly state the study was not registered, and no pre-registration reference is provided anywhere in the paper or found via independent registry search.
    • "Clinical Trial Registration: Not applicable. This study constitutes educational research rather than a clinical trial."
    • Relevant Quotes: 1) "Clinical Trial Registration: Not applicable. This study constitutes educational research rather than a clinical trial." (Declarations) 2) "For these reasons, trial registration in clinical trial registries is not applicable to this educational research study." (Declarations) Detailed Analysis: The paper explicitly states that trial registration was not undertaken, on the grounds that the authors consider this "educational research" rather than a clinical trial. No pre-registration platform, registry ID, or registration date is mentioned anywhere in the manuscript, and there is no protocol paper or statistical analysis plan cited as having been published before data collection began. A search of common trial/protocol registries (e.g. ClinicalTrials.gov, ISRCTN, OSF Registries) and the open web for a pre-registration record tied to this study, its authors, or its ethics protocol number (MHL-2024-036) found no matching registration. Criterion P is not met because the study was explicitly not pre-registered, no pre-registration reference of any kind is provided in the paper, and none was found via independent registry search.