Widening gender gaps in sexual and reproductive health knowledge, attitudes, and behaviors despite equally effective sexuality education: a cluster randomized controlled trial among Chinese vocational school adolescents

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomization was carried out at the school level across 29 schools, which satisfies the class-level requirement.
    • "Randomization was conducted at the school level to reduce contamination between study arms."
    • Relevant Quotes: 1) "Schools were randomized at the school level using a computer-generated random number generator following allocation concealment." (Abstract, p. 1) 2) "Randomization was conducted at the school level to reduce contamination between study arms. To minimize selection bias, random allocation was performed using a computer-generated random number generator (Google Random Number Generator) by the study team, and allocation was concealed from schools, teachers, and research personnel involved in recruitment until after enrollment was complete." (p. 5) 3) "In total, 29 vocational high schools were enrolled in the study." (p. 4) 4) "Schools were then randomized within strata to either the intervention group (Guangdong: n = 5; Yunnan: n = 10) or the control group (Guangdong: n = 5; Yunnan: n = 9)." (p. 5) Detailed Analysis: Criterion C requires randomisation at the class level or stronger. This study is an explicitly described cluster randomized controlled trial in which the unit of randomisation was the whole school. School-level randomisation is stronger than class-level randomisation and, per the ERCT standard, automatically satisfies the weaker class-level requirement. The allocation used a computer-generated random number generator with allocation concealment, and 29 schools were assigned to intervention or control arms. Criterion C is met because randomisation was performed at the school level, which exceeds the class-level requirement.
  • E

    Exam-based Assessment

    • Outcomes were measured with custom research-team items rather than a recognised standardised exam.
    • "The knowledge and attitude items were developed for this trial to align with the intervention content and local context and are not drawn entirely from validated scales."
    • Relevant Quotes: 1) "These items, while not from validated scales, were developed by the research team based on similar studies in China and internationally, with careful attention to cultural sensitivity and contextual appropriateness." (p. 6) 2) "Knowledge items had three response options: 'True,' 'False,' and 'I don't know.' Each correct answer earned one point, with scores converted to percentages (range: 0-100%)." (p. 6) 3) "Attitude items used a seven-point Likert scale... Confirmatory Factor Analysis (CFA) identified four attitudinal dimensions: power, homosexuality, premarital sex, and help-seeking behavior." (p. 6) 4) "The knowledge and attitude items were developed for this trial to align with the intervention content and local context and are not drawn entirely from validated scales." (p. 13) Detailed Analysis: Criterion E requires a standardised, widely recognised exam-based assessment that was not designed specifically for the study. Here the outcomes are SRH knowledge and attitude items that the authors explicitly developed for this trial and aligned to the intervention content; they state the items are not from validated scales. The instruments are custom, study-specific questionnaires, not standardised exams, and they measure SRH knowledge, attitudes, and self-reported behaviours rather than academic achievement. Criterion E is not met because the assessments were custom-developed items rather than a recognised standardised exam.
  • T

    Term Duration

    • The interval from intervention start to the analysed one-year follow-up far exceeds one academic term.
    • "The second follow-up survey was administered in June 2020, one year after the intervention."
    • Relevant Quotes: 1) "The baseline survey was administered in April 2019. Schools in the intervention group then delivered the module over 4-6 weeks..." (p. 5) 2) "The second follow-up survey was administered in June 2020, one year after the intervention." (p. 5) 3) "To capture sustained changes and emerging gendered patterns, the present analysis examines outcomes measured at baseline and one year after the intervention." (p. 3) Detailed Analysis: Criterion T requires outcomes measured at least one full academic term after the intervention begins. The intervention began around April/May 2019 and the outcomes analysed in this paper were measured at the one-year follow-up in June 2020. This interval of roughly one year is far longer than a single academic term (about 3-4 months). Criterion T is met because outcomes were measured approximately one year after the intervention, exceeding one full academic term.
  • D

    Documented Control Group

    • The control group's size and baseline demographic characteristics are documented in Table 1.
    • "Table 1 presents the baseline characteristics for the study sample (N = 2996), stratified by intervention condition due to the experimental nature of the data."
    • Relevant Quotes: 1) "The analytic sample included 2996 10th-grade students (n = 1332 control; n = 1664 intervention)." (Abstract, p. 1) 2) "Table 1 presents the baseline characteristics for the study sample (N = 2996), stratified by intervention condition due to the experimental nature of the data." (p. 7) 3) Table 1 reports for the Control group (N = 1332): Girl 604 (45.35%), Province, Previous sex education experiences 658 (49.40%), Mean age 16.01, Hukou (Rural/Urban), Ethnic Minority, boarding status, class and school characteristics, with p values comparing arms. (p. 8) Detailed Analysis: Criterion D requires that the control group be well-documented, including size, demographics, and baseline characteristics. The paper reports the control group size (n = 1332) and provides a detailed baseline characteristics table (Table 1) with demographic, family, class, and school-level variables for the control arm alongside the intervention arm. The control condition is described as business-as-usual (schools that did not receive the CSE module). Criterion D is met because the control group's size and baseline characteristics are clearly documented.

• Level 2 Criteria

  • S

    School-level RCT

    • Entire schools were the unit of randomisation across 29 vocational high schools.
    • "Schools were randomized at the school level using a computer-generated random number generator following allocation concealment."
    • Relevant Quotes: 1) "Schools were randomized at the school level using a computer-generated random number generator following allocation concealment." (Abstract, p. 1) 2) "The trial started with selection of schools within each province, then randomly selected classes within schools, then enrolled students within the selected classes, before the team randomized the schools into intervention and control groups." (p. 3) 3) "Schools were then randomized within strata to either the intervention group (Guangdong: n = 5; Yunnan: n = 10) or the control group (Guangdong: n = 5; Yunnan: n = 9)." (p. 5) Detailed Analysis: Criterion S requires randomisation at the school level, where a school is the institution implementing the intervention. The paper is explicitly a cluster RCT in which the unit of randomisation was the whole vocational high school; classes and students were selected first, but the randomised allocation to arms was done at the school level within strata, across 29 schools. Criterion S is met because randomisation was conducted at the school level.
  • I

    Independent Conduct

    • The same research team designed, implemented, collected, and analysed the trial, with no independent external evaluator.
    • "MC participated in the original randomized controlled trial as a key research staff member, leading the study design and implementation, data collection and analysis, and report writing."
    • Relevant Quotes: 1) "MC participated in the original randomized controlled trial as a key research staff member, leading the study design and implementation, data collection and analysis, and report writing." (Author's contributions, p. 14) 2) "random allocation was performed using a computer-generated random number generator (Google Random Number Generator) by the study team..." (p. 5) 3) "Professor Kun Tang, the Principal Investigator of the cluster randomized controlled trial, from Tsinghua University Vanke School of Public Health, for guidance during the design and execution of the original randomized controlled trial, in which the author participated as the main research staff responsible for the project." (Acknowledgements, p. 14) Detailed Analysis: Criterion I requires that the evaluation be conducted independently from the intervention designers, with data collection and analysis performed by parties without a stake in the intervention's success. Here the study team designed and executed the trial, delivered the intervention through participating schools, collected the data, and analysed it. The author was the main research staff responsible for the project and led design, implementation, data collection, and analysis. There is no statement of an independent, third-party evaluator or blinded external data collection team; indeed, after enrollment "neither participating schools and students nor the study team were blinded" (p. 5). Criterion I is not met because the trial was designed, implemented, and analysed by the same research team, with no independent external evaluation.
  • Y

    Year Duration

    • Outcomes were tracked to a one-year follow-up, covering a full academic year after the intervention began.
    • "The second follow-up survey was administered in June 2020, one year after the intervention."
    • Relevant Quotes: 1) "The baseline survey was administered in April 2019." (p. 5) 2) "The second follow-up survey was administered in June 2020, one year after the intervention." (p. 5) 3) "This is a secondary analysis utilizing the April 2019 and June 2020 survey data from a cluster randomized controlled trial..." (Abstract, p. 1) Detailed Analysis: Criterion Y requires outcomes measured at least 75% of a full academic year after the intervention begins. Criterion T is met. The intervention began around April/May 2019 and the analysed follow-up was in June 2020, roughly one year (and about 14 months from baseline) later. This tracking interval clearly exceeds 75% of an academic year. Criterion Y is met because outcomes were measured about one year after the intervention began.
  • B

    Balanced Control Group

    • The additional educational time and content is the CSE program itself, the explicit treatment variable tested against business-as-usual controls.
    • "You and Me [45] is a web-based CSE program that includes eight 45-minute classes covering topics such as general introduction, gender, sexual behaviors, contraceptive methods..."
    • Relevant Quotes: 1) "You and Me [45] is a web-based CSE program that includes eight 45-minute classes covering topics such as general introduction, gender, sexual behaviors, contraceptive methods, sexually transmitted diseases, sexual violence, love and relationships, pregnancy, and puberty." (p. 5) 2) "All teachers received 1.5 days of training on the principles of CSE, on how to use the module... Schools in the intervention arm then implemented the program over a period of 6-8 weeks." (p. 5) 3) "Exclusion criterion: Schools had already delivered CSE... Schools in the intervention group then delivered the module over 4-6 weeks... The trial, originally aimed at evaluating the effectiveness of a web-based CSE program." (pp. 3-5) Detailed Analysis: Applying the criterion B decision tree: the intervention adds educational time and resources (eight 45-minute CSE classes plus teacher training) that the control arm did not receive, so extra resources are present and the difference is not negligible. However, the study explicitly tests the effectiveness of providing this CSE program - the additional time and content IS the treatment variable under investigation. The control schools represent the required business-as-usual baseline against which the effect of adding the CSE program is measured. Under the standard, when the additional resource is integral to and is the explicit treatment variable, a business-as-usual control does not violate criterion B. Criterion B is met because the extra educational time and content is the CSE program itself, which is the explicit treatment variable tested against a business-as-usual control.

• Level 3 Criteria

  • R

    Reproduced

    • No independent replication of this specific study has been reported.
    • "This study provides the first evidence of gender divergence patterns in sexual and reproductive health (SRH) dimensions among Chinese adolescents despite uniform intervention effectiveness."
    • Relevant Quotes: 1) "This study provides the first evidence of gender divergence patterns in sexual and reproductive health (SRH) dimensions among Chinese adolescents despite uniform intervention effectiveness." (p. 13) 2) "The main study outcomes have been reported in two other published journal articles, which showed improvements in SRH knowledge and attitudes both immediately and at one year, but no significant effects on SRH behaviors [43, 44]." (p. 3) Detailed Analysis: Criterion R requires that this specific study be independently replicated by a different research team in a different context, published in a peer-reviewed journal. References [43] and [44] are Chen et al. (Health Education Research, 2023) and Hu et al. (Journal of Adolescent Health, 2023); both are the parent trial's own main-outcome papers based on the same "You and Me" trial and data, not independent replications of this particular gender-focused secondary analysis, which the author describes as providing the "first evidence" of these patterns. An internet search of available sources identified no independent replication of this study by other authors in a different context. Criterion R is not met because no independent replication of this specific study exists.
  • A

    All-subject Exams

    • Criterion E is not met, and only SRH outcomes (not all core school subjects) were assessed.
    • "The study examined three types of dependent variables: SRH knowledge, attitudes, and behavioral/health outcomes."
    • Relevant Quotes: 1) "The study examined three types of dependent variables: SRH knowledge, attitudes, and behavioral/health outcomes. Knowledge and attitudes were assessed using 20 and 16 items respectively." (p. 6) 2) "The knowledge and attitude items were developed for this trial... and are not drawn entirely from validated scales." (p. 13) Detailed Analysis: Criterion A requires standardised exam-based assessment across all main subjects, and it is contingent on criterion E. Criterion E is not met (custom, non- standardised items), so criterion A cannot be met. Furthermore, the study measured only SRH knowledge, attitudes, and behaviours, not the full range of core academic subjects taught in vocational high school. Criterion A is not met because criterion E is not met and only a single, narrow SRH domain was assessed.
  • G

    Graduation Tracking

    • Tracking stopped at a one-year follow-up rather than continuing to graduation.
    • "they typically remain enrolled for at least one additional academic year before beginning off-site internships in their third year, after which they become more difficult to track."
    • Relevant Quotes: 1) "As students are in their first year of vocational high school, they typically remain enrolled for at least one additional academic year before beginning off-site internships in their third year, after which they become more difficult to track." (p. 4) 2) "The second follow-up survey was administered in June 2020, one year after the intervention." (p. 5) Detailed Analysis: Criterion G requires follow-up tracking of participants through to graduation. The study followed 10th-grade students only to a one-year follow-up (into 11th grade) and explicitly notes students become difficult to track once they begin third-year internships. An internet search for subsequent publications by the author (Minne Chen) and the parent-trial team identified only the two parent-trial outcome papers (Chen et al., Health Education Research, 2023; Hu et al., Journal of Adolescent Health, 2023), both of which track outcomes only to a one-year follow-up. No follow-up publication tracking this cohort to graduation was found. Criterion G is not met because participants were tracked only to a one-year follow-up, not to graduation.
  • P

    Pre-Registered

    • The parent trial was registered, but this gender-focused secondary analysis was explicitly not pre-registered.
    • "While the parent trial was prospectively registered, the present gender-focused analyses were not separately pre-registered."
    • Relevant Quotes: 1) "The original randomized controlled trial, where data used in this article came from, was registered with the Chinese Clinical Trial Registry (chictr.org.cn). The registry number is ChiCTR1900021582. The trial was registered on Feb 28, 2019." (Abstract, p. 2) 2) "The baseline survey was administered in April 2019." (p. 5) 3) "While the parent trial was prospectively registered, the present gender-focused analyses were not separately pre-registered." (p. 3) Detailed Analysis: Criterion P requires that the full study protocol, including hypotheses and planned analyses, be pre-registered before data collection begins. The parent trial has a prospective registration (ChiCTR1900021582, Feb 28 2019, before the April 2019 baseline), which an internet check of the Chinese Clinical Trial Registry confirms as a real registration for this "You and Me" web-based CSE trial. However, this paper is a secondary analysis whose specific hypotheses and three-way gender-time-intervention interaction analyses were explicitly not pre-registered. Because the analyses reported here were not part of a pre-registered protocol, the selective-reporting protection that criterion P is designed to ensure does not apply to this study. Criterion P is not met because the gender-focused analyses reported in this paper were not pre-registered, even though the parent trial was.