Using a volitional help sheet to increase university students' attendance at on-campus lectures: A randomized controlled trial

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Participants were randomized at the individual student level (by survey software), not at the class (or higher) level, and the intervention is not one-to-one tutoring.
    • "After completing these items, they were allocated at random, by the survey software, to one of three conditions: a VHS condition; a VHS plus encoding facilitation condition; or a control condition."
    • Relevant Quotes: 1) "A randomized controlled design was used." (p. 5) 2) "After completing these items, they were allocated at random, by the survey software, to one of three conditions: a VHS condition; a VHS plus encoding facilitation condition; or a control condition." (pp. 5–6) 3) "Randomised to conditions N = 252" (Figure 1, p. 6) Detailed Analysis: Criterion C requires random assignment at the class level (or stronger, such as school-level) to reduce contamination across treatment and control participants who share the same classroom environment. The paper explicitly describes individual-level assignment using survey software, and the CONSORT flow chart presents randomization to conditions for individual students (N = 252), with no indication that intact classes (or schools/sites) were randomized. The tutoring exception does not apply: the intervention is a self-completed planning tool (a volitional help sheet, with an optional rehearsal task), not one-to-one tutoring or personal teaching delivered as the educational format. Final summary sentence: Criterion C is not met because the unit of randomization is the individual student (not class/school), and the intervention is not tutoring-style personal teaching.
  • E

    Exam-based Assessment

    • The outcome is lecture attendance (QR-based monitoring), not performance on a standardized exam-based assessment.
    • "Lecture attendance was measured using the University's attendance monitoring system."
    • Relevant Quotes: 1) "Lecture attendance was measured using the University's attendance monitoring system." (p. 8) 2) "The final measure of lecture attendance was the proportion of lectures that the participants attended over the course of the relevant 11-week teaching semester." (p. 9) Detailed Analysis: Criterion E requires outcomes to be measured using standardized exam-based assessments (e.g., national/state-wide standardized achievement tests) rather than non-exam outcomes or custom assessments. The paper's primary outcome is lecture attendance, measured via the university's attendance monitoring system, and summarized as the proportion of lectures attended over an 11-week teaching semester. This is not an exam-based academic achievement measure. The additional measures in the study (goal intention and memory ability) are also not standardized academic exams and do not assess curricular attainment. Final summary sentence: Criterion E is not met because the study measures attendance rather than standardized exam-based academic outcomes.
  • T

    Term Duration

    • Outcomes were tracked across an entire 11-week teaching semester, which is a complete academic term in this context.
    • "Lecture attendance was recorded at each lecture over the 11week teaching semester..."
    • Relevant Quotes: 1) "The programme comprised two 11-week teaching semesters per academic year." (p. 5) 2) "Lecture attendance was recorded at each lecture over the 11week teaching semester..." (p. 7) 3) "The final measure of lecture attendance was the proportion of lectures that the participants attended over the course of the relevant 11-week teaching semester." (p. 9) Detailed Analysis: Criterion T requires that outcomes be measured at least one academic term after the intervention begins. The paper defines the local term structure explicitly ("two 11-week teaching semesters per academic year") and then tracks lecture attendance for the full 11-week teaching semester after the intervention allocation to condition. Because the outcome window spans the complete teaching semester (the full term as defined in this setting), the study meets the minimum term-duration follow-up requirement. Final summary sentence: Criterion T is met because attendance was tracked over a full 11-week teaching semester (a complete term in this context).
  • D

    Documented Control Group

    • The control condition is clearly described (including what participants did), and baseline comparability and group sizes are reported.
    • "The participants allocated to the control condition (n=87) were also presented with the same critical situations and goal-directed responses contained with Elliott et al.'s (2024) VHS."
    • Relevant Quotes: 1) "After completing these items, they were allocated at random, by the survey software, to one of three conditions: a VHS condition; a VHS plus encoding facilitation condition; or a control condition." (pp. 5–6) 2) "Control condition n = 87" (Figure 1, p. 6) 3) "The participants allocated to the control condition (n=87) were also presented with the same critical situations and goal-directed responses contained with Elliott et al.'s (2024) VHS." (p. 7) 4) "There were no significant differences between the conditions on any measure (see Table 2). Randomization to the conditions was therefore successful." (p. 9) Detailed Analysis: Criterion D requires that the control group be well documented, including: (a) clear description of what the control group received or did, (b) sample sizes, and (c) sufficient baseline information to support comparability. The paper provides a clear procedural description of the control condition (including that they received the same lists of critical situations and responses but were not asked to link them), a CONSORT flow chart that reports the control sample size (n = 87), and a randomization check stating no pre-intervention differences across conditions on measured variables. This satisfies ERCT criterion D for control-group documentation. Final summary sentence: Criterion D is met because the control condition is described in detail and baseline comparability and group sizes are reported.

• Level 2 Criteria

  • S

    School-level RCT

    • Randomization occurred among individual students within one university programme, not among schools (or similar institutional units).
    • "After completing these items, they were allocated at random, by the survey software, to one of three conditions..."
    • Relevant Quotes: 1) "The sample comprised N=252 university students who were enrolled in a 4-year undergraduate Psychology degree programme at a large University in Scotland..." (p. 5) 2) "After completing these items, they were allocated at random, by the survey software, to one of three conditions..." (pp. 5–6) Detailed Analysis: Criterion S requires school-level (or site/institutional-unit) randomization, where whole schools/sites are assigned to intervention/control. This study recruits individual students from one university degree programme and assigns them individually to conditions using survey software. There is no indication that multiple schools, campuses, or sites were randomized. Final summary sentence: Criterion S is not met because the unit of randomization is the individual student within one programme, not the school/site.
  • I

    Independent Conduct

    • The paper provides no clear evidence that an independent third party conducted implementation, measurement, or analysis separate from the intervention author team.
    • "Mark A. Elliott: Conceptualization; investigation; funding acquisition; writing – original draft; writing – review and editing; visualization; methodology; validation; project administration; formal analysis; software; data curation; supervision; resources."
    • Relevant Quotes: 1) "The participants allocated to the VHS condition (n=83) were asked to complete Elliott et al.'s (2024) VHS." (p. 6) 2) "AUTHOR CONTRIBUTIONS Mark A. Elliott: Conceptualization; investigation; funding acquisition; writing – original draft; writing – review and editing; visualization; methodology; validation; project administration; formal analysis; software; data curation; supervision; resources." (pp. 13–14) Detailed Analysis: Criterion I requires that study conduct (implementation, data collection, and/or analysis) be independent from the intervention designers/providers to reduce bias (e.g., via an external evaluation team). The intervention is explicitly attributed to the author team (Elliott et al.'s (2024) VHS). The author contribution statement lists the authors performing the central study functions (including investigation, project administration, formal analysis, and data curation). The paper does not include a statement that an external or independent evaluation team conducted implementation, outcome measurement, or analysis. Final summary sentence: Criterion I is not met because independent conduct by a third party is not documented in the paper.
  • Y

    Year Duration

    • Outcomes were tracked only over an 11-week teaching semester, which is far less than 75% of an academic year.
    • "Lecture attendance was recorded at each lecture over the 11week teaching semester..."
    • Relevant Quotes: 1) "The programme comprised two 11-week teaching semesters per academic year." (p. 5) 2) "Lecture attendance was recorded at each lecture over the 11week teaching semester..." (p. 7) 3) "The final measure of lecture attendance was the proportion of lectures that the participants attended over the course of the relevant 11-week teaching semester." (p. 9) Detailed Analysis: Criterion Y requires outcomes to be measured at least 75% of one academic year after the intervention begins. The study's outcome window is a single 11-week teaching semester. Even allowing for variation in academic calendars, an 11-week follow-up is well below the ERCT threshold for year-duration tracking. Final summary sentence: Criterion Y is not met because outcomes were measured over one 11-week semester rather than most of an academic year.
  • B

    Balanced Control Group

    • The control condition is an active control receiving comparable materials and a closely matched task, and the extra time in the encoding-facilitation arm is integral to that treatment.
    • "The participants allocated to the control condition (n=87) were also presented with the same critical situations and goal-directed responses contained with Elliott et al.'s (2024) VHS."
    • Relevant Quotes: 1) "The participants allocated to the VHS condition (n=83) were asked to complete Elliott et al.'s (2024) VHS." (p. 6) 2) "The participants were asked to specify four implementation intentions by selecting four critical situations and linking each one with a goal-directed response..." (p. 6) 3) "The participants allocated to the VHS plus encoding facilitation condition (n=82) were also asked to complete Elliott et al.'s (2024) VHS. In addition, after specifying each implementation intention, they were asked to read it three times..." (p. 7) 4) "The participants allocated to the control condition (n=87) were also presented with the same critical situations and goal-directed responses contained with Elliott et al.'s (2024) VHS." (p. 7) 5) "Also, the control participants were not asked to link critical situations with goal-directed responses. Instead, they were asked to select four critical situations, and separately, four goal-directed responses." (p. 7) Detailed Analysis: Criterion B compares the nature, quantity, and quality of resources (time, materials, support) provided to intervention and control conditions, and asks whether the control condition provides a comparable substitute for the intervention's inputs, unless extra resources are explicitly the treatment variable. Here, the control group is an active control: they receive the same underlying materials (the same lists of critical situations and goal-directed responses) and complete a closely related task, differing mainly in the key mechanism (linking IF/THEN plans versus selecting items separately). There is no indication of major imbalances in staffing, budget, or instructional time beyond the intended cognitive manipulation. The VHS+encoding-facilitation condition includes additional time for rehearsal ("read it three times"). That extra time is the integral, explicitly manipulated component of that experimental arm rather than an accidental confound, and it does not represent a substantive educational resource injection (e.g., extra tutoring hours). Final summary sentence: Criterion B is met because the control group receives a closely matched active-control task with the same materials, and the additional rehearsal time is integral to the encoding-facilitation treatment manipulation.

• Level 3 Criteria

  • R

    Reproduced

    • No evidence was found of an independent replication of this lecture attendance VHS RCT by a different research team.
    • "First, Elliott et al. (2024) is the only previous study in the educational literature to have tested the efficacy of a VHS, so a replication of the results is necessary..."
    • Relevant Quotes: 1) "However, just one previous study has tested whether VHSs can modify educational behaviours." (p. 3) 2) "First, Elliott et al. (2024) is the only previous study in the educational literature to have tested the efficacy of a VHS, so a replication of the results is necessary to inform evidence-based practice..." (p. 3) 3) "The present, pre-registered, study therefore sought to test whether the efficacy of Elliott et al.'s (2024) VHS could be extended to on-campus lecture attendance." (p. 3) Detailed Analysis: Criterion R requires independent replication by a different research team (in a different context) published in a peer-reviewed outlet. The paper positions itself as a replication/extension of Elliott et al. (2024), but that earlier educational lecture-attendance VHS trial is not an independent replication by a separate author team. The current trial is also conducted by (overlapping) authors from the same institution. An internet search for peer-reviewed trials specifically replicating the Elliott et al. (2024) / Elliott et al. (2026) lecture-attendance VHS intervention in higher education did not identify any such independent replications by other author teams. Final summary sentence: Criterion R is not met because independent replication of this lecture-attendance VHS RCT by a different research team was not found.
  • A

    All-subject Exams

    • Because the study does not use standardized exam-based assessments (Criterion E not met), it cannot satisfy the all-subject standardized exam requirement.
    • "Lecture attendance was measured using the University's attendance monitoring system."
    • Relevant Quotes: 1) "Lecture attendance was measured using the University's attendance monitoring system." (p. 8) 2) "The final measure of lecture attendance was the proportion of lectures that the participants attended over the course of the relevant 11-week teaching semester." (p. 9) Detailed Analysis: Criterion A requires impacts to be assessed using standardized exam-based assessments across all main subjects, and ERCT specifies that if Criterion E is not met then Criterion A is not met. This study measures attendance rather than standardized exam performance, and it does not assess multiple subjects via standardized exams. Final summary sentence: Criterion A is not met because the study does not use standardized exam outcomes (so it cannot measure all subjects via standardized exams).
  • G

    Graduation Tracking

    • The study does not track participants to graduation and, because the year-duration criterion is not met, this criterion is automatically not met.
    • "Lecture attendance was recorded at each lecture over the 11week teaching semester..."
    • Relevant Quotes: 1) "Lecture attendance was recorded at each lecture over the 11week teaching semester..." (p. 7) 2) "The final measure of lecture attendance was the proportion of lectures that the participants attended over the course of the relevant 11-week teaching semester." (p. 9) Detailed Analysis: Criterion G requires follow-up tracking through graduation. Additionally, the ERCT specification states that if Criterion Y is not met then Criterion G is not met. This paper measures attendance only during a single teaching semester and contains no evidence of tracking participants through degree completion or graduation. An internet search for follow-up publications by the same authors reporting graduation (degree completion) outcomes for this cohort did not identify any such graduation-tracking follow-up paper. Final summary sentence: Criterion G is not met because participants were followed for one semester only (and Y is not met), with no evidence of tracking to graduation found in the paper or follow-up publications.
  • P

    Pre-Registered

    • The paper links to an OSF pre-registration but does not provide a registration timestamp, and the OSF record date could not be verified against the study start.
    • "Consistent with the above review, and as pre-registered on the Open Science Framework (OSF: https://osf.io/t6s4y), the hypotheses were:"
    • Relevant Quotes: 1) "Consistent with the above review, and as pre-registered on the Open Science Framework (OSF: https://osf.io/t6s4y), the hypotheses were:" (p. 4) 2) "The method was in accordance with the pre-registration on the Open Science Framework." (p. 5) 3) "For students at levels 3 and 4 of the programme, advertisements were posted ... during the consolidation and development week of semester 2, in the 2023–2024 academic year." (p. 5) 4) "For students at levels 1 and 2 of the programme, the same sampling procedure was used ... during the welcome and development week of semester 1, in the 2024–2025 academic year." (p. 5) Detailed Analysis: Criterion P requires that the full study protocol be pre-registered before the study begins, and that the registration date can be verified to be prior to data collection/recruitment. The paper clearly claims pre-registration and provides an OSF link (osf.io/t6s4y). The paper also indicates recruitment/data collection activity spanning the 2023–2024 and 2024–2025 academic years, which implies that a valid pre-registration would need to be dated before those activities began. However, the paper itself does not provide a pre-registration date, and the OSF record's registration/creation timestamp could not be retrieved in a way that allows verification that it preceded the study start. Final summary sentence: Criterion P is not met because the paper cites OSF pre-registration but a verifiable pre-registration date (before study start) was not available from the paper and could not be confirmed from the registry record during this check.