Abstract
Time‑management skills are an essential component of college student success, especially in online classes. Through a randomized control trial of students in a for‑credit online course at a public 4‑year university, we test the efficacy of a scheduling intervention aimed at improving students’ time management. Results indicate the intervention had positive effects on initial achievement scores; students who were given the opportunity to schedule their lecture watching in advance scored about a third of a standard deviation better on the first quiz than students who were not given that opportunity. These effects are concentrated in students with the lowest self‑reported time‑management skills. However, these effects diminish over time such that we see a marginally significant negative effect of treatment on the last week’s quiz grade and no difference in overall course scores. We examine the effect of the intervention on plausible mechanisms to explain the observed achievement effects and find no evidence that the intervention affected cramming, procrastination, or the time at which students did work.
Full
Article
ERCT Criteria Breakdown
-
Level 1 Criteria
-
C
Class-level RCT
- Individual‑level randomization within one class violates the class‑level RCT requirement.
- "Students were randomly assigned into treatment (N=79) and control (N=78) groups on the first day of the course."
Relevant Quotes:
1) "Students were randomly assigned into treatment (N=79) and control (N=78) groups on the first day of the course."
2) "The suggestion was delivered to a randomly selected group of students in each of the first 2 weeks of the course."
Detailed Analysis:
Randomization occurred at the individual‑student level within a single online class section. Under ERCT criterion C, randomization must occur at the whole‑class (or stronger) level unless the intervention is one‑to‑one tutoring. Because students in the same class could interact and contamination is possible, the design does not satisfy the class‑level requirement.
Final sentence: Criterion C is not met because randomization was at the student, not class, level.
-
E
Exam-based Assessment
- Assessments were custom course quizzes and a final, not a standardized exam.
- "Except for the week 1 quiz … all of the other quizzes had a maximum score of 6."
Relevant Quotes:
1) "Except for the week 1 quiz, which had a maximum score of 15, all of the other quizzes had a maximum score of 6 … Students’ final grades were determined by … weekly quizzes … and the final exam score."
2) "We focus our analyses on the weekly quiz scores and final course grade…"
Detailed Analysis:
Outcome measures consist of instructor‑created weekly quizzes and a course‑specific final exam. No evidence is given that these are externally standardized assessments.
Final sentence: Criterion E is not met because only instructor tests, not standardized exams, were used.
-
T
Term Duration
- Outcomes were collected after only five weeks, not a full term.
- "…course lasting 5 weeks…"
Relevant Quotes:
1) "We conducted our study in an online undergraduate STEM course lasting 5 weeks in a selective, public 4‑year university."
Detailed Analysis:
The intervention and all outcome measurements occurred within a single 5‑week summer term—much shorter than a regular 3‑ to 4‑month academic term required by ERCT.
Final sentence: Criterion T is not met because tracking lasted only five weeks.
-
D
Documented Control Group
- Demographics and baseline performance for the control group are fully reported.
- "Table 1 presents summary statistics … treatment group, and the control group."
Relevant Quotes:
1) "Table 1 presents summary statistics on student characteristics for the analytic sample, the treatment group, and the control group."
2) "The control students received emails from the course instructor asking them to respond to an online survey…"
Detailed Analysis:
The paper supplies demographic, baseline, and outcome data for the control group, and clearly describes the ‘business‑as‑usual’ condition (placebo emails only). This satisfies documentation expectations.
Final sentence: Criterion D is met; the control group is thoroughly documented.
-
Level 2 Criteria
-
S
School-level RCT
-
I
Independent Conduct
- The same team designed, implemented, and evaluated the study.
- "…delivered to a randomly selected group of students … by the course instructor…"
Relevant Quotes:
1) "The suggestion was delivered to a randomly selected group of students … by the course instructor…"
2) Acknowledgements note collaboration with the instructor and internal lab.
Detailed Analysis:
The research team designed the intervention, implemented it within their own institution, and analyzed the data—no external evaluator participated.
Final sentence: Criterion I is not met; conduct was not independent of developers.
-
Y
Year Duration
-
B
Balanced Resources
- Control and treatment groups received equivalent emails and incentives; no extra resources favored treatment.
- "…ensured that students received an equal number of contacts from the instructor."
Relevant Quotes:
1) "In order to ensure that the control students had an equal number of contacts from the instructor … control students received an email …"
2) "We provided control students with an emailed survey … it ensured that students received an equal number of contacts from the instructor."
Detailed Analysis:
The only resource added by the intervention was the content of the scheduling survey. Control students received parallel emails and extra‑credit opportunities, matching contact time and incentives. No additional instructional time, tutoring, or budget differential existed.
Final sentence: Criterion B is met; resources were balanced across groups.
-
Level 3 Criteria
-
R
Reproduced Results
- No independent replication of this RCT is reported.
Detailed Analysis:
The article reports no independent replication, and a literature search within the paper finds no mention of other teams repeating the same intervention in a separate context. Similarly, no independent replication by different researchers has been published as of 2025.
Final sentence: Criterion R is not met; results have not been independently reproduced.
-
A
All Exams
- Outcomes are limited to a single STEM course, not all subjects.
- "…weekly quizzes … and the final exam score."
Relevant Quotes:
1) "Students’ final grades were determined by … weekly quizzes … and the final exam score."
Detailed Analysis:
Learning outcomes cover only the focal STEM course; the study does not assess other academic subjects, so it fails the all‑subject requirement.
Final sentence: Criterion A is not met because only one subject was tested.
-
G
Graduation Tracking
- No tracking beyond the short 5‑week course was conducted.
- "…final exam … on the last day of the course."
Relevant Quotes:
1) "…the course also had a final exam that was held on campus on the last day of the course."
Detailed Analysis:
Outcomes were measured only up to the final exam at week 5; the study did not track students through course completion or later graduation milestones. To our knowledge, the authors did not publish any follow-up tracking these students beyond the course.
Final sentence: Criterion G is not met because there is no post‑course follow‑up.
-
P
Pre-Registered Protocol
- No evidence of prospective trial registration is provided.
Detailed Analysis:
The manuscript contains no registration ID, registry name, or statement of pre‑registration. A text search of the PDF reveals no occurrence of "prereg" or "registration" related to a study protocol. Additionally, we did not find any corresponding pre-registration in trial registries (e.g., AEA or OSF).
Final sentence: Criterion P is not met; the study was not pre‑registered.
Request an Update or Contact Us
Are you the author of this study? Let us know if you have any questions or updates.
