Abstract
Widespread smartphone bans are being implemented in classrooms worldwide, yet their causal effects on student outcomes remain unclear. In a randomized controlled trial involving nearly 17,000 students, we find that mandatory in-class phone collection led to higher grades - particularly among lower-performing, first-year, and non-STEM students - with an average increase of 0.086 standard deviations. Importantly, students exposed to the ban were substantially more supportive of phone-use restrictions, perceiving greater benefits from these policies and displaying reduced preferences for unrestricted access. This enhanced student receptivity to restrictive digital policies may create a self-reinforcing cycle, where positive firsthand experiences strengthen support for continued implementation. Despite a mild rise in reported fear of missing out, there were no significant changes in overall student well-being, academic motivation, digital usage, or experiences of online harassment. Random classroom spot checks revealed fewer instances of student chatter and disruptive behaviors, along with reduced phone usage and increased engagement among teachers in phone-ban classrooms, suggesting a classroom environment more conducive to learning. Spot checks also revealed that students appear more distracted, possibly due to withdrawal from habitual phone checking, yet, students did not report being more distracted. These results suggest that in-class phone bans represent a low-cost, effective policy to modestly improve academic outcomes, especially for vulnerable student groups, while enhancing student receptivity to digital policy interventions.
Full
Article
ERCT Criteria Breakdown
-
Level 1 Criteria
-
C
Class-level RCT
- Randomization occurred at the department-grade (cohort) level, meeting the class-level RCT requirement.
- Student cohorts were randomly assigned at the department-grade level (i.e., a cohort) to either a mandatory in-class phone collection-requiring students to deposit their devices at the start of each lecture-or to a business-as-usual control group, where phone use remained unrestricted.
Relevant Quotes:
1) "Student cohorts were randomly assigned at the department-grade level (i.e., a cohort) to either a mandatory in-class phone collection-requiring students to deposit their devices at the start of each lecture-or to a business-as-usual control group, where phone use remained unrestricted." (pp. 2-3)
2) "We randomized academic departments within each HEIs, stratified by semester, into treatment and control courses with equal probability." (p. 14)
Detailed Analysis:
The unit of randomization is the department-grade cohort within each higher education institute. This is a group-level assignment (above the individual student) and is at least as strong as a class-level RCT for preventing contamination between treatment and control students.
Criterion C is met because treatment was randomized at a cohort (department-grade) level rather than within the same classroom.
-
E
Exam-based Assessment
- The study measured outcomes using official final grades from institutional examination records.
- Specifically, we obtained official final grades from institutional examination records and cross-checked them with departmental transcripts.
Relevant Quotes:
1) "First, we obtained each student's objective final grades and attendance records from both before and during the study period." (p. 14)
2) "Specifically, we obtained official final grades from institutional examination records and cross-checked them with departmental transcripts." (p. 21)
Detailed Analysis:
Academic outcomes are based on official final grades taken from institutional examination records. These are standard assessments used by the institutions for their courses and were not created by researchers for this intervention.
Criterion E is met because outcomes use official exam-based grades rather than a custom assessment designed for the study.
-
T
Term Duration
- The study ran for the full Spring 2024 semester (February-May), meeting the term-duration requirement.
- The treatment period spanned the full Spring 2024 semester (February-May 2024), although start and end dates varied slightly by institution.
Relevant Quotes:
1) "The treatment period spanned the full Spring 2024 semester (February-May 2024), although start and end dates varied slightly by institution." (p. 19)
2) "This protocol was applied in all classes for the treatment arm throughout the Spring 2024 semester, while students in the control departments maintained their usual access to phones." (p. 14)
Detailed Analysis:
The intervention and outcome tracking cover the full Spring 2024 semester (February-May), which corresponds to an academic term (approximately one semester).
Criterion T is met because outcomes were measured after at least one full academic term of exposure.
-
D
Documented Control Group
- The control condition is clearly described as business-as-usual with unrestricted phone access and documented baseline balance.
- In control departments, teachers and students continued their regular classroom routines, retaining access to smartphones during lectures.
Relevant Quotes:
1) "Student cohorts were randomly assigned ... to a business-as-usual control group, where phone use remained unrestricted." (pp. 2-3)
2) "In control departments, teachers and students continued their regular classroom routines, retaining access to smartphones during lectures." (p. 19)
3) "Baseline academic performance is balanced across treatment and control groups, with no statistically significant differences in prior GPA levels under any specification (SI Table B.1)." (p. 3)
Detailed Analysis:
The paper describes the control condition as business-as-usual with unrestricted phone access. It also documents baseline comparability using prior GPA balance tests, supporting that the control group is clearly specified and empirically characterized.
Criterion D is met because the control condition is described and baseline performance balance is documented.
-
Level 2 Criteria
-
S
School-level RCT
- Randomization was within institutions (departments/cohorts), not between institutions, so it is not a school-level RCT.
- We randomized academic departments within each HEIs, stratified by semester, into treatment and control courses with equal probability.
Relevant Quotes:
1) "We randomized academic departments within each HEIs, stratified by semester, into treatment and control courses with equal probability." (p. 14)
2) "Notably, many classrooms were shared by both treatment and control arms." (p. 14)
Detailed Analysis:
Randomization occurred within each higher education institute by assigning departments (and cohorts) to treatment or control. Because institutions themselves were not randomized, the study is not a school-level RCT in the ERCT sense.
Criterion S is not met because assignment was within schools, not among schools.
-
I
Independent Conduct
- The authors designed, implemented, and analyzed the study themselves, with no independent evaluator described.
- A.S. and A.B.N. jointly developed the initial research concept and study design.
Relevant Quotes:
1) "A.S. and A.B.N. jointly developed the initial research concept and study design." (p. 15)
2) "A.S. and P.K.C. led field operations and coordination of the research assistant team." (p. 15)
3) "A.S. and A.B.N. conducted the data analysis and drafted the manuscript." (p. 15)
Detailed Analysis:
The author contribution statement shows that the same author team designed the study, led field operations, and conducted the analysis. The paper does not describe an independent third-party evaluation organization or an external analysis team separated from the intervention designers.
Criterion I is not met because the study was implemented and analyzed by the same team that designed it, without independent evaluation.
-
Y
Year Duration
- The study covers one semester rather than a full academic year, so the year-duration requirement is not met.
- The treatment period spanned the full Spring 2024 semester (February-May 2024), although start and end dates varied slightly by institution.
Relevant Quotes:
1) "The treatment period spanned the full Spring 2024 semester (February-May 2024), although start and end dates varied slightly by institution." (p. 19)
2) "This protocol was applied in all classes for the treatment arm throughout the Spring 2024 semester, while students in the control departments maintained their usual access to phones." (p. 14)
Detailed Analysis:
Outcome tracking is limited to a single semester (February-May 2024), which is substantially shorter than a full academic year.
Criterion Y is not met because the study duration does not cover a full academic year.
-
B
Balanced Control Group
- Phone boxes were installed in all classrooms and the intervention was a policy enforcement rather than added instructional resources, so inputs are balanced.
- The key difference between the two arms lies not in the presence of the boxes themselves, but in the enforcement protocol requiring students in the treatment arm to deposit their phones.
Relevant Quotes:
1) "Notably, many classrooms were shared by both treatment and control arms. Therefore, we placed phone-boxes in all classrooms." (p. 14)
2) "The key difference between the two arms lies not in the presence of the boxes themselves, but in the enforcement protocol requiring students in the treatment arm to deposit their phones." (p. 14)
3) "In control departments, teachers and students continued their regular classroom routines, retaining access to smartphones during lectures." (p. 19)
Detailed Analysis:
The intervention did not add instructional time, tutoring, or other educational resources; it changed a classroom policy (mandatory phone deposit). The main material input (phone boxes) was installed in all classrooms because rooms were shared, so the control condition matched the physical resource. The remaining difference is the enforcement protocol, which is the intervention being tested.
Criterion B is met because the only notable material resource was provided in both arms and the study does not introduce extra instructional time or budget in treatment beyond the intended policy enforcement.
-
Level 3 Criteria
-
R
Reproduced
- No peer-reviewed independent replication of this specific RCT was found.
Relevant Quotes:
1) "We report on a randomized controlled trial involving 16,955 students across 10 higher ed-ucation institutions, conducted in Spring 2024." (p. 2)
Detailed Analysis:
The paper reports a single original RCT. The paper does not claim that it is a replication of a prior RCT, and it does not report an independent replication conducted by other authors. A web search (by title and DOI) did not identify any peer-reviewed, independent reproduction of this specific RCT as of the ERCT check date.
Criterion R is not met because no independent, peer-reviewed replication of this study was found.
-
A
All-subject Exams
- The outcome reflects overall grades across (nearly) all courses, providing an all-subject style assessment rather than a single subject test.
- Because assignment was at the department-grade level, students experienced the same treatment condition consistently across nearly all their courses throughout the semester.
Relevant Quotes:
1) "Because assignment was at the department-grade level, students experienced the same treatment condition consistently across nearly all their courses throughout the semester." (p. 3)
2) "We obtained each student's objective final grades and attendance records from both before and during the study period." (p. 14)
Detailed Analysis:
The main academic outcome is overall grades (GPA / average grades) and the treatment assignment applies across nearly all courses taken by students in the cohort. This means the outcome aggregates performance across many subjects/courses rather than measuring only a single subject. Criterion E is met, so exam-based course grades can be used for this criterion.
Criterion A is met because the study evaluates broad academic performance across (nearly) all courses rather than a single subject outcome.
-
G
Graduation Tracking
- The paper reports only semester-end outcomes and does not track students to graduation.
- Our findings also suggest that improved academic performance due to classroom phone bans could have downstream effects on graduation rates.
Relevant Quotes:
1) "The treatment period spanned the full Spring 2024 semester (February-May 2024), although start and end dates varied slightly by institution." (p. 19)
2) "Our findings also suggest that improved academic performance due to classroom phone bans could have downstream effects on graduation rates." (p. 5)
Detailed Analysis:
The study measures outcomes during a single semester and does not report follow-up until graduation. The only mention of graduation is speculative ("could have downstream effects") rather than reporting observed graduation outcomes. In addition, criterion Y is not met (the study does not span a full academic year), and under the ERCT rule this implies criterion G cannot be met for this paper.
Criterion G is not met because participants were not tracked to graduation and outcomes were measured only within one semester.
-
P
Pre-Registered
- The paper links to a pre-registration, and the registry date is before the Spring 2024 data collection period.
- Pre-registration details can be found at https://aspredicted.org/8dv5- qd78.pdf Further details of the study is available in SI Appendix Section A.
Relevant Quotes:
1) "Pre-registration details can be found at https://aspredicted.org/8dv5- qd78.pdf Further details of the study is available in SI Appendix Section A." (p. 13)
2) "Our pre-registered randomized controlled trial protocol comprised three avenues of data collection." (p. 14)
3) "Our pre-registered analysis plan is available at https://aspredicted.org/8dv5-qd78.pdf." (p. 26)
Additional Evidence (registry):
4) "Pre-registered on 2024/01/30 13:43 (PT)" (AsPredicted #160,190)
5) "No, no data have been collected for this study yet." (AsPredicted #160,190)
Detailed Analysis:
The paper links to a specific pre-registration record on AsPredicted and repeatedly describes the protocol and analysis plan as pre-registered. The registry page reports a pre-registration timestamp of 2024/01/30, which is before the Spring 2024 study period described in the paper.
Criterion P is met because the study provides a pre-registration link and the registry shows it was registered before data collection began.
Request an Update or Contact Us
Are you the author of this study? Let us know if you have any questions or updates.
