Abstract
We compared students’ self-reported perception of learning with their actual learning under controlled conditions in large-enrollment introductory college physics courses taught using active instruction and passive lecture. Both groups received identical content and handouts, and students were randomly assigned without persuasion. Students in active classrooms learned more but perceived they learned less than peers in passive settings. This disconnect, driven by increased cognitive effort, may impair motivation and self-regulation. We discuss early-semester strategies instructors can use to improve student response to active engagement.
Full
Article
ERCT Criteria Breakdown
-
Level 1 Criteria
-
C
Class-level RCT
- Students were randomized at the individual level rather than by classroom, so cross-group contamination could occur.
- “Students were randomly assigned to 2 groups and told to report to 2 different classrooms: room A with instructor A and room B with instructor B.” (p. 2)
Relevant Quotes:
1) “Students were randomly assigned to 2 groups and told to report to 2 different classrooms: room A with instructor A and room B with instructor B.” (p. 2)
Detailed Analysis:
The ERCT C criterion requires randomisation at the class level (or stronger school level). Here, randomisation occurred at the individual student level for each session, which does not satisfy the requirement for class-level assignment.
Final sentence explaining if criterion C is met/not met: Criterion C is not met because randomisation was done at the student level, not at the class level.
-
E
Exam-based Assessment
- The learning outcomes were measured using custom-built tests, not standardized exams.
- “At the end of each class period, students completed ... followed by a multiple-choice test of learning (TOL).” (p. 2)
Relevant Quotes:
1) “At the end of each class period, students completed a brief survey ... followed by a multiple-choice test of learning (TOL).” (p. 2)
2) “The author of the TOLs ... wrote the tests based only on a list of detailed learning objectives.” (p. 2)
Detailed Analysis:
The ERCT E criterion demands the use of widely recognized standardized exam-based assessments. In this study, the TOL instruments were custom-designed by the authors and are not standard exams.
Final sentence explaining if criterion E is met/not met: Criterion E is not met because custom assessments were used instead of standardized exams.
-
T
Term Duration
- Outcomes were measured immediately after two class meetings, not after a full academic term.
- “The experimental intervention took place during 2 consecutive class meetings in week 12 of each course.” (p. 2)
Relevant Quotes:
1) “The experimental intervention took place during 2 consecutive class meetings in week 12 of each course.” (p. 2)
2) “At the end of each class period, students completed ... a multiple-choice test of learning (TOL).” (p. 2)
Detailed Analysis:
The ERCT T criterion requires that outcomes be measured at least one full academic term after intervention start. Here the measurement occurred within two sessions in one week.
Final sentence explaining if criterion T is met/not met: Criterion T is not met because the follow‑up was shorter than a term.
-
D
Documented Control Group
- The control condition is clearly described with baseline group characteristics and identical materials.
- “Students ... these groups were indistinguishable on several measures of physics background and proficiency (Table 1).” (p. 2)
Relevant Quotes:
1) “Students were randomly assigned ... and these groups were indistinguishable on several measures of physics background and proficiency (Table 1).” (p. 2)
2) “The lecture slides, handouts, and written feedback provided during each class were identical for active instruction and for passive lecture.” (p. 2)
Detailed Analysis:
ERCT D requires detailed documentation of the control group’s composition, baseline data, and procedures. This study provides both demographic and proficiency data for the control group and describes their conditions.
Final sentence explaining if criterion D is met/not met: Criterion D is met because the control group is fully documented.
-
Level 2 Criteria
-
S
School-level RCT
- Randomisation occurred at the student level, not at the school level.
- “Students were randomly assigned to 2 groups ...” (p. 2)
Relevant Quotes:
1) “Students were randomly assigned to 2 groups...” (p. 2)
Detailed Analysis:
ERCT S requires randomisation at the school level. This study’s design assigns individual students, not entire schools.
Final sentence explaining if criterion S is met/not met: Criterion S is not met because school-level assignment did not occur.
-
I
Independent Conduct
- The same research team designed and conducted the intervention, with no third-party evaluator.
- “Both instructors had extensive ... training in active learning ...” (p. 2)
Relevant Quotes:
1) “Both instructors had extensive, identical training in active learning ...” (p. 2)
2) “The author of the TOLs did not have access to the course materials ...” (p. 2)
Detailed Analysis:
ERCT I requires that the study be conducted independently of its designers. Here, the same team created and ran the study, without external evaluation.
Final sentence explaining if criterion I is met/not met: Criterion I is not met due to lack of independent conduct.
-
Y
Year Duration
-
B
Balanced Resources
- Time and materials were identical for both conditions, with only active engagement toggled.
- “The lecture slides, handouts, and written feedback ... were identical for active instruction and for passive lecture.” (p. 2)
Relevant Quotes:
1) “The lecture slides, handouts, and written feedback provided during each class were identical for active instruction and for passive lecture.” (p. 2)
2) “Only active engagement with the material was toggled on and off.” (p. 4)
Detailed Analysis:
ERCT B requires balanced time and resources. Both conditions used identical content, time, and budget; the only variable was mode of engagement.
Final sentence explaining if criterion B is met/not met: Criterion B is met because resources and time are balanced.
-
Level 3 Criteria
-
R
Reproduced Results
- The study has been independently replicated by a different research team.
- “Participants in the large-group interactive session scored 0.27 standard deviations higher on the test of learning (p = 0.010) than in the passive lecture.” (Boedeker et al., 2024)
Relevant Quotes:
1) “Participants in the large-group interactive session scored 0.27 standard deviations higher on the test of learning (p = 0.010) than in the lecture-based session.” (Boedeker et al., 2024)
2) “The feeling of learning was 0.56 standard deviations higher in the large-group interactive session (p < 0.001).” (Boedeker et al., 2024)
Detailed Analysis:
ERCT R requires independent replication by a separate team. In this case, an independent study by Boedeker et al. (2024) reproduced the experiment in a medical school context. The quotes above show that they observed improved learning outcomes with active learning, confirming the generalizability of the original findings.
Final sentence explaining if criterion R is met/not met: Criterion R is met because an independent replication by another team has been published.
-
A
All Exams
-
G
Graduation Tracking
-
P
Pre-Registered Protocol
Request an Update or Contact Us
Are you the author of this study? Let us know if you have any questions or updates.
