Abstract
This randomized experiment implemented with school children in India directly tests an input incentive designed to increase effort on learning activities against both an output incentive that rewards test performance and a control. Students in the input incentive treatment perform .58σ better than those in the control, and .34σ better than students in the output incentive treatment. Thus, the input incentive is approximately twice as cost-effective as the output incentive. The input incentive increases the intensive margin of student effort on the learning activity, and it is particularly effective for students that are present-biased as measured at baseline.
Full
Article
ERCT Criteria Breakdown
-
Level 1 Criteria
-
C
Class-level RCT
- Randomization occurred at the classroom level, preventing contamination across students in the same class.
- "Classrooms were randomized into treatments using a partial rotation design over two units." (Section 3.4)
Relevant Quotes:
1) "Classrooms were randomized into treatments using a partial rotation
design over two units." (Section 3.4)
Detailed Analysis:
The paper explicitly describes randomization at the classroom level. By assigning entire classes to input‑incentive, output‑incentive, or control, the study prevents any within‑class contamination between treatment groups. This fully satisfies the ERCT ‘Class‑level RCT’ requirement.
Final sentence: Criterion C is met because the study randomized treatments at the classroom level, ensuring proper separation of groups.
-
E
Exam-based Assessment
- The study employs bespoke KA Lite assessments, not established standardized exams.
- "Each learning module covers a specific topic such as two‑digit by two‑digit multiplication, and the tests at the end of unit draw only on the material covered in the core modules in that unit." (Section 3.2)
Relevant Quotes:
1) "Each learning module covers a specific topic such as two‑digit by
two‑digit multiplication, and the tests at the end of unit draw only on
the material covered in the core modules in that unit." (Section 3.2)
2) "During the study, students took two tests at the end of each unit;
the first is the output activity and the second measures the main
outcome of the study." (Section 3.2)
Detailed Analysis:
The outcome measures rely on custom‑designed KA Lite tests aligned to the modules, not on any standardized, widely recognized examination. Thus, the assessments cannot be considered standardized exam‑based.
Final sentence: Criterion E is not met because the study uses custom KA Lite tests rather than a standardized exam.
-
T
Term Duration
- The study measures outcomes after approximately six weeks, not a full term.
- "Two units of KA Lite content were included in the study, with each unit taking six weeks on average." (Section 3.2)
Relevant Quotes:
1) "Two units of KA Lite content were included in the study, with each
unit taking six weeks on average." (Section 3.2)
Detailed Analysis:
Each instructional unit lasted approximately six weeks, and outcomes were measured immediately thereafter. This period is shorter than a full academic term (typically 3–4 months), so the study does not meet the term‑long follow‑up requirement.
Final sentence: Criterion T is not met because the measurement occurs after six weeks, less than one full academic term.
-
D
Documented Control Group
- The control group’s makeup and activities are thoroughly described.
- "All classrooms in the study, including those in the control and in both the treatments, were expected to complete the same activities on the KA Lite platform." (Section 3.2)
Relevant Quotes:
1) "All classrooms in the study, including those in the control and in
both the treatments, were expected to complete the same activities
on the KA Lite platform." (Section 3.2)
2) "The 17 classrooms initially assigned to the control remained in the
control for both units." (Section 3.4)
Detailed Analysis:
The control group is clearly documented: its composition (17 classrooms), its activities (completion of the same modules and tests without incentives), and its role across both units. This satisfies the requirement for a well‑documented control.
Final sentence: Criterion D is met because the control condition is fully described and its baseline and activities are documented.
-
Level 2 Criteria
-
S
School-level RCT
- The study randomized individual classrooms, not whole schools.
- "Classrooms were randomized into treatments using a partial rotation design over two units." (Section 3.4)
Relevant Quotes:
1) "Classrooms were randomized into treatments using a partial rotation
design over two units." (Section 3.4)
Detailed Analysis:
Randomization occurred at the classroom level within schools, not at the level of entire schools. Therefore, the study does not satisfy the stronger school‑level RCT criterion.
Final sentence: Criterion S is not met because randomization did not occur at the school level.
-
I
Independent Conduct
- The same research team designed and evaluated the intervention.
- "This randomized experiment implemented with school children in India directly tests an input incentive designed to increase effort on learning activities against both an output incentive that rewards test performance and a control." (Abstract)
Relevant Quotes:
1) "This randomized experiment implemented with school children in
India directly tests an input incentive designed to increase effort on
learning activities against both an output incentive that rewards test
performance and a control." (Abstract)
Detailed Analysis:
The author and her team designed, implemented, and analyzed the intervention without mention of an independent third‑party evaluator. All key activities—curriculum development, data collection, and analysis—were conducted by the study team, introducing potential bias.
Final sentence: Criterion I is not met because the intervention and evaluation were conducted by the same research team.
-
Y
Year Duration
- The intervention and measurement occur within ~12 weeks, not a full year.
- "Two units of KA Lite content were included in the study, with each unit taking six weeks on average." (Section 3.2)
Relevant Quotes:
1) "Two units of KA Lite content were included in the study, with each
unit taking six weeks on average." (Section 3.2)
Detailed Analysis:
The total study duration (~12 weeks) is far shorter than a full academic year. No year‑long tracking is reported.
Final sentence: Criterion Y is not met because the follow‑up spans only ~12 weeks, not a full academic year.
-
B
Balanced Resources
- Treatment and control groups received identical time and resources.
- "All classrooms in the study, including those in the control and in both the treatments, were expected to complete the same activities on the KA Lite platform." (Section 3.2)
Relevant Quotes:
1) "All classrooms in the study, including those in the control and in
both the treatments, were expected to complete the same activities
on the KA Lite platform." (Section 3.2)
Detailed Analysis:
No additional time, budget, or materials were provided to the input or output incentive groups beyond what was given to the control. All groups had identical educational inputs, so resource balance is maintained.
Final sentence: Criterion B is met because all groups received the same core inputs without extra resources introduced by the treatments.
-
Level 3 Criteria
-
R
Reproduced Results
-
A
All Exams
- Only mathematics outcomes are measured, not all core subjects.
- "Each learning module covers a specific topic such as two‑digit by two‑digit multiplication, and the tests at the end of unit draw only on the material covered in the core modules in that unit." (Section 3.2)
Relevant Quotes:
1) "Each learning module covers a specific topic such as two‑digit by
two‑digit multiplication, and the tests at the end of unit draw only on
the material covered in the core modules in that unit." (Section 3.2)
Detailed Analysis:
The study measures outcomes only in mathematics. No assessments of other core subjects (e.g., reading, science) are conducted, so cross‑subject balance is not evaluated.
Final sentence: Criterion A is not met because only mathematics is assessed.
-
G
Graduation Tracking
-
P
Pre-Registered Protocol
- The protocol was pre‑registered in the AEA registry before implementation.
- "This study was approved by IRB at UC San Diego and IFMR, and is in the AEA registry as AEARCTR-0000643." (Footnote * on page 3)
Relevant Quotes:
1) "This study was approved by IRB at UC San Diego and IFMR, and is
in the AEA registry as AEARCTR-0000643." (Footnote * on page 3)
Detailed Analysis:
Registration in the AEA registry indicates the study protocol was publicly recorded before data collection. This fulfills the pre‑registration requirement.
Final sentence: Criterion P is met because the study was pre‑registered in the AEA registry (AEARCTR-0000643).
Request an Update or Contact Us
Are you the author of this study? Let us know if you have any questions or updates.
