Abstract
Some education policymakers focus on bringing down pupil–teacher ratios. Others argue that resources will have limited impact without systematic reforms to education governance, teacher incentives, and pedagogy. We examine a program under which school committees at randomly selected Kenyan schools were funded to hire an additional teacher on an annual contract renewable conditional on performance, outside normal Ministry of Education civil-service channels, at one- quarter normal compensation levels. For students randomly assigned to stay with existing classes, test scores did not increase significantly, despite a reduction in class size from 82 to 44 on average. In contrast, scores increased for students assigned to be taught by locally-hired contract teachers. One reason may be that contract teachers had low absence rates, while centrally-hired civil-service teachers in schools randomly assigned contract teachers endogenously reduced their effort. Civil-service teachers also captured rents for their families, with approximately one-third of contract teacher positions going to relatives of existing teachers. A governance program that empowered parents within school committees reduced both forms of capture. The best contract teachers obtained civil-service jobs over time, and we estimate large potential dynamic benefits from supplementing a civil-service system with locally-hired contract teachers brought in on a probationary basis and granted tenure conditional on performance.
Full
Article
ERCT Criteria Breakdown
-
Level 1 Criteria
-
C
Class-level RCT
- Randomisation at the school level satisfies the requirement for a class-level RCT.
- "The 210 schools were randomly divided into a comparison group (70 schools), and an Extra Teacher Program (ETP) group (140 schools)." (p. 8)
Relevant Quotes:
1) "The 210 schools were randomly divided into a comparison group (70 schools), and an Extra Teacher Program (ETP) group (140 schools)." (p. 8)
2) "Students and the contract teacher were then randomly allocated across sections." (p. 4)
Detailed Analysis:
Randomisation was implemented at the school level for the ETP intervention, assigning entire schools to treatment or control. Within treated schools, students were then randomly assigned to classes taught by contract or civil-service teachers. Under the ERCT standard, school-level randomisation meets or exceeds the requirement for class-level RCT.
Final sentence explaining criterion C is met because randomisation was conducted at the school level, satisfying class-level RCT.
-
E
Exam-based Assessment
- The assessments were study-designed instruments, not recognised standardized exams.
- "Standardized tests covering math and literacy questions ranging from identifying numbers and letters to subtracting two-digit numbers and writing words were administered in all schools after 5 school terms (19 months total, but only 15 months of instruction), just before the program ended (November 2006)." (p. 6)
Relevant Quotes:
1) "Standardized tests covering math and literacy questions ranging from identifying numbers and letters to subtracting two-digit numbers and writing words were administered in all schools after 5 school terms (19 months total, but only 15 months of instruction), just before the program ended (November 2006)." (p. 6)
Detailed Analysis:
Although described as "standardized tests," these instruments were created for this study rather than drawn from recognized national or state exams. No official standardized assessment (e.g., KCPE) is cited.
Final sentence explaining criterion E is not met because the study did not employ a widely recognised standardized exam.
-
T
Term Duration
- Measurement occurred after five terms, satisfying at least one full academic term of follow-up.
- "Standardized tests covering math and literacy questions ranging from identifying numbers and letters to subtracting two-digit numbers and writing words were administered in all schools after 5 school terms (19 months total, but only 15 months of instruction), just before the program ended (November 2006)." (p. 6)
Relevant Quotes:
1) "Standardized tests covering math and literacy questions ranging from identifying numbers and letters to subtracting two-digit numbers and writing words were administered in all schools after 5 school terms (19 months total, but only 15 months of instruction), just before the program ended (November 2006)." (p. 6)
Detailed Analysis:
Outcomes were measured after five school terms (~1.5 academic years) following the intervention start, exceeding the requirement to measure at least one full term after the intervention begins.
Final sentence explaining criterion T is met because outcomes were measured more than one term after the intervention start.
-
D
Documented Control Group
- The paper provides detailed baseline characteristics and conditions for the control group in Table 1.
- "Panels A and B of Table 1 present summary statistics at the school level." (p. 8)
Relevant Quotes:
1) "Panels A and B of Table 1 present summary statistics at the school level." (p. 8)
2) "Panel A shows no significant differences in observable characteristics at baseline." (p. 8)
Detailed Analysis:
The control (comparison) group is well documented in Table 1 with baseline demographics and school characteristics, and no special treatments beyond business-as-usual.
Final sentence explaining criterion D is met because the control group is clearly documented with baseline data.
-
Level 2 Criteria
-
S
School-level RCT
- Entire schools, not just classes, were randomly assigned to treatment or control.
- "The 210 schools were randomly divided into a comparison group (70 schools), and an Extra Teacher Program (ETP) group (140 schools)." (p. 8)
Relevant Quotes:
1) "The 210 schools were randomly divided into a comparison group (70 schools), and an Extra Teacher Program (ETP) group (140 schools)." (p. 8)
Detailed Analysis:
Randomisation at the level of entire schools satisfies the Level 2 requirement for school-level RCT, capturing broader school-wide factors and real-world implementation scenarios.
Final sentence explaining criterion S is met because randomisation occurred at the school level.
-
I
Independent Conduct
- The same research team and ICS officers designed, implemented, and analyzed the intervention without independent oversight.
Relevant Quotes:
None found.
Detailed Analysis:
The intervention design, implementation, data collection, and analysis were conducted by the authors and affiliated organizations without mention of an independent third-party evaluator.
Final sentence explaining criterion I is not met because no external evaluation is reported.
-
Y
Year Duration
- The study tracked outcomes for more than an academic year, satisfying the Year Duration requirement.
- "The same tests were administered again during a follow-up one year after the program ended (November 2007)." (p. 6)
Relevant Quotes:
1) "Over the course of the program, five unannounced visits were made to each school by the research team." (pp. 6–7)
2) "The same tests were administered again during a follow-up one year after the program ended (November 2007)." (p. 6)
Detailed Analysis:
Outcomes were tracked through five school terms (~19 months) and included a one-year follow-up after program end, covering more than a full academic year.
Final sentence explaining criterion Y is met because tracking covered at least one full academic year.
-
B
Balanced Resources
- The additional teacher is the treatment variable, so business-as-usual resourcing in the control group is acceptable.
- "... school committees at randomly selected Kenyan schools were funded to hire an additional teacher on an annual contract renewable conditional on performance, outside normal Ministry of Education civil-service channels, at one-quarter normal compensation levels." (p. 1)
Relevant Quotes:
1) "... school committees at randomly selected Kenyan schools were funded to hire an additional teacher on an annual contract renewable conditional on performance, outside normal Ministry of Education civil-service channels, at one-quarter normal compensation levels." (p. 1)
Detailed Analysis:
Providing an extra teacher is the core treatment variable. Under the ERCT decision tree, when additional resources are the treatment, business-as-usual resourcing in the control group is acceptable.
Final sentence explaining criterion B is met because the extra teacher is integral to the intervention and the control group received standard resourcing.
-
Level 3 Criteria
-
R
Reproduced Results
- Multiple independent studies have replicated this finding.
- "At the end of two years, students in schools with an extra contract teacher performed significantly better than those in comparison schools by 0.16σ and 0.15σ, in math and language tests respectively." (Muralidharan & Sundararaman 2013)
Relevant Quotes:
1) "At the end of two years, students in schools with an extra contract teacher performed significantly better than those in comparison schools by 0.16σ and 0.15σ," in math and language tests respectively." (Muralidharan & Sundararaman 2013)
2) "In previous RCTs in India and Western Kenya, hiring contract teachers has been shown to improve test scores among primary school students." (World Bank 2012)
Detailed Analysis:
The positive impacts of this contract teacher intervention have been reproduced in other settings. For instance, a randomized study in India reported significant test score gains for students in schools that received an extra contract teacher. Additionally, a multi-district trial in Kenya found that contract teachers raised test scores when implemented by an NGO, mirroring the original study’s result.
Final sentence explaining criterion R is met because there is evidence of independent replication of the results.
-
A
All Exams
- Only math and reading outcomes were measured, failing to cover all core subjects.
- "Standardized tests covering math and literacy questions ..." (p. 6)
Relevant Quotes:
1) "Standardized tests covering math and literacy questions ..." (p. 6)
Detailed Analysis:
The study only assesses outcomes in mathematics and reading, without measuring other core subjects. No justification for this narrow focus is provided.
Final sentence explaining criterion A is not met because only two subjects were assessed.
-
G
Graduation Tracking
-
P
Pre-Registered Protocol
Request an Update or Contact Us
Are you the author of this study? Let us know if you have any questions or updates.
