Abstract
Social‑Emotional Learning (SEL) programs are school‑based preventive interventions that aim to improve children’s social‑emotional skills and behaviors. Although meta‑analytic research has shown that SEL programs implemented in early childhood can improve academic and behavioral outcomes in the short‑term, there is limited work examining program effects on children’s math and language skills in the longer‑term. Moreover, few studies have considered variation in impacts by children’s pre‑intervention academic skills. Using an experimental design, the current study leveraged administrative data available through school records (N = 353) to examine the impacts of one SEL program—INSIGHTS into Children’s Temperament— implemented in early elementary school on math and language standardized test scores from third through sixth grade. Findings revealed positive average treatment effects on English/Language Arts (ELA) test scores in third and fourth grade, but not in fifth and sixth grade. Students who had higher academic skills at study enrollment showed lasting impacts on ELA scores in fourth, fifth, and sixth grade. There were no treatment impacts on math skills, and no variation in effects on math achievement by baseline skills. Implications are discussed.
Full
Article
ERCT Criteria Breakdown
-
Level 1 Criteria
-
C
Class-level RCT
- The study randomized entire schools to treatment or control, satisfying the requirement for class‑level RCT.
- “22 elementary schools … were randomly assigned to participate in the INSIGHTS intervention or to an attention‑control condition.” (p. 9)
Relevant Quotes:
1) “Between 2008 and 2012, 22 elementary schools … were randomly assigned to participate in the INSIGHTS intervention or to an attention‑control condition.” (p. 9)
Detailed Analysis:
The unit of randomization was entire schools, not individual students or classes within schools. Under the ERCT Standard, school‑level random assignment satisfies or exceeds the requirement for class‑level RCTs.
Final sentence: Criterion C is met because randomization occurred at the school level, ensuring whole‑class (and whole‑school) assignment.
-
E
Exam-based Assessment
- The study employed official state standardized exams for ELA and math, meeting the exam‑based assessment requirement.
- “We accessed students’ scores on the state English Language Arts (ELA) and math standardized test scores through administrative records.”
Relevant Quotes:
1) “As noted above, we accessed students’ scores on the state English Language Arts (ELA) and math standardized test scores through administrative records.” (Methods)
Detailed Analysis:
The outcomes were measured with state‑administered standardized exams in ELA and math, not custom instruments developed by the authors. Therefore, the assessment tool is a recognized, externally validated exam.
Final sentence: Criterion E is met because the study used official state standardized tests.
-
T
Term Duration
- Outcomes were measured in subsequent grades, well after at least one full academic term had elapsed, fulfilling the term‑duration requirement.
- “Children began participating … in kindergarten … and continued … through the end of their first grade year.”
Relevant Quotes:
1) “Children began participating in the study when they were in kindergarten (Study Year 1) and continued … through the end of their first grade year (Study Year 2).” (Methods)
2) Follow‑up test scores were collected in third through sixth grade, spanning multiple academic years.
Detailed Analysis:
Outcomes were measured at least one full academic term after the intervention began (indeed, several years later). This far exceeds the minimum requirement of term‑long follow‑up.
Final sentence: Criterion T is met because outcomes were assessed at least a full term—and in fact multiple years—after the intervention started.
-
D
Documented Control Group
- The control group’s makeup, baseline statistics, and alternate program are clearly documented, satisfying the requirement for a documented control group.
- “Schools not assigned to INSIGHTS participated in a 10‑week, supplemental reading program after school hours …”
Relevant Quotes:
1) “Schools not assigned to INSIGHTS participated in a 10‑week, supplemental reading program after school hours … teachers and parents attended two 2‑h workshops …” (Methods)
2) Table 2 and Table 3 describe attrition and baseline equivalence for both INSIGHTS and control groups. (Results)
Detailed Analysis:
The paper provides demographic and baseline test‑score information for the attention‑control group, and clearly documents that they received a specified alternate activity rather than the SEL program. This level of detail meets the requirement for a well‑documented control group.
Final sentence: Criterion D is met because the control condition’s composition, baseline data, and “attention‑control” activities are fully described.
-
Level 2 Criteria
-
S
School-level RCT
- Randomization at the school level fulfills the school‑level RCT criterion.
- “22 elementary schools … were randomly assigned to participate …”
Relevant Quotes:
1) “… 22 elementary schools … were randomly assigned …” (p. 9)
Detailed Analysis:
School‑level randomization clearly meets the standard for S (School‑level RCT), which requires entire schools to be assigned as units.
Final sentence: Criterion S is met because the randomization occurred at the school level.
-
I
Independent Conduct
- The study was implemented and analyzed by the same team that developed the program, so there is no independent evaluation group.
Relevant Quotes:
1) The authors and their institutions both designed and evaluated the INSIGHTS program. No external evaluation team is mentioned.
Detailed Analysis:
The same research team who developed INSIGHTS conducted data collection and analysis. There is no indication of an independent third‑party evaluator.
Final sentence: Criterion I is not met because the evaluation was conducted by the intervention developers themselves, not an independent group.
-
Y
Year Duration
- Students’ outcomes were measured over multiple grades, covering at least a full academic year of follow‑up.
Relevant Quotes:
1) Follow‑up test scores were collected in third through sixth grade, covering multiple years post‑intervention.
Detailed Analysis:
Although the core intervention spanned 10 weeks per year, the study tracked academic outcomes for several years—well beyond one full academic year.
Final sentence: Criterion Y is met because the study follows students through an entire school year and beyond.
-
B
Balanced Resources
- The control group’s activities were far less intensive than INSIGHTS, so resource allocation was not balanced.
Relevant Quotes:
1) INSIGHTS schools received extensive teacher, parent, and classroom sessions over 10 weeks per year.
2) Attention‑control schools received only two 2‑h workshops for teachers and parents, with no comparable classroom sessions.
Detailed Analysis:
The control condition did not match the treatment group in terms of session frequency or duration. This imbalance in instructional input means that differences in outcomes could be partly due to resource differences, violating the balanced‑control requirement.
Final sentence: Criterion B is not met because the control group did not receive equivalent instructional time or resources.
-
Level 3 Criteria
-
R
Reproduced Results
- There is no reference to an external, independent replication of the INSIGHTS trial.
Relevant Quotes:
1) No mention is made of any replication of the INSIGHTS trial by an independent research team in this paper.
Detailed Analysis:
The original study has not been reproduced by a separate team in a peer‑reviewed context, nor does this article reference any such work.
Final sentence: Criterion R is not met because there is no evidence of independent replication.
-
A
All Exams
- The authors measured only ELA and math; other core subjects were not assessed.
Relevant Quotes:
1) Outcomes were limited to ELA and math standardized test scores.
Detailed Analysis:
The study did not assess academic performance in other subjects (e.g., science, social studies). Thus it fails the “All‑subject Exams” requirement.
Final sentence: Criterion A is not met because only two subjects were tested, and no justification for exclusion of others is provided.
-
G
Graduation Tracking
- No data are reported beyond sixth grade (middle school entry), so graduation tracking is incomplete.
Relevant Quotes:
1) Final test scores were collected in sixth grade, with no tracking to graduation from elementary or middle school.
Detailed Analysis:
Although outcomes were measured several years post‑intervention, there was no follow‑up through the end of the relevant school level (e.g., graduation from elementary or middle school).
Final sentence: Criterion G is not met because the study does not follow participants through to a formal graduation milestone.
-
P
Pre-Registered Protocol
- No pre‑registered protocol or registry reference is provided in the paper.
Relevant Quotes:
1) There is no mention of pre‑registration or registry identifiers anywhere in the article.
Detailed Analysis:
The study does not cite any pre‑registered protocol or registry ID before data collection, failing the pre‑registration requirement.
Final sentence: Criterion P is not met because no pre‑registration is documented.
Request an Update or Contact Us
Are you the author of this study? Let us know if you have any questions or updates.
