A national experiment reveals where a growth mindset improves achievement

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomisation was conducted at the individual student level within schools rather than at the class level, leading to potential contamination across students in the same class.
    • "12,490 ninth-grade adolescents who were individually randomized to condition." (p. 1)
    • Relevant Quotes: 1) "12,490 ninth-grade adolescents who were individually randomized to condition." (p. 1) 2) "The coin-tossing symbol indicates that random assignment was made during session 1." (Fig. 1, p. 2) Detailed Analysis: The paper describes random assignment of individual students within schools ("individually randomized"), rather than assigning entire classes to treatment or control. The ERCT 'C' criterion requires at minimum class-level randomisation to prevent contamination between groups. This individual-level randomisation fails to satisfy the class-level RCT requirement. Thus, criterion C is not met because randomization occurred at the student level within classes rather than at the class level, introducing a risk of contamination.
  • E

    Exam-based Assessment

    • The primary outcome is based on post‑intervention grade point averages from school records, not a standardized exam‑based assessment.
    • "The primary outcome was the post‑intervention grade point average (GPA) in core ninth‑grade classes…" (p. 3)
    • Relevant Quotes: 1) "The primary outcome was the post‑intervention grade point average (GPA) in core ninth‑grade classes…" (p. 3) 2) "Grades and course taking… After ninth grade." (Fig. 1, p. 2) Detailed Analysis: The ERCT 'E' criterion requires use of a standardized exam‑based assessment. Here, the study uses administrative GPAs, which vary by school and teacher and are not a uniform, standardized test. Therefore, the criterion is not satisfied. Thus, criterion E is not met because the study's outcomes are GPAs rather than standardized exam scores.
  • T

    Term Duration

    • Outcomes (end‑of‑year GPAs) were measured at the end of the academic year, at least one full term after the intervention.
    • "Grades and course taking… After ninth grade." (Fig. 1, p. 2)
    • Relevant Quotes: 1) "Intervention and survey… Before data collection Ninth grade 1–4 weeks later* After ninth grade" (Fig. 1, p. 2) Detailed Analysis: The timeline shows that the intervention occurred early in the ninth‑grade year, and outcomes were measured at the end of the same school year. This covers more than one full academic term (~9 months), satisfying the requirement for term‑duration follow‑up. Therefore, criterion T is met because outcomes were measured at the end of the school year, more than one full term after the intervention began.
  • D

    Documented Control Group

    • The control condition and its baseline data are clearly described, including content, fidelity, and demographics.
    • "The control condition, focusing on brain functions, was similar to the growth mindset intervention, but did not address beliefs about intelligence." (p. 8)
    • Relevant Quotes: 1) "The control condition, focusing on brain functions, was similar to the growth mindset intervention, but did not address beliefs about intelligence." (p. 8) Detailed Analysis: The paper documents the control group’s materials, delivery protocol and fidelity metrics. This level of documentation meets the ERCT 'D' criterion for a documented control group. Thus, criterion D is met as the control group was thoroughly documented.

• Level 2 Criteria

  • S

    School-level RCT

    • Randomisation occurred at the student level within schools, not at the school level.
    • "individually randomized to condition." (p. 1)
    • Relevant Quotes: 1) "12,490 ninth‑grade adolescents who were individually randomized to condition." (p. 1) Detailed Analysis: Level 2 criterion 'S' requires randomisation among schools as units. This study randomized individual students within schools, so it fails to meet the school‑level RCT requirement. Hence, criterion S is not met since randomization was done at the student level rather than the school level.
  • I

    Independent Conduct

    • Independent professional research companies conducted data collection and processing, separate from the intervention designers.
    • "Two different professional research companies… were contracted." (p. 7)
    • Relevant Quotes: 1) "One company (ICF)… obtained administrative data… clean and merged data… blind to treatment conditions." (p. 7) 2) "A second professional research company (MDRC) processed the data… blind to the consequences… as described in Supplementary Information section 12." (p. 7) Detailed Analysis: The study explicitly describes the use of independent firms for recruitment, data collection, and data processing, separate from the design team. This satisfies the ERCT 'I' criterion for independent conduct. Therefore, criterion I is met because independent external firms handled the implementation and data, minimizing bias.
  • Y

    Year Duration

    • Outcomes were tracked through the end of ninth grade, covering a full academic year.
    • "After ninth grade" (Fig. 1, p. 2)
    • Relevant Quotes: 1) "After ninth grade" (Fig. 1 timeline) Detailed Analysis: The study measured outcomes at the end of ninth grade, approximately one academic year after start, thus meeting the Year‑Duration requirement. Thus, criterion Y is met, having tracked outcomes through approximately one academic year.
  • B

    Balanced Resources

    • Both intervention and control groups received equivalent session time and attention, balancing educational inputs.
    • "The intervention consisted of two self-administered online sessions… The control condition… was similar… but did not address beliefs about intelligence." (p. 8)
    • Relevant Quotes: 1) "The intervention consisted of two self-administered online sessions… lasted approximately 25 min each." (p. 8) 2) "The control condition, focusing on brain functions, was similar…" (p. 8) Detailed Analysis: Both groups engaged in two sessions of equal duration and structure. There was no difference in time or materials beyond content focus, satisfying the Balanced Control Group criterion. Therefore, criterion B is met since both the intervention and control groups received equal session time and similar materials (only the content differed).

• Level 3 Criteria

  • R

    Reproduced Results

    • No independent replication of this national study by a different team is reported.
    • Relevant Quotes: 1) No mention of subsequent, independent replication of this specific experiment appears in the paper. Detailed Analysis: The paper references previous pilot studies but does not report an independent replication of the National Study of Learning Mindsets. Since publication, an independent analysis by a third party (MDRC) corroborated the main results using the original data, and a similar intervention was tested in another country (Norway) with support from the original authors. However, no fully independent replication study has been published to date. As such, criterion R is not met because no independent replication by an external team has been reported.
  • A

    All Exams

    • No standardized exam-based assessments across all core subjects; the study relies on administrative GPAs.
    • Relevant Quotes: 1) "The primary outcome was the post‑intervention grade point average (GPA) in core ninth‑grade classes…" (p. 3) Detailed Analysis: All‑subject Exams requires standardized exam scores across major subjects. Using GPAs from school records does not meet the standardized exam requirement. Hence, criterion A is not met as no standardized exams across all core subjects were used.
  • G

    Graduation Tracking

    • Participants were only tracked through ninth grade; no graduation tracking is reported.
    • Relevant Quotes: 1) No follow-up beyond ninth grade is described. Detailed Analysis: The study concludes data collection at the end of ninth grade with no evidence of further graduate tracking, failing the Graduation Tracking criterion. Thus, criterion G is not met because the study did not follow students through to high school graduation.
  • P

    Pre-Registered Protocol

    • The analysis plan and moderation hypotheses were pre-registered on OSF prior to data analysis.
    • "The tick symbol indicates that a comprehensive analysis plan was pre-registered at https://osf.io/tn6g4." (Fig. 1 legend, p. 2)
    • Relevant Quotes: 1) "The tick symbol indicates that a comprehensive analysis plan was pre-registered at https://osf.io/tn6g4." (Fig. 1 legend, p. 2) Detailed Analysis: The study explicitly notes pre-registration of hypotheses and analysis plan before conducting data collection and analysis, satisfying the Pre-registered Protocol criterion. Therefore, criterion P is met because the study protocol and analysis plan were pre-registered before data collection.