A randomized controlled trial of project-based learning for middle-school financial literacy

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • The unit of randomization was the intact class section, which satisfies the class-level RCT requirement.
    • "Two intact class sections were randomly assigned" (p. 3)
    • Relevant Quotes: 1) "This class-randomized, posttest-only trial (two intact sections)" (p. 1) 2) "Two intact class sections were randomly assigned" (p. 3) 3) "Classes were randomly assigned to one of two conditions:" (p. 3) 4) "Because randomization occurred at the class level with two intact sections (k = 2)" (p. 5) Detailed Analysis: Criterion C requires that randomization occurs at the class level (or stronger), to reduce contamination between treatment and control students within the same classroom. The paper explicitly characterizes the design as "class-randomized" and describes assignment using "two intact class sections" as the unit of randomization. It further reiterates that "randomization occurred at the class level" (k = 2), confirming that the experimental unit was the class section, not individual students within a single class. Final Summary: Criterion C is met because intact class sections were the unit of randomization.
  • E

    Exam-based Assessment

    • The outcome measure was a researcher-created posttest rather than a standardized, widely recognized exam.
    • "The full measure was created by the research team" (p. 4)
    • Relevant Quotes: 1) "Students completed a 12-item posttest in Microsoft Forms" (p. 3) 2) "Open-ended responses (five items) were scored using a rubric developed by the research team" (p. 3) 3) "The full measure was created by the research team" (p. 4) Detailed Analysis: Criterion E requires a standardized, widely recognized exam-based assessment (e.g., state/national standardized achievement tests), not a bespoke instrument created for the study. The paper describes a "12-item posttest" administered via Microsoft Forms and explicitly states that the "full measure was created by the research team." The scoring rubric for open-ended items was also "developed by the research team." These statements confirm that the assessment was custom-built for the study rather than a standardized exam. Final Summary: Criterion E is not met because the primary assessment was a researcher-created posttest rather than a standardized exam.
  • T

    Term Duration

    • Outcomes were measured immediately after a three-session unit, which is shorter than one academic term from start to measurement.
    • "The intervention took place over three 40-min class periods" (p. 3)
    • Relevant Quotes: 1) "The intervention took place over three 40-min class periods" (p. 3) 2) "posttest-only trial" (p. 1) 3) "Students completed a 12-item posttest in Microsoft Forms" (p. 3) Detailed Analysis: Criterion T requires that outcomes be measured at least one full academic term (about 3 to 4 months) after the intervention begins, even if the intervention itself is shorter. The intervention is described as occurring over "three 40-min class periods" and the study is described as "posttest-only," with the outcome measured via a posttest. No term-lagged follow-up assessment is described; the measurement is consistent with an immediate posttest after the short unit. Final Summary: Criterion T is not met because the start-to-measurement interval is far shorter than one academic term.
  • D

    Documented Control Group

    • Although the control activities are described, the posttest-only design provides no baseline performance data documenting control group comparability.
    • "no prior budgeting knowledge was assumed or measured" (p. 3)
    • Relevant Quotes: 1) "posttest-only trial" (p. 1) 2) "In the Standards-Based Learning condition, students were taught through recorded PowerPoint-style lectures" (p. 3) 3) "no prior budgeting knowledge was assumed or measured" (p. 3) Detailed Analysis: Criterion D requires a well-documented control group, including baseline performance information (or equivalent documentation of starting comparability) and a clear description of what the control group received. The paper does describe what the control group received (Standards- Based Learning instruction delivered via recorded lectures and practice activities). However, the paper also describes the design as "posttest-only" and explicitly states that no prior budgeting knowledge was measured. In this design, the absence of baseline performance documentation means the control group is not sufficiently documented for comparability on the outcome domain under ERCT D. Final Summary: Criterion D is not met because baseline performance information for the control group is not provided in this posttest-only study.

• Level 2 Criteria

  • S

    School-level RCT

    • The trial randomized two classes within one school rather than randomizing across schools.
    • "from the same public middle school in the United States" (p. 3)
    • Relevant Quotes: 1) "from the same public middle school in the United States" (p. 3) 2) "Two science classes at the school participated" (p. 3) 3) "the 3rd-period and the 7th-period classes" (p. 3) Detailed Analysis: Criterion S requires school-level randomization, meaning schools (or equivalent sites) are assigned to treatment versus control. The paper states all participants came from the same public middle school and that two science classes (two periods) participated. This is class-level assignment within a single school, not a multi-school, school-randomized design. Final Summary: Criterion S is not met because randomization occurred within a single school rather than across schools.
  • I

    Independent Conduct

    • The paper does not document an independent evaluation team and states the author conducted all study components themselves.
    • "were conducted independently by me" (p. 8)
    • Relevant Quotes: 1) "The full measure was created by the research team" (p. 4) 2) "were conducted independently by me" (p. 8) Detailed Analysis: Criterion I requires that the evaluation be conducted independently from the intervention designers/providers to reduce bias in implementation, measurement, analysis, and reporting. The paper indicates that key components (the measure) were created by the research team and includes a statement that the core study activities "were conducted independently by me," indicating the authors themselves conducted the study rather than an external, independent evaluator. The paper does not describe any third-party evaluation organization running implementation, data collection, or analysis. Final Summary: Criterion I is not met because the evaluation is not documented as independent from the authors who conducted the study.
  • Y

    Year Duration

    • Outcomes were measured after a three-session unit and the authors state long-term outcomes were not measured, failing the 75%-of- year tracking requirement.
    • "we did not measure long-term outcomes" (p. 6)
    • Relevant Quotes: 1) "The intervention took place over three 40-min class periods" (p. 3) 2) "we did not measure long-term outcomes" (p. 6) Detailed Analysis: Criterion Y requires outcome measurement at least 75% of an academic year after the intervention begins. The intervention is explicitly only three class periods, and the paper acknowledges that long-term outcomes were not measured. This is incompatible with year-long (or near-year-long) tracking. Additionally, per the ERCT dependency rule, if Criterion T is not met then Criterion Y is not met; here, both the explicit duration and the stated lack of long-term outcomes confirm failure. Final Summary: Criterion Y is not met because the study did not track outcomes for anything close to an academic year.
  • B

    Balanced Control Group

    • Both conditions used the same scheduled class time and the same delivery platform, with no evidence of extra real resources given only to the intervention group.
    • "Instruction in both conditions was delivered through Nearpod" (p. 3)
    • Relevant Quotes: 1) "The intervention took place over three 40-min class periods" (p. 3) 2) "Instruction in both conditions was delivered through Nearpod" (p. 3) 3) "students completed a simulation to design a class trip" (p. 3) 4) "students were taught through recorded PowerPoint-style lectures" (p. 3) Detailed Analysis: Criterion B evaluates whether the intervention condition received additional time, budget, staffing, or materials that the control did not receive, unless that additional resourcing is explicitly the treatment variable being tested. Both groups received instruction during the same scheduled class time ("three 40-min class periods"). The paper also states that instruction in both conditions used the same digital delivery platform (Nearpod) with pre-scripted materials. The main difference is pedagogical structure (simulation/project activity versus lectures/practice), not added instructional time or added real-world financial/material inputs. The "fixed USD 1,000 budget" is described as part of the in-class simulation scenario, not as additional funding/resources provided to students or the classroom. Final Summary: Criterion B is met because there is no evidence the intervention group received extra time or real resources beyond the instructional approach itself.

• Level 3 Criteria

  • R

    Reproduced

    • No independent replication by other authors was identified, and the paper frames this study type as novel in this domain.
    • "To date, no known study has conducted a randomized controlled trial" (p. 2)
    • Relevant Quotes: 1) "To date, no known study has conducted a randomized controlled trial" (p. 2) 2) "few studies have examined Project-Based Learning" (p. 2) Detailed Analysis: Criterion R requires that an independent research team reproduces the study (or its central experimental claim) in a different context and publishes the replication in a peer-reviewed outlet. The paper positions this RCT as novel in its domain, stating that no known prior study has conducted such an RCT and that few studies have examined this type of design. Additionally, a targeted web search performed on 2026-03-13 did not identify any peer-reviewed, independent replication of this specific two-class Nearpod-delivered budgeting unit RCT (and given the publication date of 2026-02-05, independent replication would be unlikely to have appeared already). Final Summary: Criterion R is not met because no independent replication evidence was found.
  • A

    All-subject Exams

    • The study does not use standardized exams (Criterion E not met), so it cannot satisfy the all-subject standardized exam requirement.
    • "Students completed a 12-item posttest in Microsoft Forms" (p. 3)
    • Relevant Quotes: 1) "Students completed a 12-item posttest in Microsoft Forms" (p. 3) 2) "The full measure was created by the research team" (p. 4) Detailed Analysis: Criterion A requires standardized exam-based measurement across all main subjects. ERCT also specifies that if Criterion E is not met, Criterion A is not met. The paper documents a researcher-created posttest (and explicitly that the measure was created by the research team), which is not a standardized exam. Therefore, regardless of which subjects were taught/tested, the study cannot meet the standardized all-subject exam requirement. Final Summary: Criterion A is not met because standardized exam-based assessment is absent (Criterion E not met).
  • G

    Graduation Tracking

    • The study does not include graduation tracking and also fails the year-duration prerequisite (Criterion Y).
    • "we did not measure long-term outcomes" (p. 6)
    • Relevant Quotes: 1) "we did not measure long-term outcomes" (p. 6) Detailed Analysis: Criterion G requires tracking participants until graduation, and ERCT specifies that if Criterion Y is not met then Criterion G is not met. The paper explicitly states that long-term outcomes were not measured, which is inconsistent with graduation tracking. A web search performed on 2026-03-13 also did not identify subsequent follow-up publications by these authors reporting tracking of this cohort through graduation. Final Summary: Criterion G is not met because the study does not track students to graduation (and Criterion Y is not met).
  • P

    Pre-Registered

    • The paper provides no protocol pre-registration ID or link, and no registry record could be verified from the information provided.
    • Relevant Quotes: (No relevant quotes found about pre-registration, registry IDs, or registration dates.) Detailed Analysis: Criterion P requires explicit evidence of pre-registration (a named registry plus an ID/link) and confirmation that registration occurred before data collection began. The paper includes ethics and disclosure information, but it does not provide a registry name/ID/link or any registration date. A web search performed on 2026-03-13 using the paper title, DOI, and author names did not locate a corresponding publicly registered protocol that could be confidently matched to this study. Final Summary: Criterion P is not met because no verifiable pre-registered protocol is reported or identifiable for this study.