The effects of parental involvement in homework: two randomised controlled trials in financial education

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomisation occurred at the school level, which satisfies the class‑level RCT requirement.
    • "Randomisation was done at school level in order to avoid contamination at class or teacher level." (p. 6)
    • Relevant Quotes: 1) "Randomisation was done at school level in order to avoid contamination at class or teacher level." (p. 6) 2) "Registered schools were randomised in four experimental groups that is one control group and three treatment groups." (p. 6) Detailed Analysis: The paper explicitly describes randomisation at the **school** level, which is stronger than the required class‑level RCT. There is no indication of student‑level randomisation within classes, and no tutoring exception applies. Final sentence explaining if criterion C is met or not because the unit of randomisation was the school, fulfilling the class‑level requirement.
  • E

    Exam-based Assessment

    • The authors used a bespoke nine‑item quiz rather than a recognised standardised exam.
    • "Financial literacy was assessed on nine multiple choice questions about the course material …" (p. 6)
    • Relevant Quotes: 1) "Financial literacy was assessed on nine multiple choice questions about the course material, comprising the two dimensions of knowledge and behaviour." (p. 6) 2) "Six of the nine financial literacy items measured financial knowledge … The other three items captured behavioural aspects …" (p. 6) Detailed Analysis: The study employs a custom nine‑item multiple‑choice instrument developed by the authors. It is not a widely recognised standardised exam, but a bespoke assessment aligned to the intervention. Final sentence explaining if criterion E is met or not because only a custom test was used, not a standardised exam.
  • T

    Term Duration

    • The intervention lasted only a few hours over several weeks, not a full term.
    • "80% of students received the classes in four blocks of one hour each …" (p. 6)
    • Relevant Quotes: 1) "Teachers could choose to teach the four class periods separately or in one block. 80% of students received the classes in four blocks of one hour each …" (p. 6) 2) "The post-test was completed 6.5 weeks after the pretest; the second post‐test seven weeks later." (p. 7) Detailed Analysis: The core instructional intervention spans only four one‑hour sessions and a few weeks of follow‑up, far shorter than an academic term (3–4 months). Final sentence explaining if criterion T is met or not because the intervention lasted only a few hours over weeks, not a full term.
  • D

    Documented Control Group

    • The control group’s composition, baseline performance, and lack of intervention are clearly documented.
    • "Students in the control group only completed the day 0 and day 7 pre‑ and posttests and did not receive any learning interventions." (p. 6)
    • Relevant Quotes: 1) "Students in the control group only completed the day 0 and day 7 pre‑ and posttests and did not receive any learning interventions. These students provided a baseline comparison …" (p. 6) 2) "Table 1 shows pretest and posttest scores and student demographics for the control group (n = 850)." (p. 8) Detailed Analysis: The control group’s size, baseline measures, and lack of intervention are clearly documented in text and Table 1. Final sentence explaining if criterion D is met or not because the control group is fully described.

• Level 2 Criteria

  • S

    School-level RCT

    • Randomisation at the school level fulfills the school‑level RCT criterion.
    • "Randomisation was done at school level in order to avoid contamination at class or teacher level." (p. 6)
    • Relevant Quotes: 1) "Randomisation was done at school level in order to avoid contamination at class or teacher level." (p. 6) 2) "Registered schools were randomised in four experimental groups …" (p. 6) Detailed Analysis: The randomisation occurred among **schools**, not just classes, satisfying the school‑level RCT requirement. Final sentence explaining if criterion S is met or not because the study randomised whole schools.
  • I

    Independent Conduct

    • The intervention was designed, delivered, and assessed by the authors’ team with no third‑party evaluation.
    • "The game was developed by senior teachers in the project team …" (pp. 2–3)
    • Relevant Quotes: 1) "The game was developed by senior teachers in the project team and the online quiz by the authors; no third‑party evaluator is mentioned." (pp. 2–3) 2) "A team of three trained PhD students visited classes … confirmed standardised implementation, but no independent external evaluator." (p. 5) Detailed Analysis: All design, delivery, and evaluation were conducted by the authors or their team; no independent party ran the trial. Final sentence explaining if criterion I is met or not because no external evaluator was involved.
  • Y

    Year Duration

    • The experiment ran under one academic year, so year‐long criterion is unmet.
    • "Wave 1: February–mid‑April 2018; Wave 2: mid‑April–June 2018." (Fig. 1)
    • Relevant Quotes: 1) "First wave: February–mid‑April 2018; second wave: mid‑April–June 2018." (Fig. 1 timeline) Detailed Analysis: The study spans under five months across two waves but does not cover a full academic year (≈9 months). Final sentence explaining if criterion Y is met or not because the duration is under one year.
  • B

    Balanced Resources

    • The control group received no equivalent time or resources, so control was not balanced.
    • "Students in the control group only completed the day 0 and day 7 pre‑ and posttests and did not receive any learning interventions." (p. 6)
    • Relevant Quotes: 1) "The classroom intervention … and the homework groups … The control group did not receive any learning interventions." (p. 6) 2) "Students in the control group only completed tests; no extra time or materials were provided." (p. 6) Detailed Analysis: The treatment groups received four hours of instruction plus homework; the control group received none. No resource balance or “business‑as‑usual” matching is provided. Final sentence explaining if criterion B is met or not because resources were unbalanced.

• Level 3 Criteria

  • R

    Reproduced Results

    • No independent replication of this intervention has been reported.
    • Relevant Quotes: None found in the paper indicating any independent replication. Detailed Analysis: There is no mention of a subsequent study by an external team replicating these trials in a different context. Final sentence explaining if criterion R is met or not because no replication is reported.
  • A

    All Exams

    • Only financial literacy was assessed, so all‐subject exam criterion is unmet.
    • "Financial literacy was assessed on nine multiple choice questions …" (p. 6)
    • Relevant Quotes: 1) "Financial literacy was assessed on nine multiple choice questions …" (p. 6) Detailed Analysis: Only financial literacy (knowledge and behaviour) was measured; no core subjects (math, science, language) were assessed. Final sentence explaining if criterion A is met or not because only one subject was tested.
  • G

    Graduation Tracking

    • The study tracked outcomes only up to seven weeks post‑intervention, not through graduation.
    • "Two weeks after the end of the intervention period … second post‑test …" (p. 7)
    • Relevant Quotes: 1) "The control group received the post‑test six weeks after the pretest; treatment groups also had a second post‑test seven weeks later." (p. 7) Detailed Analysis: Follow‑up ends seven weeks post‑intervention; no graduation tracking. Final sentence explaining if criterion G is met or not because no long‑term tracking occurred.
  • P

    Pre-Registered Protocol

    • The trial was pre‑registered in the AEA RCT Registry before data collection.
    • "This study is registered in the AEA RCT Registry under the unique identifier AEARCTR‑0003481." (abstract)
    • Relevant Quotes: 1) "This study is registered in the AEA RCT Registry under the unique identifier AEARCTR‑0003481." (abstract) 2) "Initial registration date: October 23, 2018." (AEA registry) Detailed Analysis: The paper cites registration in a recognised RCT registry, implying pre‑registration of protocol prior to data collection. Final sentence explaining if criterion P is met or not because the study was pre‑registered.