Fostering Students' Measurement Estimation Skills in a Digital Teaching- Learning Environment: A Class-Wise Randomized Controlled Trial in Grade 5

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Classes were randomly assigned to conditions, satisfying a class-level RCT.
    • "The trial used a class-wise randomized assignment of classes to treatment conditions, not a student-wise randomized individual assignment." (p. 11)
    • Relevant Quotes: 1) "A class-wise randomized controlled trial was conducted with 14 classes, with a total intervention sample of n = 310 students who completed the pretest and posttest." (p. 7) 2) "classes were randomly assigned" (p. 7) 3) "The trial used a class-wise randomized assignment of classes to treatment conditions, not a student-wise randomized individual assignment." (p. 11) Detailed Analysis: The ERCT criterion C requires randomization at the class (or stronger) level. The paper states that the trial was a class-wise randomized controlled trial and that classes were randomly assigned to treatment conditions, explicitly contrasting this with student-wise assignment. This confirms intact classes were the unit of randomization. Final sentence explaining if criterion C is met because entire classes were randomly assigned to conditions.
  • E

    Exam-based Assessment

    • The outcome measures were study-specific tests rather than standardized exams.
    • "A 17-minute test of estimation skills with 28 items was conducted as a pretest and posttest..." (p. 9)
    • Relevant Quotes: 1) "A 17-minute test of estimation skills with 28 items was conducted as a pretest and posttest..." (p. 9) 2) "we developed a rating system for mathematical correctness and the discursive elaboration of the argument" (p. 10) 3) "the test adopted 12 items from other studies on length estimation" (p. 10) Detailed Analysis: ERCT criterion E requires standardized, widely recognized exam-based assessments for the primary outcomes. Here, the primary outcome is estimation skills measured via a bespoke pretest-posttest instrument. Even though some items were adopted from prior research, the test as a whole is not described as a standardized exam, and the authors explicitly developed their own rating system for explanations. Therefore, the assessment does not meet the ERCT requirement for standardized exams. Final sentence explaining if criterion E is not met because the outcomes were measured with study-built instruments rather than standardized exams.
  • T

    Term Duration

    • Outcomes were measured in a short pre-post design around a 90-minute intervention, not one term after the start.
    • "The teaching-learning environment encompassed two 45-minute sessions..." (p. 1)
    • Relevant Quotes: 1) "The teaching-learning environment encompassed two 45-minute sessions..." (p. 1) 2) "A class-wise randomized controlled trial was conducted with 14 classes, with a total intervention sample of n = 310 students who completed the pretest and posttest." (p. 7) 3) "The short time of the intervention (restricted to 90 min) does not allow for capturing long-term learning gains" (p. 21) Detailed Analysis: ERCT criterion T requires that outcomes are measured at least one full academic term after the intervention begins. The intervention is described as two 45-minute sessions, and the study design is a pretest-posttest trial. The discussion explicitly notes the short 90-minute duration and the lack of long-term measurement. There is no indication of term-long follow-up from intervention start to the primary outcome measurement. Final sentence explaining if criterion T is not met because measurement was not conducted one term after intervention start.
  • D

    Documented Control Group

    • The control group is described and the paper provides group composition and background characteristics.
    • "The control group, CG, also worked in a digital teaching-learning environment, but on another aspect of mass units, namely understanding mass unit conversions." (p. 8)
    • Relevant Quotes: 1) "The control group, CG, also worked in a digital teaching-learning environment, but on another aspect of mass units, namely understanding mass unit conversions." (p. 8) 2) "Three treatment conditions were compared" (p. 8) 3) "Table 2. Descriptive characteristics of the intervention sample and the three treatment groups." (p. 12) Detailed Analysis: ERCT criterion D requires a well-documented control group, including what the control group did and who was in it. The paper describes the control group's activity (a digital environment focused on mass unit conversions) and explains the rationale for the active control. It also provides a table of descriptive characteristics for the intervention sample and each treatment group, supporting comparability checks. Final sentence explaining if criterion D is met because the control group condition and sample characteristics are documented in the paper.

• Level 2 Criteria

  • S

    School-level RCT

    • Randomization occurred at the class level, not by random assignment of schools.
    • "The trial used a class-wise randomized assignment of classes to treatment conditions..." (p. 11)
    • Relevant Quotes: 1) "A class-wise randomized controlled trial was conducted with 14 classes" (p. 7) 2) "The trial used a class-wise randomized assignment of classes to treatment conditions, not a student-wise randomized individual assignment." (p. 11) Detailed Analysis: ERCT criterion S requires school-level randomization. The paper repeatedly describes the trial as class-wise randomized and specifies that classes were the randomized units. No evidence is provided that whole schools were randomized to conditions. Final sentence explaining if criterion S is not met because the unit of randomization was classes, not schools.
  • I

    Independent Conduct

    • The intervention designers and university-affiliated staff were involved in implementation and study conduct.
    • "design of learning environment, N.P. and S.P.; data gathering and data curation, N.P." (p. 22)
    • Relevant Quotes: 1) "The classes of the intervention and control groups were taught by carefully prepared research-near teachers (from the university group)" (p. 9) 2) "design of learning environment, N.P. and S.P.; data gathering and data curation, N.P." (p. 22) 3) "formal analysis, S.P., N.P. and J.W." (p. 22) Detailed Analysis: ERCT criterion I requires independent conduct by evaluators separate from the intervention designers. The author contribution statement indicates that authors who designed the learning environment also participated in data gathering and analysis. In addition, teaching was done by research- near teachers from the university group. The paper does not present an external, independent evaluation team that implemented and evaluated the intervention separately from the designers. Final sentence explaining if criterion I is not met because the designers were involved in conducting and analyzing the trial.
  • Y

    Year Duration

    • The study did not track outcomes for a full academic year, and criterion T is not met.
    • "The short time of the intervention (restricted to 90 min) does not allow for capturing long-term learning gains" (p. 21)
    • Relevant Quotes: 1) "The teaching-learning environment encompassed two 45-minute sessions..." (p. 1) 2) "A class-wise randomized controlled trial was conducted with 14 classes, with a total intervention sample of n = 310 students who completed the pretest and posttest." (p. 7) 3) "The short time of the intervention (restricted to 90 min) does not allow for capturing long-term learning gains" (p. 21) Detailed Analysis: ERCT criterion Y requires outcome tracking at least one full academic year after intervention start. This study is a short pretest-posttest design around a 90-minute intervention, with no evidence of year-long follow-up. Additionally, per ERCT rule, if criterion T is not met then criterion Y is not met. Final sentence explaining if criterion Y is not met because there is no year-long follow-up after intervention start.
  • B

    Balanced Control Group

    • The intervention did not add extra time or budget relative to control; all groups used comparable digital environments.
    • "classes in all three conditions worked in a digital teaching-learning environment on mass units" (p. 8)
    • Relevant Quotes: 1) "classes in all three conditions worked in a digital teaching-learning environment on mass units" (p. 8) 2) "The teaching-learning environment encompassed two 45-minute sessions" (p. 1) 3) "CG control group with intervention on other topic, converting mass units (3 sessions of 45 minutes each)" (p. 7, Figure 3) 4) "IG-HS intervention on estimation skills with highly structured scaffolds (2 sessions of 45 minutes each)" (p. 7, Figure 3) Detailed Analysis: Criterion B asks whether extra time, budget, or resources are provided to the intervention condition without a comparable substitute in control, unless extra resources are explicitly the treatment variable. Here, all groups used a digital teaching-learning environment on mass units, keeping the platform context comparable. The intervention itself is short (two 45-minute sessions), and the control group is described as receiving an active digital alternative and even slightly more session time (three 45-minute sessions). Therefore, there is no unbalanced increase of time or resources favoring the intervention. Final sentence explaining if criterion B is met because the intervention does not provide extra resources relative to control, and the control has a comparable digital alternative activity.

• Level 3 Criteria

  • R

    Reproduced

    • No independent replication study was identified in the paper or in an external literature search.
    • Relevant Quotes: 1) "In this paper, we present a digital teaching-learning environment" (p. 1) Detailed Analysis: ERCT criterion R requires an independent replication by other authors in a different context, published in a peer-reviewed venue. The paper presents an original efficacy study and does not cite an independent replication. An external search (by DOI, title, and author names) did not identify a peer-reviewed independent replication of this specific intervention study as of the ERCT check date. Final sentence explaining if criterion R is not met because no independent replication was found.
  • A

    All-subject Exams

    • Outcomes were not assessed with standardized exams across all main subjects, and criterion E is not met.
    • "classes were randomly assigned, using estimation skills as the dependent variable." (p. 7)
    • Relevant Quotes: 1) "classes were randomly assigned, using estimation skills as the dependent variable." (p. 7) 2) "A 17-minute test of estimation skills with 28 items was conducted as a pretest and posttest..." (p. 9) Detailed Analysis: ERCT criterion A requires standardized exam-based assessment across all main subjects. This study measures a single domain outcome (estimation skills) via a study-specific test. Additionally, per ERCT rule, if criterion E is not met then criterion A is not met. Final sentence explaining if criterion A is not met because the study did not use standardized all-subject exams and measured only estimation skills.
  • G

    Graduation Tracking

    • There is no evidence of tracking participants to graduation, and criterion Y is not met.
    • "The short time of the intervention (restricted to 90 min) does not allow for capturing long-term learning gains" (p. 21)
    • Relevant Quotes: 1) "The short time of the intervention (restricted to 90 min) does not allow for capturing long-term learning gains" (p. 21) Detailed Analysis: ERCT criterion G requires follow-up tracking until graduation, potentially in later follow-up papers by the same authors. The paper explicitly frames the intervention as short-term and notes it cannot capture long-term gains. An external search (by DOI, title, and author names) did not identify a follow-up publication tracking this cohort to graduation as of the ERCT check date. Per ERCT rule, if criterion Y is not met then criterion G is not met. Final sentence explaining if criterion G is not met because there is no graduation tracking evidence and year-long tracking is absent.
  • P

    Pre-Registered

    • The paper does not report pre-registration, and no matching registry record was identified in an external search.
    • Relevant Quotes: 1) "This study was conducted in full accordance with the Declaration of Helsinki of 1975" (p. 22) 2) "Written informed consent was obtained" (p. 22) Detailed Analysis: ERCT criterion P requires a publicly accessible pre-registered protocol with registration dated before data collection starts. The paper reports ethics and consent information but does not provide a pre-registration identifier or registry link. An external search (by DOI, title, and author names) did not identify a public pre-registration record for this trial as of the ERCT check date. Final sentence explaining if criterion P is not met because no pre- registration statement or registry record was found.