Problem posing and motivation: the effects of posing and solving one’s own modelling problems on autonomy, competence, relatedness, and self-efficacy

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomization was at the student (within-school) level rather than at the class (or school) level, and the intervention is not one-to-one tutoring.
    • "The study used an experimental design in which students were randomly assigned to one of two intervention conditions: “Posing and Solving One’s Own Modelling Problems” (PSOM) or “Solving Given Modelling Problems” (SGM)." (p. 7)
    • Relevant Quotes: 1) "The study used an experimental design in which students were randomly assigned to one of two intervention conditions: “Posing and Solving One’s Own Modelling Problems” (PSOM) or “Solving Given Modelling Problems” (SGM)." (p. 7) 2) "From the first school, six new groups were formed (three per condition); and from the second school, four groups were formed (two per condition), resulting in 10 intervention groups—five in the PSOM condition and five in the SGM condition." (p. 7) Detailed Analysis: Criterion C requires that randomization occur at the class level (or a stronger unit, such as schools), to reduce contamination that can occur when students in the same class/school environment receive different conditions. The paper explicitly states that "students were randomly assigned" to the two conditions. It also describes that new groups were formed from within participating schools/classes, which is consistent with student-level assignment into groups rather than whole classes being assigned as intact units. The ERCT exception allowing student-level randomization applies to one-to-one tutoring/personal teaching interventions. This intervention is delivered in small-group/classroom instructional settings (pair work and whole-class discussions are described elsewhere), not as individual tutoring. Criterion C is not met because students (not whole classes or schools) were randomized and no tutoring-style exception applies.
  • E

    Exam-based Assessment

    • Outcomes were measured via Likert-scale questionnaires rather than standardized exam-based assessments.
    • "Self-efficacy in mathematics was measured before and after the intervention using four items, also rated on a 5-point Likert scale." (p. 8)
    • Relevant Quotes: 1) "Experiences of autonomy, competence, and relatedness were measured using validated scales adapted from Schukajlow and Krug (2014) and Wiehe et al. (2025)." (p. 8) 2) "Each scale consisted of three items rated on 5-point Likert scales ranging from 1 (not at all true) to 5 (completely true)." (p. 8) 3) "Self-efficacy in mathematics was measured before and after the intervention using four items, also rated on a 5-point Likert scale." (p. 8) Detailed Analysis: Criterion E requires standardized exam-based achievement assessments (for example, national/state standardized tests or widely recognized standardized achievement measures). This paper’s outcomes are motivational and self-belief constructs measured through questionnaire items on Likert scales. Although the scales are described as validated, they are not exam-based assessments of academic achievement and are not described as standardized external mathematics exams or standardized achievement tests. Criterion E is not met because the study uses Likert-scale questionnaires rather than standardized exam-based outcome measures.
  • T

    Term Duration

    • The intervention lasted four 45-minute lessons with a posttest immediately after the intervention, which is far shorter than an academic term.
    • "Each lesson lasted 45 min and took place on a single day." (p. 7)
    • Relevant Quotes: 1) "Following a pre-intervention questionnaire on self-efficacy in mathematics, students in the PSOM condition received a four-lesson intervention on posing and solving their own modelling problems, whereas the SGM condition received a parallelized version of the intervention focusing on solving given modelling problems." (p. 7) 2) "Each lesson lasted 45 min and took place on a single day." (p. 7) 3) "A posttest on self-efficacy was administered after the intervention." (p. 7) Detailed Analysis: Criterion T requires that outcomes be measured at least one full academic term after the intervention begins (typically ~3–4 months, or a clearly documented term-equivalent timeframe). The intervention is described as a "four-lesson intervention" with lessons lasting 45 minutes. The posttest was administered "after the intervention," indicating immediate post-measurement rather than term-delayed follow-up. Criterion T is not met because the start-to-measurement interval is only a short sequence of lessons rather than at least one academic term.
  • D

    Documented Control Group

    • The control condition (SGM) is clearly described, and baseline outcome data by condition are reported, enabling treatment-control comparison.
    • "In the SGM condition, students engaged in: Lessons 1 & 2: Solving problems... Lessons 3 & 4: Solving problems..." (p. 8)
    • Relevant Quotes: 1) "The study used an experimental design in which students were randomly assigned to one of two intervention conditions: “Posing and Solving One’s Own Modelling Problems” (PSOM) or “Solving Given Modelling Problems” (SGM)." (p. 7) 2) "In the SGM condition, students engaged in: Lessons 1 & 2: Solving problems based on the Shared Apartment Situation (one per lesson) ... Lessons 3 & 4: Solving problems based on the Backpack Situation (one per lesson) ..." (p. 8) 3) "Table 3 Means, standard deviations, and correlations for the study variables (Models 1a, 1b, and 1c)" (p. 9) 4) "1. Self-efficacy (pretest) 3.00 0.95 2.81 1.02" (Table 3, p. 9) Detailed Analysis: Criterion D requires a well-documented control group, including a clear description of what the control group received and evidence enabling treatment-control comparisons (such as baseline measures). The paper explicitly defines the control/comparison condition as "Solving Given Modelling Problems (SGM)" and describes the lesson activities in that condition. The paper also reports baseline (pretest) self-efficacy by condition in Table 3, supporting comparability checks for a key measured construct. While demographic breakdowns by condition are not prominently presented in the quoted excerpts, the control instruction itself is described in sufficient detail and baseline outcome data by condition are provided. Criterion D is met because the control condition is described and baseline outcome values by condition are reported.

• Level 2 Criteria

  • S

    School-level RCT

    • Schools were not randomized; instead, students within participating schools/classes were randomly assigned to conditions.
    • "The study used an experimental design in which students were randomly assigned to one of two intervention conditions..." (p. 7)
    • Relevant Quotes: 1) "The sample included N = 194 tenth-grade students ... from eight classes across two German middle track schools..." (p. 6) 2) "The study used an experimental design in which students were randomly assigned to one of two intervention conditions: “Posing and Solving One’s Own Modelling Problems” (PSOM) or “Solving Given Modelling Problems” (SGM)." (p. 7) Detailed Analysis: Criterion S requires randomization at the school (site) level, meaning entire schools are assigned to treatment versus control. Although two schools participated, the paper states that "students were randomly assigned" to conditions. There is no indication that the schools themselves were randomized to PSOM versus SGM. Criterion S is not met because assignment was not conducted at the school level.
  • I

    Independent Conduct

    • The study was implemented and evaluated within the project team, with no clearly independent third-party evaluator for conduct and analysis.
    • "The classes were taught by six master’s students who participated in the project as part of their theses." (p. 7)
    • Relevant Quotes: 1) "The classes were taught by six master’s students who participated in the project as part of their theses." (p. 7) 2) "Fidelity of implementation was monitored through observation protocols completed by non-teaching master’s students." (p. 7) 3) "Author Contribution Janina Krawitz: Conceptualization, Methodology, Investigation, Writing—original draft preparation, Formal analysis, Funding acquisition; ..." (p. 15) Detailed Analysis: Criterion I requires that the evaluation be conducted independently from those who designed the intervention, to reduce bias in implementation, measurement, and analysis (e.g., an external evaluation team). Here, teaching was conducted by master’s students participating in the project, and fidelity monitoring was also performed by other master’s students. The author contribution statement indicates that core scientific tasks (including investigation and formal analysis) were conducted by the author team. The paper does not describe an independent external evaluator organization/team that was clearly separate from the intervention design and study leadership. Criterion I is not met because independent third-party conduct of the evaluation is not documented.
  • Y

    Year Duration

    • The intervention and measurement occurred over a very short period, not over at least 75% of an academic year.
    • "A posttest on self-efficacy was administered after the intervention." (p. 7)
    • Relevant Quotes: 1) "Each lesson lasted 45 min and took place on a single day." (p. 7) 2) "A posttest on self-efficacy was administered after the intervention." (p. 7) Detailed Analysis: Criterion Y requires that outcomes be measured at least 75% of one academic year after the intervention begins. The intervention is a short four-lesson sequence, and measurement is described as occurring immediately after the intervention via a posttest. There is no indication of follow-up measurement months later, let alone across most of an academic year. Criterion Y is not met because outcomes were not tracked across (most of) an academic year after the intervention began.
  • B

    Balanced Control Group

    • The two conditions are described as parallelized and appear to match in lesson time and materials; the difference is the instructional activity, not added time/budget.
    • "Teaching methods (pair work, whole-class discussions) X X" (Table 1, p. 7)
    • Relevant Quotes: 1) "Following a pre-intervention questionnaire on self-efficacy in mathematics, students in the PSOM condition received a four-lesson intervention on posing and solving their own modelling problems, whereas the SGM condition received a parallelized version of the intervention focusing on solving given modelling problems." (p. 7) 2) "Each lesson lasted 45 min and took place on a single day." (p. 7) 3) "Teaching methods (pair work, whole-class discussions) X X" (Table 1, p. 7) 4) "Both conditions used the same two real-world situations: the Shared Apartment Situation ... and the Backpack Situation ..." (p. 7) Detailed Analysis: Criterion B compares the nature, quantity, and quality of resources (time, materials, adult support) provided to intervention and control conditions. If the intervention adds time/budget beyond control, the control should receive comparable inputs unless the added resources are explicitly the treatment variable. The paper states the SGM condition received a "parallelized version" of the PSOM intervention, and both conditions have the same number of lessons and the same lesson duration ("Each lesson lasted 45 min"). Table 1 shows both conditions use the same overarching teaching methods (pair work and whole-class discussions). The paper also states both conditions used the same two real-world situations. The core difference between conditions is the activity (posing and solving one’s own problems vs solving given problems), not additional instructional time or distinct material resources. There is no indication of added budget/time being provided exclusively to the PSOM group. Criterion B is met because instructional time and stated resources appear balanced across treatment and control, with differences reflecting the intended pedagogical contrast.

• Level 3 Criteria

  • R

    Reproduced

    • No independent replication of this specific experimental study was found in the paper or via web search as of 2026-03-03.
    • Relevant Quotes: (No relevant quotes about independent replication of this specific study were found in the paper.) Detailed Analysis: Criterion R requires that the study be independently replicated by a different research team, in a different context, and published in a peer-reviewed outlet; such evidence may appear after the original publication. The paper does not present itself as a replication and does not report an independent replication of the same PSOM vs SGM experimental design. An internet search (by DOI, full title, and key condition labels such as "PSOM" and "SGM") did not identify any published independent replication of this specific study as of the ERCT check date (2026-03-03). Criterion R is not met because independent replication evidence for this specific study was not found.
  • A

    All-subject Exams

    • The study does not use standardized exam outcomes (and thus fails Criterion E), so it cannot meet the all-subject standardized exam requirement.
    • "Self-efficacy in mathematics was measured before and after the intervention using four items, also rated on a 5-point Likert scale." (p. 8)
    • Relevant Quotes: 1) "Self-efficacy in mathematics was measured before and after the intervention using four items, also rated on a 5-point Likert scale." (p. 8) Detailed Analysis: Criterion A requires standardized exam-based assessments across all main subjects and, per the ERCT rule, cannot be met if Criterion E is not met. This study does not use standardized exam-based achievement outcomes; it measures self-efficacy and basic psychological needs via Likert-scale questionnaires. Criterion A is not met because standardized exam-based assessments across subjects are not used (and Criterion E is not met).
  • G

    Graduation Tracking

    • The study did not track students until graduation; additionally, because Criterion Y is not met, Criterion G cannot be met under the ERCT rules.
    • "A posttest on self-efficacy was administered after the intervention." (p. 7)
    • Relevant Quotes: 1) "A posttest on self-efficacy was administered after the intervention." (p. 7) 2) "However, this approach has the disadvantage that it is limited in duration." (p. 14) Detailed Analysis: Criterion G requires following participants until graduation from the relevant educational stage. Under the ERCT dependency rule, if Criterion Y (year-duration tracking) is not met, then Criterion G is not met. The paper describes only an immediate posttest "after the intervention" and explicitly notes the study is "limited in duration," which is incompatible with tracking students through graduation. A web search for subsequent follow-up publications by the same author team that tracked this cohort to graduation did not identify any such graduation-tracking follow-up as of 2026-03-03. Criterion G is not met because the study does not include (and no follow-up evidence was found for) tracking participants until graduation, and the prerequisite year-duration criterion is not met.
  • P

    Pre-Registered

    • No pre-registration statement or registry ID/date was found in the paper, and no corresponding public pre-registration record was identified online.
    • Relevant Quotes: (No relevant quotes indicating pre-registration, a registry name/ID, or a registration date were found in the paper.) Detailed Analysis: Criterion P requires a publicly pre-registered protocol and evidence that registration occurred before data collection began. Searching the paper for common indicators of pre-registration (e.g., "pre-registered," "registered," "OSF," "protocol") did not yield any pre-registration statement or registry identifier. An internet search using the paper title, DOI, and author names did not identify a public pre-registration record that could be confidently linked to this specific study and verified as being registered before data collection. Criterion P is not met because pre-registration was not documented in the paper and no verifiable public pre-registration record was found.