Problem posing and motivation: the effects of posing and solving one’s own modelling problems on autonomy, competence, relatedness, and self-efficacy

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Students (not intact classes or schools) were randomly assigned, so the trial is not a class-level (or stronger) RCT.
    • "The study used an experimental design in which students were randomly assigned to one of two intervention conditions: “Posing and Solving One’s Own Modelling Problems” (PSOM) or “Solving Given Modelling Problems” (SGM)." (p. 7)
    • Relevant Quotes: 1) "In an experimental study, 194 tenth-grade students were randomly assigned to either pose and solve their own modelling problems or solve given modelling problems." (p. 1) 2) "The study used an experimental design in which students were randomly assigned to one of two intervention conditions: “Posing and Solving One’s Own Modelling Problems” (PSOM) or “Solving Given Modelling Problems” (SGM)." (p. 7) 3) "From the first school, six new groups were formed (three per condition); and from the second school, four groups were formed (two per condition), resulting in 10 intervention groups—five in the PSOM condition and five in the SGM condition." (p. 7) Detailed Analysis: Criterion C requires randomization at the class level (or stronger), unless the intervention is clearly one-to-one tutoring/personal teaching. The paper explicitly states that students were "randomly assigned" and describes that "new groups were formed" within each school, which indicates student-level assignment into groups rather than intact classes (or whole schools) being randomized. The intervention is a classroom instructional format (problem posing and/or modelling problem solving in lessons), not one-to-one tutoring, so the tutoring exception does not apply. Final sentence: Criterion C is not met because randomization was at the student/group level rather than at the intact class (or school) level.
  • E

    Exam-based Assessment

    • Outcomes were measured with Likert-scale questionnaire items rather than a standardized exam-based assessment.
    • "Self-efficacy in mathematics was measured before and after the intervention using four items, also rated on a 5-point Likert scale." (p. 8)
    • Relevant Quotes: 1) "Each scale consisted of three items rated on 5-point Likert scales ranging from 1 (not at all true) to 5 (completely true)." (p. 8) 2) "Self-efficacy in mathematics was measured before and after the intervention using four items, also rated on a 5-point Likert scale." (p. 8) Detailed Analysis: Criterion E requires a standardized exam-based assessment (e.g., a recognized state/national standardized test), rather than study-created or study-selected self-report items. The quoted methods text shows that both the basic-needs measures and the outcome (self-efficacy in mathematics) were measured via Likert- scale questionnaire items. These are not standardized academic exams and therefore do not satisfy ERCT’s exam-based requirement. Final sentence: Criterion E is not met because the study used Likert- scale questionnaire measures, not a standardized exam.
  • T

    Term Duration

    • The intervention and outcome measurement occurred over four 45-minute lessons with an immediate posttest, not at least one academic term after the intervention began.
    • "Each lesson lasted 45 min and took place on a single day." (p. 7)
    • Relevant Quotes: 1) "Following a pre-intervention questionnaire on self-efficacy in mathematics, students in the PSOM condition received a four-lesson intervention on posing and solving their own modelling problems, whereas the SGM condition received a parallelized version of the intervention focusing on solving given modelling problems." (p. 7) 2) "Each lesson lasted 45 min and took place on a single day." (p. 7) 3) "A posttest on self-efficacy was administered after the intervention." (p. 7) Detailed Analysis: Criterion T requires that outcomes are measured at least one full academic term (about 3–4 months) after the intervention begins (or that the study otherwise tracks outcomes for at least a term after start). The study describes a four-lesson intervention and states that the posttest was administered after the intervention, implying measurement immediately after a short sequence of lessons. This is far shorter than a school term. Final sentence: Criterion T is not met because outcomes were measured immediately after a four-lesson intervention rather than at least one academic term after the intervention began.
  • D

    Documented Control Group

    • The control condition (SGM) is clearly described and baseline outcome data (pretest self-efficacy) are reported by condition.
    • "In the SGM condition, students engaged in: Lessons 1 & 2: Solving problems based on the Shared Apartment Situation (one per lesson) ... Lessons 3 & 4: Solving problems based on the Backpack Situation (one per lesson) ..." (p. 8)
    • Relevant Quotes: 1) "In the SGM condition, students engaged in: Lessons 1 & 2: Solving problems based on the Shared Apartment Situation (one per lesson) (Fig. 4 shows the first one). Comparing and discussing their solutions to the respective problem." (p. 8) 2) "Lessons 3 & 4: Solving problems based on the Backpack Situation (one per lesson): • Problem 1: “How many backpacks could be made from the ocean plastic currently floating in the sea? • Problem 2: “How would you assess the impact of the backpack company on ocean pollution? Justify your assessment with a calculation!” • Comparing and discussing their solutions to the respective problem." (p. 8) 3) "Self-efficacy (pretest) 3.00 0.95 2.81 1.02" (Table 3, p. 9) Detailed Analysis: Criterion D requires that the control group is well documented, including what it received and baseline performance information for comparison. The paper explicitly defines the control condition as the SGM condition and describes its lesson activities in detail (what students did across lessons). It also reports pretest self-efficacy means for both conditions in Table 3, providing baseline outcome information by condition. A limitation is that demographic variables are reported for the full sample rather than separately by condition, but the control condition’s instructional experience and baseline outcome level are documented sufficiently to support comparisons. Final sentence: Criterion D is met because the control condition is clearly described and baseline outcome data are reported by group.

• Level 2 Criteria

  • S

    School-level RCT

    • The study involved two schools but did not randomize at the school level; students were randomized within schools into conditions.
    • "The study used an experimental design in which students were randomly assigned to one of two intervention conditions: “Posing and Solving One’s Own Modelling Problems” (PSOM) or “Solving Given Modelling Problems” (SGM)." (p. 7)
    • Relevant Quotes: 1) "The sample included N = 194 tenth-grade students ... from eight classes across two German middle track schools: a “Realschule” ... and a “Gesamtschule” ..." (p. 6) 2) "The study used an experimental design in which students were randomly assigned to one of two intervention conditions: “Posing and Solving One’s Own Modelling Problems” (PSOM) or “Solving Given Modelling Problems” (SGM)." (p. 7) 3) "Five classes from one school and three classes from another school participated." (p. 7) Detailed Analysis: Criterion S requires randomization among schools (or equivalent institutions/sites). Although the sample spans two schools, the paper describes assignment as student randomization into two conditions, not school-level randomization. There is no statement that entire schools were assigned to PSOM versus SGM; instead, the description implies both conditions occurred within each school via multiple groups per condition. Final sentence: Criterion S is not met because randomization was not conducted at the school level.
  • I

    Independent Conduct

    • The paper does not document a clearly independent third-party evaluation; implementation and monitoring were conducted by project- affiliated master’s students.
    • "The classes were taught by six master’s students who participated in the project as part of their theses." (p. 7)
    • Relevant Quotes: 1) "The classes were taught by six master’s students who participated in the project as part of their theses." (p. 7) 2) "Fidelity of implementation was monitored through observation protocols completed by non-teaching master’s students." (p. 7) Detailed Analysis: Criterion I requires that the study be conducted independently from the intervention designers/providers (or at minimum that data collection/analysis is clearly conducted by an independent third party) to reduce risk of bias. The study describes delivery by master’s students participating in the project and fidelity monitoring by other master’s students. The text does not state that an external evaluation team (separate institution/agency) conducted implementation, data collection, or analysis, nor does it provide an explicit independence statement. Final sentence: Criterion I is not met because the paper does not provide clear evidence of independent third-party conduct.
  • Y

    Year Duration

    • Because term-duration tracking is not met (and the study is far shorter than 75% of an academic year), the year-duration criterion is not satisfied.
    • "A posttest on self-efficacy was administered after the intervention." (p. 7)
    • Relevant Quotes: 1) "Each lesson lasted 45 min and took place on a single day." (p. 7) 2) "A posttest on self-efficacy was administered after the intervention." (p. 7) Detailed Analysis: Criterion Y requires outcomes to be measured at least 75% of an academic year after the intervention begins. The intervention and measurement occurred immediately after a short four-lesson sequence, which is far less than one academic year. Additionally, per the ERCT dependency rule, if Criterion T is not met then Criterion Y is not met. Final sentence: Criterion Y is not met because outcomes were measured immediately after a short intervention, far short of a school-year timeframe.
  • B

    Balanced Control Group

    • The control condition is parallelized in instructional time and structure (same number/length of lessons and similar teaching methods), so resources are balanced.
    • "Following a pre-intervention questionnaire on self-efficacy in mathematics, students in the PSOM condition received a four-lesson intervention ... whereas the SGM condition received a parallelized version of the intervention ..." (p. 7)
    • Relevant Quotes: 1) "Following a pre-intervention questionnaire on self-efficacy in mathematics, students in the PSOM condition received a four-lesson intervention on posing and solving their own modelling problems, whereas the SGM condition received a parallelized version of the intervention focusing on solving given modelling problems." (p. 7) 2) "Each lesson lasted 45 min and took place on a single day." (p. 7) 3) "Teaching methods (pair work, whole-class discussions) X X" (Table 1, p. 7) Detailed Analysis: Criterion B compares the nature, quantity, and quality of resources (time, materials, adult support) provided to intervention and control conditions, and asks whether any extra time/budget/materials were provided only to the intervention without being matched (unless extra resources are explicitly the treatment variable). The paper indicates both conditions received the same dosage (four lessons; each 45 minutes) and explicitly describes the control as a "parallelized version" of the intervention. Table 1 also indicates shared teaching methods across both conditions. There is no indication that the PSOM group received additional instructional time, budget, or materials beyond what is integral to delivering a different instructional activity during the same lesson time. Final sentence: Criterion B is met because instructional time and general instructional structure are matched between conditions, so added resources are not a confound.

• Level 3 Criteria

  • R

    Reproduced

    • No peer-reviewed independent replication of this specific PSOM vs. SGM experiment was found in the paper or via an internet literature search.
    • Relevant Quotes: (No relevant quotes indicating independent replication were found in the paper.) Detailed Analysis: Criterion R requires an independently replicated study (different research team, different context, peer-reviewed) reproducing the same intervention claim/design. This article does not present itself as a replication and does not cite a published independent replication of this specific PSOM vs. SGM experiment. Internet literature searches conducted on 2026-03-04 (e.g., by DOI, title phrase, intervention-condition names, and author names) did not identify a peer-reviewed publication by an independent author team that explicitly reports a direct replication of this exact study. Final sentence: Criterion R is not met because no independent replication of this specific study was found.
  • A

    All-subject Exams

    • Because the study does not use standardized exam-based assessments (Criterion E is not met), the all-subject standardized-exams criterion is not met.
    • "Self-efficacy in mathematics was measured before and after the intervention using four items, also rated on a 5-point Likert scale." (p. 8)
    • Relevant Quotes: 1) "Self-efficacy in mathematics was measured before and after the intervention using four items, also rated on a 5-point Likert scale." (p. 8) Detailed Analysis: Criterion A requires effects to be measured across all main subjects using standardized exam-based assessments. Per ERCT rules, if Criterion E is not met, then Criterion A is not met. This study measures self-efficacy via Likert-scale questionnaire items, not standardized exams, and it does not assess all core academic subjects with standardized tests. Final sentence: Criterion A is not met because the study does not use standardized exams and therefore cannot provide all-subject standardized exam outcomes.
  • G

    Graduation Tracking

    • The study does not track students through graduation and (because Year Duration is not met) Graduation Tracking cannot be met.
    • "A posttest on self-efficacy was administered after the intervention." (p. 7)
    • Relevant Quotes: 1) "A posttest on self-efficacy was administered after the intervention." (p. 7) Detailed Analysis: Criterion G requires follow-up tracking through graduation (typically via longer-term follow-up measurement or administrative records). The paper describes only an immediate posttest after a short intervention and provides no description of later follow-up waves, administrative linkage, or graduation outcomes. Additionally, per the ERCT dependency rule, if Criterion Y is not met then Criterion G is not met. Separate internet searches on 2026-03-04 for follow-up publications by the same author team reporting graduation tracking for this cohort did not identify any such graduation-tracking report. Final sentence: Criterion G is not met because the study ends with an immediate posttest and provides no evidence of tracking students until graduation.
  • P

    Pre-Registered

    • The paper provides no registry link/ID or dated statement indicating pre-registration before data collection.
    • Relevant Quotes: (No relevant quotes about pre-registration were found in the paper or its electronic supplementary material.) Detailed Analysis: Criterion P requires explicit evidence that the study protocol was pre-registered before data collection began (e.g., a registry name and ID plus a registration date that precedes study start). No pre-registration statement (e.g., OSF/AsPredicted registry link/ID) appears in the paper PDF. The electronic supplementary material provides scales and additional model outputs but does not provide a pre-registration identifier. Internet searches conducted on 2026-03-04 using the DOI, title, and author names did not identify a clearly linked public pre-registration record for this specific study. Final sentence: Criterion P is not met because no pre-registration record (registry/ID and timing) is reported or clearly identifiable.