Developing dialogic teaching: genesis, process, trial

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomisation occurred at the school level with classes equally divided into intervention and control groups, satisfying the Class‑level RCT criterion.
    • “The trial organised independently by a separate team at another university (Sheffield Hallam) used a three-level clustered RCT design (pupils within classes within schools), with randomisation at school level and the classes divided equally into intervention and control groups.”
    • Relevant Quotes: 1) “The trial organised independently by a separate team at another university (Sheffield Hallam) used a three-level clustered RCT design (pupils within classes within schools), with randomisation at school level and the classes divided equally into intervention and control groups.” Detailed Analysis: This statement confirms that entire classes were allocated to treatment or control, with the randomisation process explicitly operating at the school level. Under the ERCT Standard, class‑level randomisation is satisfied when whole classes are assigned to conditions—here the design is even stronger (school‑level), which also fulfills the weaker class‑level requirement. Final sentence: Criterion C is met because the study explicitly randomized whole classes to conditions.
  • E

    Exam-based Assessment

    • The study used GL Assessment Progress Tests—standardized instruments in English, mathematics, and science—scored blind by the test publisher, fulfilling the ERCT Standard’s Exam‑based Assessment criterion.
    • “The evaluation team used as outcome measures GL Assessment Progress Tests in English, mathematics and science. … Tests were scored by GL Assessment, the company that published the tests. The scoring was blind and for each measure the raw, unstandardised score was used in the analysis.”
    • Relevant Quotes: 1) “The evaluation team used as outcome measures GL Assessment Progress Tests in English, mathematics and science.” 2) “Tests were scored by GL Assessment, the company that published the tests. The scoring was blind and for each measure the raw, unstandardised score was used in the analysis.” Detailed Analysis: GL Assessment Progress Tests are well‑established, validated instruments widely used in UK primary education. Blind scoring by the publisher and use of raw scores ensure objectivity and comparability. This satisfies the requirement for using standardized exam‑based assessments rather than bespoke measures. Final sentence: Criterion E is met because the study employed recognized, standardized exams.
  • T

    Term Duration

    • Primary outcomes were measured 20 weeks after intervention start, exceeding a single academic term.
    • “after 20 weeks students in the intervention group were two months ahead…”
    • Relevant Quotes: 1) “after 20 weeks students in the intervention group were two months ahead of their control group peers…” (p. 1) Detailed Analysis: A UK primary term is approximately 12–14 weeks. Measuring outcomes after 20 weeks therefore meets the Term Duration requirement of at least one full term. Final sentence: Criterion T is met because 20 weeks ≥ 1 term.
  • D

    Documented Control Group

    • Control cohorts are described alongside intervention cohorts, with baseline comparisons implied.
    • “combined intervention/control cohorts of nearly 5000 year 5 (4th grade) students and 208 teachers”
    • Relevant Quotes: 1) “combined intervention/control cohorts of nearly 5000 year 5 (4th grade) students and 208 teachers” (p. 1) Detailed Analysis: The paper clearly distinguishes intervention and control cohorts and reports comparative outcomes, fulfilling the requirement for a documented control group. Final sentence: Criterion D is met because control group data are provided.

• Level 2 Criteria

  • S

    School-level RCT

    • The study’s RCT was conducted at the whole-school level: entire schools were randomly assigned to either the dialogic teaching intervention or the control condition, fulfilling the School‑level RCT criterion.
    • “randomisation at school level and the classes divided equally into intervention and control groups.”
    • Relevant Quotes: 1) “The trial … used a three-level clustered RCT design (pupils within classes within schools), with randomisation at school level and the classes divided equally into intervention and control groups.” (p. 19) Detailed Analysis: Randomising at the school level means that all classes within a given school were assigned to the same condition. This clustered design meets the School‑level RCT requirement by ensuring that entire schools (not just individual classrooms) were allocated to either the intervention or control group. Final sentence: Criterion S is met because whole schools were randomized to conditions.
  • I

    Independent Conduct

    • The RCT was conducted by an independent evaluation team, separate from the intervention’s designers.
    • “it was subjected to randomised control trial (RCT) by an independent team.”
    • Relevant Quotes: 1) “it was subjected to randomised control trial (RCT) by an independent team.” (p. 1) Detailed Analysis: The intervention’s evaluation was carried out by an external team rather than the program developers, satisfying the Independent Conduct criterion. Final sentence: Criterion I is met because an independent evaluator was used.
  • Y

    Year Duration

    • The study lasted 20 weeks, well short of a full academic year.
    • “after 20 weeks students in the intervention group were two months ahead…”
    • Relevant Quotes: 1) “after 20 weeks students in the intervention group were two months ahead of their control group peers…” (p. 1) Detailed Analysis: Measuring outcomes over 20 weeks does not cover an entire school year (~39 weeks), so the Year Duration criterion is not met. Final sentence: Criterion Y is not met because 20 weeks < 1 year.
  • B

    Balanced Resources

    • Control group teachers did not receive the intervention’s teacher induction, training, or mentoring support.
    • “Its professional strand entailed teacher induction and training followed by a cyclic programme of planning, target-setting and review using mentoring and video/audio analysis.”
    • Relevant Quotes: 1) “Its professional strand entailed teacher induction and training followed by a cyclic programme of planning, target-setting and review using mentoring and video/audio analysis.” (p. 1) Detailed Analysis: The intervention provided substantial additional professional development to teachers (induction, mentoring, and guided analysis of practice) which the control group did not receive. Without any comparable investment of time or support for the control teachers, the Balanced Resources criterion is not met. Final sentence: Criterion B is not met due to unmatched inputs between groups.

• Level 3 Criteria

  • R

    Reproduced Results

    • No independent replication of this dialogic teaching trial is reported in the paper or elsewhere.
    • null
    • Analysis: There is no mention of any external or subsequent replication studies in the paper or the wider literature. To date, no independent team has published a replication of this trial’s findings. Therefore, the Reproduced criterion is not satisfied. Final sentence: Criterion R is not met due to lack of replication.
  • A

    All Exams

    • Student performance was assessed in English, mathematics and science, covering the core primary curriculum.
    • “two months ahead of their control group peers in English, mathematics and science tests” (p. 1)
    • Relevant Quotes: 1) “after 20 weeks students in the intervention group were two months ahead of their control group peers in English, mathematics and science tests” (p. 1) Detailed Analysis: The paper measured outcomes across all three core subjects of the UK primary curriculum—English, maths and science—fulfilling the All‑subject Exams requirement. Final sentence: Criterion A is met because multiple subjects were assessed.
  • G

    Graduation Tracking

    • No follow‑up tracking to graduation is described in the study or in any subsequent publications.
    • null
    • Analysis: The study ends with the 20‑week post‑test; there is no evidence of any long‑term follow‑up tracking the participants through later schooling or graduation. Thus, the Graduation Tracking criterion is not met. Final sentence: Criterion G is not met due to lack of long‑term follow‑up.
  • P

    Pre-Registered Protocol

    • No pre‑registration or protocol identifier is provided in the paper (the trial was only registered retrospectively on ISRCTN, after completion).
    • null
    • Relevant Quotes: 1) No reference to any trial registry or pre‑registered protocol appears in the paper. Detailed Analysis: Without any statement of pre‑registration or a registry ID in the publication, the Pre‑registered Protocol criterion is not satisfied. (Notably, the trial was registered on ISRCTN in October 2016, after data collection was completed, rather than before the study began.) Final sentence: Criterion P is not met due to absence of a prospective pre‑registration.