Improving Quality of Teaching and Learning in Classes by using Augmented Reality Video

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Randomization was conducted at the class (tutorial) level, satisfying the class‑level RCT requirement.
    • “all students were randomly allocated into different … classes by the PolyU computer system, which randomly selected the tutorial class designated Group A (received handouts) or Group B (viewed AR videos).”
    • Relevant Quotes: 1) “To reduce selection bias and obtain more accurate results, a randomized controlled trial (RCT) was run. Study participants were randomly allocated to two different groups: either the intervention group … or the control group.” (p. 6) Detailed Analysis: The paper explicitly describes randomization at the level of tutorial classes—the PolyU system randomly assigned entire tutorial sections to either handout or AR‑video conditions. This is class‑level randomization. Final sentence explaining if criterion C is met because randomization was done at the class level.
  • E

    Exam-based Assessment

    • The study used author‑designed pre/post tests rather than a standardized exam.
    • “A handout with instructions …” and “the AR component … displayed a video…” (no reference to any standardized test)
    • Relevant Quotes: 1) “A traditional handout with instructions … was provided to Group A.” (p. 12) 2) “Group B students … scanned a symbol … and the AR component … displayed a video that would allow them to learn about threading.” (p. 12) Detailed Analysis: All assessments were custom‑designed by the authors (pre‑test, post‑test, questionnaire) and no state‑ or national‑level exam was used. Final sentence explaining if criterion E is not met because only custom tests were used.
  • T

    Term Duration

    • Outcome measures were collected within days, not after a full academic term.
    • “... from 26 to 29 September 2017 …”
    • Relevant Quotes: 1) “Students were administered a threading task … from 26 to 29 September 2017.” (p. 7) Detailed Analysis: Outcomes were measured over a span of days within a single week, far shorter than the one‑term minimum required by ERCT. Final sentence explaining if criterion T is not met because the follow‑up was under one week.
  • D

    Documented Control Group

    • The handout control group is well described with baseline tasks and scores.
    • “a traditional handout … was provided to Group A.”
    • Relevant Quotes: 1) “Group A received the handout … Group B viewed AR videos. The same materials were then made available to all students after the trial.” (p. 6) 2) “For both the pre‑test and post‑test … the score depended on each threading step.” (p. 10) Detailed Analysis: The paper clearly documents the control group’s composition, baseline performance, and confirms no unintended interventions. Final sentence explaining if criterion D is met because the control group is fully documented.

• Level 2 Criteria

  • S

    School-level RCT

    • Randomisation occurred at the tutorial‑class level, not at the school level.
    • “... randomly allocated into different … classes …”
    • Relevant Quotes: 1) “To reduce selection bias … participants were randomly allocated to two different groups: either the intervention group … or the control group.” (p. 6) Detailed Analysis: Randomization was at the class level; no evidence of school‑level assignment is provided. Final sentence explaining if criterion S is not met because only tutorial classes were randomized.
  • I

    Independent Conduct

    • The authors’ own team both developed and evaluated the intervention.
    • “... the app was developed by the Institute of Textiles and Clothing …”
    • Relevant Quotes: 1) “This study was run … by the Institute of Textiles and Clothing …” (p. 6) 2) “The mobile app … was developed by ITC VR AR team.” (p. 6) Detailed Analysis: The same institution that developed the AR app also conducted the trial and analysis, with no independent external evaluator. Final sentence explaining if criterion I is not met because designers and evaluators are the same.
  • Y

    Year Duration

    • Follow‑up lasted only days, not a full academic year.
    • “... from 26 to 29 September 2017.”
    • Relevant Quotes: 1) “Threading task … from 26 to 29 September 2017.” (p. 7) Detailed Analysis: No tracking beyond the brief workshop period; far shorter than one academic year. Final sentence explaining if criterion Y is not met because follow‑up lasted under one week.
  • B

    Balanced Resources

    • The AR intervention entailed multimedia app access not matched by the handout.
    • “... handout …” vs. “... AR video …”
    • Relevant Quotes: 1) “Group A received a handout … Group B viewed AR videos.” (p. 6) Detailed Analysis: The AR group used an interactive mobile app; the control had only printed materials—resources were unbalanced. Final sentence explaining if criterion B is not met because resources were unbalanced.

• Level 3 Criteria

  • R

    Reproduced Results

    • Independent teams reproduced similar AR‐enhanced learning gains in other contexts.
    • “The experimental results indicated that integrating AR into PBL activities both increased students' learning achievement and promoted their positive attitudes towards physics subjects.”
    • Relevant Quotes: 1) “The experimental results indicated that integrating AR into PBL activities both increased students' learning achievement and promoted their positive attitudes towards physics subjects.” (Fidan & Tuncel, 2019) 2) “Students in the experimental group were found to have higher levels of achievement and more positive attitudes towards the course than those in the control.” (Şahin & Yılmaz, 2020) Detailed Analysis: Two independent studies, in physics education and middle‑school science, by different teams have replicated the core finding that AR improves learning outcomes and attitudes. Final sentence explaining if criterion R is met because the study’s findings have been independently reproduced.
  • A

    All Exams

    • Only sewing/textiles skills were assessed, not all core subjects.
    • “... threading task …”
    • Relevant Quotes: 1) “A simple test … on knitting needles.” (p. 7) 2) “Threading task … in sewing workshops.” (p. 9) Detailed Analysis: Academic outcomes were confined to textiles‑related skills; no exams across all core subjects were administered. Final sentence explaining if criterion A is not met because only a single domain was tested.
  • G

    Graduation Tracking

    • No follow‑up beyond the immediate post‑workshop period.
    • Relevant Quotes: None. Detailed Analysis: The original paper and any follow‑up by the authors contain no tracking through to graduation or long‑term outcomes. Final sentence explaining if criterion G is not met because graduation tracking is absent.
  • P

    Pre-Registered Protocol

    • No evidence of pre‑registration is provided.
    • Relevant Quotes: None. Detailed Analysis: The study makes no reference to any registry platform or pre‑analysis plan, and no entry was found in standard registries before data collection. Final sentence explaining if criterion P is not met because no pre‑registered protocol exists.