Effectiveness of Lib-Rungog: Audiobook on the Listening Comprehension Skills of Grade 7 Learners: Basis for Intervention Program

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Treatment was randomly assigned to intact class sections (Section 1 vs Section 2), so randomisation was at the class level rather than among students within one class.
    • "the manner of selecting which section will serve as the experimental or control group was also done through simple random sampling." (p. 8)
    • Relevant Quotes: 1) "In this study, the research design used was randomized pre-test post-test control group design because the participants were designated to either be subjected to an intervention (experimental group) or not (control group.)" (p. 7) 2) "They were grouped into control and experimental groups though simple random sampling. One section was applied with the Lib-RUNGOG: Audiobook Intervention (Section 1) while the other section was applied with the other conventional listening technique (Section 2)." (p. 8) 3) "The researchers randomly selected twenty (20) students from each section through a pure chance selection by drawing cards with their names on it to be the representatives of the entire population. In similar vein, the manner of selecting which section will serve as the experimental or control group was also done through simple random sampling." (p. 8) Detailed Analysis: Criterion C requires that randomisation be conducted at the class level (or stronger), not among individual students within a single class. In this study the treatment was assigned to intact class sections: Section 1 received the Lib-RUNGOG Audiobook intervention and Section 2 received the conventional listening strategy, and the paper explicitly states that "the manner of selecting which section will serve as the experimental or control group was also done through simple random sampling." The random drawing of 20 students from each section was a sampling step (choosing participants), not the treatment assignment; the unit of treatment assignment was the whole section (class). Although only two classes were involved, which is statistically very weak, the criterion as written asks whether entire classes rather than students within one class were randomised to conditions, and that is the case here. Contamination between treatment and control students within a single classroom is avoided because each condition was applied to a separate intact section. Criterion C is met because the two intact class sections were randomly assigned to the experimental and control conditions, i.e., randomisation occurred at the class level.
  • E

    Exam-based Assessment

    • Outcomes were measured with self-made, researcher-designed pre- and post-tests, not a recognised standardised exam.
    • "The instruments used in this action research were the researchers-made audio book in English and local language media and self-made pre-test and post-test items." (p. 9)
    • Relevant Quotes: 1) "The research instruments include a Mother Tongue-Based Multilingual Education (MTB-MLE)-integrated audiobook, pre-test, and post-test items." (p. 1) 2) "The instruments used in this action research were the researchers-made audio book in English and local language media and self-made pre-test and post-test items." (p. 9) 3) "Initially, the students will answer a researcher-made pre-test before listening to the audiobook formulated by the researchers... the students will be asked to complete a post-test that was designed by the researchers to ensure that they may gauge their level of comprehension." (p. 4) Detailed Analysis: Criterion E requires the outcome to be measured with a widely recognised standardised exam rather than a test created by the researchers for the study. The paper is explicit that both the pre-test and the post-test were "self-made" and "researcher-made" instruments designed specifically for this intervention. No national, state-wide, or otherwise standardised examination is mentioned anywhere in the paper, and no evidence of validity or reliability of the custom instrument is reported. This is exactly the problem the criterion targets: a custom test may be aligned to the intervention content (the audiobook story), inflating apparent effectiveness. Criterion E is not met because outcomes were measured with researcher-made pre-test and post-test items rather than a recognised standardised exam.
  • T

    Term Duration

    • The post-test was given immediately after a brief audiobook listening session within one school quarter, so no term-long interval between intervention start and measurement exists.
    • "After listening to the narrative, the students will be asked to complete a post-test that was designed by the researchers to ensure that they may gauge their level of comprehension." (p. 4)
    • Relevant Quotes: 1) "The participants (samples) of this action research were forty (40) Grade 7 Junior High School (JHS) students from Colegio de Tablas, during their third quarter of the Academic Year 2023-2024." (p. 8) 2) "Initially, the students will answer a researcher-made pre-test before listening to the audiobook formulated by the researchers. Following that, the students will listen to a local Romblomanon story entitled 'Old Awing and the Whale'. After listening to the narrative, the students will be asked to complete a post-test." (p. 4) 3) "The data for analysis and interpretation were collected before and after the intervention from among the twenty (20) Grade Seven students in the experimental group (Section 1) and twenty (20) Grade Seven students in the control group (Section 2)." (p. 9) Detailed Analysis: Criterion T requires that outcomes be measured at least one full academic term (roughly 3-4 months) after the intervention begins. The paper never states specific start and measurement dates, but the described procedure is very short: students take a pre-test, listen to a single audiobook story ("Old Awing and the Whale"), and then immediately complete a post-test. The whole study was conducted within the third quarter of the academic year 2023-2024, and nothing in the paper suggests any follow-up tracking beyond the immediate post-test administered after the listening activity. The interval from intervention start to outcome measurement is therefore on the order of days at most, far short of one academic term, and the paper provides no documentation of any term-long follow-up. Criterion T is not met because the post-test was administered immediately after a brief audiobook listening intervention within a single school quarter, far less than one full academic term of tracking.
  • D

    Documented Control Group

    • The control group is documented with its size (20 Grade 7 students, Section 2), baseline pre-test mean and SD, and the conventional listening condition it received.
    • "Both groups exposed to the same conditions except for the intervention program wherein the experimental group was subjected to the Lib-RUNGOG: Audiobook Intervention while the control group was exposed to the traditional listening strategy" (p. 8)
    • Relevant Quotes: 1) "Control Group: 20 students were not exposed to Lib-RUNGOG: Audiobook Intervention" (Figure 1, p. 7) 2) "Both groups exposed to the same conditions except for the intervention program wherein the experimental group was subjected to the Lib-RUNGOG: Audiobook Intervention while the control group was exposed to the traditional listening strategy" (p. 8) 3) "The participants (samples) of this action research were forty (40) Grade 7 Junior High School (JHS) students from Colegio de Tablas, during their third quarter of the Academic Year 2023-2024." (p. 8) 4) "Pre test scores of Section 2 (Control Group) 24.35 [Mean], 6.95 [SD] ... Post test scores of Section 2 (Control Group) 27.1 [Mean], 5.73 [SD]" (Table 1, p. 10) Detailed Analysis: Criterion D requires the control group to be documented in terms of who they are, their baseline performance, and what treatment they received. The paper identifies the control group as 20 Grade 7 students of Section 2 at Colegio de Tablas, reports their baseline (pre-test) mean score (24.35, SD 6.95), reports their post-test scores, and states what they received instead of the intervention (the conventional/ traditional listening strategy, with all other conditions the same as the experimental group). This matches the level of documentation accepted in the ERCT examples (group size, condition received, and baseline performance). The documentation is thin on demographics (no gender, age, or socioeconomic breakdown is given beyond grade level and school), but the group's identity, size, baseline scores, and treatment conditions are all explicitly reported, allowing comparison with the experimental group. Criterion D is met because the control group's size, grade level, school, baseline pre-test performance, and the conventional-strategy condition it received are all explicitly documented.

• Level 2 Criteria

  • S

    School-level RCT

    • The trial involved only two class sections within one school (Colegio de Tablas), so there was no school-level randomisation.
    • "The intervention will be implemented out by the researchers at Colegio de Tablas, more precisely with the students of Grade 7 enrolled in sections 1 and 2." (p. 4)
    • Relevant Quotes: 1) "The intervention will be implemented out by the researchers at Colegio de Tablas, more precisely with the students of Grade 7 enrolled in sections 1 and 2." (p. 4) 2) "One section was applied with the Lib-RUNGOG: Audiobook Intervention (Section 1) while the other section was applied with the other conventional listening technique (Section 2)." (p. 8) Detailed Analysis: Criterion S requires randomisation among schools (or equivalent implementing institutions). This study took place in a single school, Colegio de Tablas, and the units assigned to conditions were two class sections within that one school. No multiple schools, centres, or sites were involved, and no school-level randomisation is described anywhere in the paper. Randomisation at the class-section level within one school does not satisfy the school-level requirement. Criterion S is not met because the study was conducted in a single school with randomisation only between two class sections, not among schools.
  • I

    Independent Conduct

    • The same researchers who created the audiobook and the tests also delivered the intervention and collected and analysed all data, with no independent evaluator.
    • "Furthermore, the researchers administered the pre-test, post-test, and the intervention program in the target population sample." (p. 9)
    • Relevant Quotes: 1) "The intervention will be implemented out by the researchers at Colegio de Tablas, more precisely with the students of Grade 7 enrolled in sections 1 and 2." (p. 4) 2) "the students will listen to the audiobook formulated by the researchers" (p. 4) 3) "The instruments used in this action research were the researchers-made audio book in English and local language media and self-made pre-test and post-test items." (p. 9) 4) "Furthermore, the researchers administered the pre-test, post-test, and the intervention program in the target population sample." (p. 9) Detailed Analysis: Criterion I requires the study to be conducted independently from the designers of the intervention, or at least to have third-party oversight of data collection and analysis. Here the same two researchers designed the Lib-RUNGOG audiobook, created the pre- and post-tests, implemented the intervention themselves, administered all assessments, and analysed the results. No external evaluators, independent enumerators, or third-party oversight of any kind is mentioned. This is the textbook case of non-independent conduct that the criterion is designed to flag. Criterion I is not met because the intervention designers themselves implemented the intervention, administered the tests, and analysed the data with no independent oversight.
  • Y

    Year Duration

    • The entire study, including the immediate post-test, took place within one school quarter, far short of 75% of an academic year (and criterion T is also not met).
    • "The participants (samples) of this action research were forty (40) Grade 7 Junior High School (JHS) students from Colegio de Tablas, during their third quarter of the Academic Year 2023-2024." (p. 8)
    • Relevant Quotes: 1) "The participants (samples) of this action research were forty (40) Grade 7 Junior High School (JHS) students from Colegio de Tablas, during their third quarter of the Academic Year 2023-2024." (p. 8) 2) "After listening to the narrative, the students will be asked to complete a post-test that was designed by the researchers." (p. 4) Detailed Analysis: Criterion Y requires that outcomes be measured at least 75% of a full academic year after the intervention begins. The weaker prerequisite criterion T (term duration) is already not met: the study consisted of a brief audiobook listening activity with an immediate post-test, all within the third quarter of a single academic year. Per the ranking instructions, if criterion T is not met then criterion Y cannot be met. There is no year-long tracking of outcomes anywhere in the paper; indeed the authors themselves recommend that "Future research should explore the long-term effects" (p. 1), confirming that no long-duration follow-up was done. Criterion Y is not met because the intervention and measurement occurred within a single school quarter with an immediate post-test, nowhere near 75% of an academic year.
  • B

    Balanced Control Group

    • The control group was an active control doing a conventional listening activity under the same procedure and time, so inputs were balanced and the audiobook medium was the treatment contrast.
    • "Both groups exposed to the same conditions except for the intervention program wherein the experimental group was subjected to the Lib-RUNGOG: Audiobook Intervention while the control group was exposed to the traditional listening strategy" (p. 8)
    • Relevant Quotes: 1) "Both groups exposed to the same conditions except for the intervention program wherein the experimental group was subjected to the Lib-RUNGOG: Audiobook Intervention while the control group was exposed to the traditional listening strategy" (p. 8) 2) "Table 2: Data Collection ... Pre-test / Utilization of Lib-RUNGOG: Audiobook Intervention / Utilization of the Conventional Listening Strategy / Taking down notes and observations / Writing reflections / Estimating students' mastery level of the macro skill / Post test" (p. 9) 3) "the study compares the performance of an experimental group exposed to the audiobook intervention with a control group utilizing traditional listening strategies." (p. 1) Detailed Analysis: Criterion B requires the control group to receive balanced time and resources so that the specific intervention effect is isolated. Applying the current decision tree: the first question is whether the intervention group was given extra time or budget relative to the control group at all. Here both sections followed the same overall procedure (pre-test, a single listening activity, notes/reflections, post-test) during the same class time; the only difference is the content medium used during that shared listening slot (Lib-RUNGOG audiobook vs. the conventional listening strategy). The paper states both groups were "exposed to the same conditions except for the intervention program," and Table 2's data-collection sequence is identical for both arms. Since no additional instructional time, budget, or materials beyond swapping the listening medium were given to the experimental group, this falls into the "no extra resources present" branch of the decision tree, which is met by default without needing to assess whether the difference is the treatment variable. Even if the audiobook medium were treated as an "extra resource," it is clearly integral to the treatment being tested (the whole point of the study is comparing the audiobook medium to the conventional strategy within the same time slot), so the criterion would still be met under the treatment-variable branch. Criterion B is met because both groups received the same class time and procedural structure, with the audiobook medium being the treatment contrast itself rather than an unmatched additional resource.

• Level 3 Criteria

  • R

    Reproduced

    • This 2024 study of a novel, locally developed audiobook intervention has no independent replication by another research team in the peer-reviewed literature, confirmed by internet searches during verification.
    • Relevant Quotes: 1) "Lib-RUNGOG is an educational intervention patterned through audiobook to provide supplemental aid for students' listening comprehension skills. Etymologically speaking, the project title is derived from the terms 'libro' which relates to books and 'rungog' from the Asi Language of Romblon." (p. 3) 2) "Future research should explore the long-term effects and establish collaborations among teachers, parents, and researchers to optimize educational strategies, especially in the context of listening skills." (p. 1) Detailed Analysis: Criterion R requires that the study be independently replicated by a different research team in a different context and published in a peer-reviewed journal. Lib-RUNGOG is a novel, locally developed intervention (a Romblomanon audiobook created by the two authors) first reported in this January-February 2024 paper. The paper itself cites no replication of this specific intervention, and the authors explicitly frame long-term and further study as future work. Independent internet searches conducted for this verification (Google Scholar search for "Lib-RUNGOG", and general web search engines) returned no citing papers, no follow-up studies, and no independent replications of this specific intervention by any research team; Google Scholar explicitly reported no matching articles for the term "Lib-RUNGOG." Given the very recent publication date, the highly localised nature of the intervention (an Asi-language-named audiobook of a local Romblomanon story), and the obscure venue, no independent published replication of this specific study by a different team is known or referenced. Criterion R is not met because no independent peer-reviewed replication of this specific Lib-RUNGOG audiobook trial by a different research team exists or is referenced, and none was located through internet searches conducted during this verification.
  • A

    All-subject Exams

    • Only listening comprehension was assessed with a custom test (criterion E fails), so no all-subject standardised assessment took place.
    • "This study specifically centers on the Lib-RUNGOG Audiobook and its impact on listening comprehension skills; therefore, other teaching methods or interventions are not extensively explored." (p. 6)
    • Relevant Quotes: 1) "The instruments used in this action research were the researchers-made audio book in English and local language media and self-made pre-test and post-test items." (p. 9) 2) "This study specifically centers on the Lib-RUNGOG Audiobook and its impact on listening comprehension skills; therefore, other teaching methods or interventions are not extensively explored." (p. 6) Detailed Analysis: Criterion A requires standardised exam-based assessment across all main school subjects, and it explicitly has criterion E as a prerequisite. Criterion E is not met here because the outcome instrument was a self-made test; per the ranking instructions, criterion A therefore automatically fails. In addition, only a single narrow outcome (listening comprehension of one story) was measured; no other subjects such as mathematics, science, or social studies were assessed at all, and no justified exception applies for a general Grade 7 population. Criterion A is not met because criterion E fails (custom test) and only listening comprehension was measured, not all main subjects.
  • G

    Graduation Tracking

    • Tracking ended at the immediate post-test and the authors defer long-term effects to future research, so no graduation tracking occurred, confirmed by internet search during verification.
    • "Furthermore, the study prompts a call for ongoing research initiatives to delve into the long-term impact of audiobook interventions on reading skills and student engagement." (p. 13)
    • Relevant Quotes: 1) "Furthermore, the study prompts a call for ongoing research initiatives to delve into the long-term impact of audiobook interventions on reading skills and student engagement. Exploring the sustained effects over extended periods will provide invaluable insights into these interventions and inform future educational strategies." (p. 13) 2) "After listening to the narrative, the students will be asked to complete a post-test." (p. 4) Detailed Analysis: Criterion G requires tracking participants until graduation from their educational stage, and per the ranking instructions it cannot be met if criterion Y is not met. Criterion Y fails here. Measurement ended with the immediate post-test after the listening activity in the third quarter of Grade 7; the students were not followed to the end of junior high school or any graduation point. The authors themselves recommend that future research explore long-term effects, confirming no long-term follow-up occurred. Independent internet searches conducted for this verification (Google Scholar for "Lib-RUNGOG", and general web search engines for the authors and intervention name) found no follow-up publications by Fallasgon or Federico tracking this Grade 7 cohort, and no evidence that graduation tracking was ever conducted or published. Criterion G is not met because measurement stopped at the immediate post-test with no tracking of the Grade 7 cohort to graduation (and criterion Y is also not met); no follow-up publications were located during this verification.
  • P

    Pre-Registered

    • The paper contains no mention of any protocol pre-registration or trial registry, confirmed by internet search during verification.
    • Relevant Quotes: 1) "Prior to the conduct of the experimental study, the researchers sought approval from the subject teacher and class adviser whose supervision covered the target level and section for both the control and experimental groups which were targeted by the study." (p. 9) 2) "In the pursuit of this research, the students', their parents' or guardians', as well as their adviser's free prior and informed consent were sought for." (p. 10) Detailed Analysis: Criterion P requires pre-registration of the full study protocol (hypotheses, methods, planned analyses) on a public registry before data collection began. The paper mentions only teacher approval and participant consent; it contains no reference to any registry (e.g., ClinicalTrials.gov, OSF, AEA RCT Registry), no registration ID, and no registration date. Independent internet searches conducted for this verification (general web search engines, checking for any registry entry under the authors' names or the intervention name "Lib-RUNGOG") found no evidence of a pre-registration record on any registry platform. Given this is a small, locally published action-research study with no mention of a registered protocol anywhere in the text, there is no evidence anywhere that a protocol was pre-registered before data collection. Criterion P is not met because no pre-registration statement, registry ID, or registration date appears anywhere in the paper, and none was found through internet searches conducted during this verification.