Peer-Assisted Learning Strategies for English Language Learners With Learning Disabilities

ERCT Criteria Breakdown

• Level 1 Criteria

  • C

    Class-level RCT

    • Classrooms (teachers), not individual students within a classroom, were the unit of random assignment, satisfying the class-level requirement.
    • "Classrooms were stratified based on grade level and campus. Then, classrooms were randomly assigned to either the PALS condition or the contrast condition, resulting in PALS and contrast classrooms at each campus." (p. 234)
    • Relevant Quotes: 1) "Participants were 12 general educators in third- through sixth-grade transitional bilingual education classrooms from one school district in South Texas." (p. 234) 2) "Classrooms were stratified based on grade level and campus. Then, classrooms were randomly assigned to either the PALS condition or the contrast condition, resulting in PALS and contrast classrooms at each campus." (p. 234) 3) "Each participating teacher taught only one class included in the study." (p. 234) 4) "All students in each class participated in PALS." (p. 234) Detailed Analysis: The randomisation unit here is the classroom (equivalently, the teacher, since each participating teacher taught only one study class). All students within a given classroom received the same condition (PALS or contrast), which prevents within-classroom contamination between treatment and control students. This satisfies the ERCT Class-level RCT criterion directly; the stronger school-level criterion (S) is assessed separately and is not met, since both conditions were present within the same campuses. Criterion C is met because randomisation was conducted at the classroom level, with all students in a class receiving the same condition.
  • E

    Exam-based Assessment

    • Outcomes were measured with the CRAB, a researcher-modified instrument built from folktales rewritten by the authors' research group, not a standardised, widely recognised exam.
    • "The CRAB makes use of four 400-word traditional folktales, which were rewritten by Fuchs, Fuchs, and Hamlett (1989) to approximate a second- to third-grade readability level (Fry, 1968) while maintaining their meaning." (p. 240)
    • Relevant Quotes: 1) "Reading: The Comprehensive Reading Assessment Battery (CRAB)." (p. 240) 2) "The CRAB makes use of four 400-word traditional folktales, which were rewritten by Fuchs, Fuchs, and Hamlett (1989) to approximate a second- to third-grade readability level (Fry, 1968) while maintaining their meaning." (p. 240) 3) "The CRAB generates three scores: number of words read correctly, number of comprehension questions answered correctly, and maze choices correct." (p. 240) 4) "Test-retest reliability for words correct ranges from .93 to .96 (Fuchs, Deno, & Marston, 1983) and concurrent validity with the reading comprehension subtest of the Stanford Achievement Test (SAT) is .91 (Fuchs, Fuchs, & Maxwell, 1988)." (p. 240) 5) Table 2 separately reports "TAAS scores" (Texas Assessment of Academic Skills) but only as a baseline/demographic descriptor used to rank students into achievement groups, not as the study's pre/post outcome measure (p. 236, Table 2). Detailed Analysis: The primary and only outcome instrument (CRAB) is a curriculum-based measure that the authors' own research group constructed by rewriting folktales to a target readability level, rather than a widely recognised standardised achievement test. Although the authors report reliability and concurrent validity against the SAT reading comprehension subtest, correlating a custom instrument with a standardised test does not make the instrument itself standardised. The one genuinely standardised measure mentioned (TAAS) appears only as a baseline demographic variable used to stratify students into achievement tiers, not as the pre-post treatment outcome analysed in the Results section. Criterion E is not met because the study's outcome measure (CRAB) is a custom, researcher-developed instrument rather than a standardised, widely recognised exam.
  • T

    Term Duration

    • The tracked period from the start of student training through posttesting spanned roughly 16 weeks, approximating one academic term.
    • "PALS was conducted during regularly scheduled reading instruction three times a week for 35 min each session. The duration of PALS implementation was 15 weeks (not including pre- and posttesting or training)." (p. 237)
    • Relevant Quotes: 1) "PALS was conducted during regularly scheduled reading instruction three times a week for 35 min each session. The duration of PALS implementation was 15 weeks (not including pre- and posttesting or training)." (p. 237) 2) "Set-up procedures (e.g., student roles, student materials, rules), Partner Reading, and Paragraph Shrinking were taught by teachers in the PALS condition 1 week prior to the beginning of the 15 weeks of PALS implementation." (p. 239) 3) "Then, during Weeks 6 through 16, all three PALS activities were conducted." (p. 239) 4) "TAAS scores[c] ... [c]Administered after Week 5 of PALS implementation." (p. 237, Table 2 note) Detailed Analysis: Combining the 1-week pretraining period with the 15 weeks of PALS implementation yields a tracked span of approximately 16 weeks (about 3.7-4 months) from intervention start to the final posttreatment CRAB administration. This meets the ERCT definition of at least one academic term (approximately 3-4 months) between intervention start and the primary outcome measurement. Criterion T is met because the interval from the start of training/implementation to posttreatment testing spans approximately one academic term (about 16 weeks).
  • D

    Documented Control Group

    • The contrast group's normal instructional practice, demographics, and baseline performance are documented in detail, including lesson-plan comparisons and Tables 1-2.
    • "Teachers in the contrast condition were asked to conduct reading instruction in their normal fashion. To evaluate the type of instruction provided in PALS and contrast classrooms, all participating teachers submitted their lesson plans for review during Weeks 6 and 12 of PALS implementation." (p. 239)
    • Relevant Quotes: 1) "Teachers in the contrast condition were asked to conduct reading instruction in their normal fashion." (p. 239) 2) "To evaluate the type of instruction provided in PALS and contrast classrooms, all participating teachers submitted their lesson plans for review during Weeks 6 and 12 of PALS implementation." (p. 239) 3) "Findings revealed 26% of the activities in the PALS classrooms were done via one-to-one instruction, compared to only 13% in the contrast classrooms... Teacher-led instruction was used for 78% of activities conducted in PALS classrooms and 94% of activities done in the contrast classrooms." (pp. 239-240) 4) Table 1 (Teacher Demographic Data) reports age, certification, coursework hours, degree, sex, race, and teaching experience separately for the 6 PALS and 6 contrast classrooms (Table 1, p. 235). 5) Table 2 (Student Demographic Data by Treatment and Student Type) reports age, English language proficiency, grade, migrant status, reading grade level, sex, special education placement, and TAAS scores for contrast-condition students by student type (Table 2, p. 236). Detailed Analysis: The contrast condition is thoroughly documented: the paper describes what contrast teachers were instructed to do ("normal fashion" reading instruction), independently verifies actual practice via lesson-plan coding, and reports detailed teacher- and student-level demographic and baseline data for the contrast classrooms alongside the PALS classrooms, including statistical comparisons confirming baseline equivalence on most variables. Criterion D is met because the control (contrast) group's composition, instructional practice, and baseline characteristics are documented in detail.

• Level 2 Criteria

  • S

    School-level RCT

    • Randomisation occurred at the classroom level within campuses, not at the level of whole schools.
    • "Classrooms were stratified based on grade level and campus. Then, classrooms were randomly assigned to either the PALS condition or the contrast condition, resulting in PALS and contrast classrooms at each campus." (p. 234)
    • Relevant Quotes: 1) "Classrooms were stratified based on grade level and campus. Then, classrooms were randomly assigned to either the PALS condition or the contrast condition, resulting in PALS and contrast classrooms at each campus." (p. 234) 2) "Participants were 12 general educators in third- through sixth-grade transitional bilingual education classrooms from one school district in South Texas." (p. 234) Detailed Analysis: The quote explicitly states that both PALS and contrast classrooms existed "at each campus," confirming that whole schools were not randomised to a single condition; instead individual classrooms within the same schools were split between conditions. This is class-level, not school-level, randomisation. Criterion S is not met because randomisation occurred within campuses at the classroom level, not across whole schools.
  • I

    Independent Conduct

    • The PALS developers (co-authors of this paper) designed the training and provided direct, ongoing technical assistance during the study, with no independent evaluator described.
    • "Training materials, classrooms materials, teacher and student training methods, and reading activities were identical to those used by Fuchs et al. (1997)." (p. 237)
    • Relevant Quotes: 1) "Training materials, classrooms materials, teacher and student training methods, and reading activities were identical to those used by Fuchs et al. (1997)." (p. 237) 2) "Project staff was present for each day of training to provide technical assistance to the classroom teacher as necessary. This assistance consisted of helping the teacher set up and organize materials prior to the training lessons, modeling peer-tutoring strategies with the teacher, and monitoring students to identify pairs that were having difficulty." (p. 239) 3) "Upon completion of training, project staff visited each classroom for the duration of the implementation of PALS." (p. 239) 4) Authors are Laura M. Sáenz, Lynn S. Fuchs, and Douglas Fuchs; Fuchs et al. (1997) developed the PALS Grades 2-6 program cited throughout as the intervention's origin (p. 232, References p. 245). Detailed Analysis: The intervention (PALS) was designed by members of the same research team who conducted this evaluation (Fuchs et al., the original PALS developers, are co-authors), and the paper describes "project staff" - presumably the same research team - directly training teachers, modeling strategies, and monitoring implementation throughout the study. There is no statement of an external, third-party organisation independently collecting data or overseeing implementation without involvement from the intervention developers. Criterion I is not met because the study was designed, implemented, and monitored by the same research team that developed the PALS intervention, with no independent oversight described.
  • Y

    Year Duration

    • The tracked period was approximately 16 weeks, far short of the 75%-of-academic-year threshold required for Y.
    • "The duration of PALS implementation was 15 weeks (not including pre- and posttesting or training)." (p. 237)
    • Relevant Quotes: 1) "The duration of PALS implementation was 15 weeks (not including pre- and posttesting or training)." (p. 237) 2) "Set-up procedures... were taught by teachers in the PALS condition 1 week prior to the beginning of the 15 weeks of PALS implementation." (p. 239) Detailed Analysis: Including the pretraining week, the total tracked interval is approximately 16 weeks (roughly 4 months), which is well short of 75% of a full academic year (approximately 6.75-7.5 months). The study explicitly frames its duration as a single-term implementation, not a year-long one. Criterion Y is not met because the intervention and follow-up spanned only about 16 weeks, far less than 75% of an academic year.
  • B

    Balanced Control Group

    • PALS was delivered within the same regularly scheduled reading-instruction time as the contrast condition, with no extra class time or budget provided to either group; the additional teacher training and technical-assistance support given to PALS teachers is integral to implementing the tutoring method itself.
    • "Thus, the amount of time devoted to reading instruction was controlled by the school schedule." (p. 234)
    • Relevant Quotes: 1) "Because of this teaching arrangement, each participating teacher taught two different reading classes for an equivalent amount of time daily. Thus, the amount of time devoted to reading instruction was controlled by the school schedule." (p. 234) 2) "PALS was conducted during regularly scheduled reading instruction three times a week for 35 min each session." (p. 237) 3) "Teachers in the contrast condition were asked to conduct reading instruction in their normal fashion." (p. 239) 4) "Statistically significant differences were found between treatment conditions for the amount of one-to-one instruction provided... Findings revealed 26% of the activities in the PALS classrooms were done via one-to-one instruction, compared to only 13% in the contrast classrooms... However, because of the nature of PALS, the difference in the types of activities planned by PALS and contrast teachers might be expected." (p. 239) 5) "Project staff was present for each day of training to provide technical assistance to the classroom teacher as necessary... Upon completion of training, project staff visited each classroom for the duration of the implementation of PALS." (p. 238) Detailed Analysis: Applying the decision tree for Criterion B: PALS was delivered entirely within the teachers' regularly scheduled reading-instruction time, replacing rather than adding to that time; the contrast classrooms received the same amount of daily reading instruction time as usual. No extra budget, materials cost, or additional class time was allocated to the PALS group beyond the existing reading period. The differences found between conditions (proportions of one-to-one vs. teacher-led activities) reflect the pedagogical structure of the interventions being compared (peer-mediated PALS vs. normal teacher-led instruction), not an inequality of overall time or resources devoted to reading instruction. A separate resource to consider is the daily/weekly technical assistance and coaching that "project staff" provided only to PALS teachers during training and implementation, which was not offered to contrast teachers. This support is a form of professional development/implementation coaching needed to deliver the PALS tutoring method with fidelity, and is therefore integral to what is being tested (whether teachers, once trained and supported to run PALS, can improve reading outcomes), analogous to the "teacher supports as integral to the intervention package" exception in the ERCT standard, rather than a separable extra budget/time input comparable to, say, extra tutoring hours for students. It does not, on its own, indicate an unbalanced allocation of student instructional time or budget between conditions. Because no additional student instructional time or budget was provided to the intervention group beyond what is required to implement the PALS method itself, the balance requirement is satisfied. Criterion B is met because PALS was implemented within the same regularly scheduled reading-instruction time as the contrast condition, with no extra student instructional time or budget provided to either group, and the additional teacher training/coaching support is integral to delivering the PALS intervention being tested.

• Level 3 Criteria

  • R

    Reproduced

    • No independent replication of this specific PALS-for-ELL-with-LD study (Grades 3-6, Spanish-speaking students, South Texas) by a different, non-overlapping research team was found.
    • Relevant Quotes: 1) "To our knowledge, this is the first study of PALS conducted with ELL." (p. 234) 2) "Moreover, relative to Fuchs et al.'s (1997) findings, our comprehension results for ELL with LD were more robust..." (p. 243) - the comparison study (Fuchs et al., 1997) is by the same overlapping author team (Fuchs and Fuchs are co-authors of both), not an independent replication. 3) "In light of study limitations, future research should be conducted to further test the efficacy of PALS at Grades 3 to 6 for improving the reading performance of ELL with and without LD." (p. 244) - the authors themselves call for, but do not report, future replication. Internet Search Findings: A search for later studies of PALS with English language learners identified Calhoon, Al Otaiba, Greenberg, King, & Avalos (2006), "Improving Reading Skills in Predominantly Hispanic Title I First-Grade Classrooms: The Promise of Peer-Assisted Learning Strategies," Learning Disabilities Research & Practice, 21(4), 261-272. Full text was not directly accessible to extract verbatim quotes, but secondary sources indicate this study tested the Grade 1 version of PALS (targeting letter sounds, segmentation, and decoding) with predominantly Hispanic, English-language- learning first graders at risk for reading difficulty, not the Grades 2-6 version tested in the present study, and not a population specifically identified with learning disabilities. Co-author S. Al Otaiba had previously collaborated with the PALS developers (Fuchs, Fuchs, Al Otaiba et al., 2001, cited in this paper's own references), so the research team is not clearly independent of the original PALS developers. No other independent, peer-reviewed replication of this specific study's design, population, and grade range was located. Detailed Analysis: The authors explicitly state this is the first study of PALS with ELL, and the paper's own discussion calls for future research to further test PALS's efficacy with ELL, indicating no replication existed at time of writing. The only related later study identified (Calhoon et al., 2006) differs in grade level, program version, target population (general at-risk ELL rather than LD-identified students), and shares a co-author with the original PALS development team, so it does not constitute an independent replication of this specific study's central claim. Criterion R is not met because no independent replication of this specific study was found.
  • A

    All-subject Exams

    • Only reading outcomes were assessed, not all main subjects, and the underlying assessment also fails the exam-based (E) prerequisite.
    • "This study assessed the effects of Peer-Assisted Learning Strategies (PALS)... on the reading performance of native Spanish-speaking students with learning disabilities (LD)..." (p. 231, Abstract)
    • Relevant Quotes: 1) "This study assessed the effects of Peer-Assisted Learning Strategies (PALS)... on the reading performance of native Spanish-speaking students with learning disabilities (LD) and their low-, average-, and high-achieving classroom peers." (p. 231) 2) "Reading: The Comprehensive Reading Assessment Battery (CRAB)." (p. 240) - the only outcome instrument described is a reading measure; no math, science, or other subject assessments are reported. Detailed Analysis: Per the criteria-specific instruction, since Criterion E (Exam-based Assessment) is not met, Criterion A cannot be met either. In addition, on its own merits, the study only measured reading outcomes (fluency, comprehension, and maze scores via the CRAB); no other core subjects such as mathematics or science were assessed, and no rationale for a narrowly-focused specialised intervention exemption is provided (this is a general elementary reading intervention, not a specialised vocational program). Criterion A is not met because only reading was assessed and the prerequisite Criterion E is not met.
  • G

    Graduation Tracking

    • Outcomes were only measured immediately before and after the 15-16 week intervention, with no tracking toward graduation, and the prerequisite Criterion Y is not met.
    • Relevant Quotes: 1) "Students were tested before and after treatment." (p. 231, Abstract) 2) "PALS was conducted during regularly scheduled reading instruction three times a week for 35 min each session. The duration of PALS implementation was 15 weeks (not including pre- and posttesting or training)." (p. 237) 3) The Discussion/Limitations section discusses statistical power, external validity (technical assistance, single-language classroom composition), and internal validity (grade retention imbalance) but does not mention any longer-term or graduation-tracking follow-up (pp. 243-244). Internet Search Findings: A search for subsequent publications tracking this same cohort of 132 Spanish-speaking ELL students (or a graduation follow-up authored by Sáenz, L. Fuchs, or D. Fuchs on this specific sample) did not locate any such paper. Later Fuchs/Al Otaiba PALS publications identified (e.g., first-grade and kindergarten PALS extensions) address different cohorts and grade bands, not a longitudinal extension of this study's Grades 3-6 sample. Detailed Analysis: Per the criteria-specific instruction, since Criterion Y (Year Duration) is not met, Criterion G cannot be met either. Independently, the paper describes only a pretest/posttest design bracketing the roughly 16-week implementation period, with no mention of tracking students beyond this point, let alone through to graduation. No evidence of a follow-up publication tracking this specific cohort was found in the paper or via internet search. Criterion G is not met because tracking stopped at the immediate posttest, with no longer-term or graduation follow-up located, and the prerequisite Criterion Y is not met.
  • P

    Pre-Registered

    • No statement of pre-registration, registry platform, or registration date appears anywhere in the paper, and no registry record was located via internet search.
    • Relevant Quotes: No quotes referencing a study registry, pre-registration platform, or a registration date could be found anywhere in the manuscript (Method, Results, Discussion, or References sections were all reviewed). Internet Search Findings: A search of pre-registration resources (e.g., ClinicalTrials.gov, OSF Registries) for this study, its authors, or the PALS Grades 3-6/ELL trial did not locate any registry entry, which is expected given the manuscript was submitted in October 2003 and accepted in April 2004, well before prospective trial registration became a common expectation in education research. Detailed Analysis: The paper (submitted 2003, accepted 2004, published 2005) predates the widespread adoption of prospective trial pre-registration in education research and contains no mention of any registry, protocol publication, or pre-specified analysis plan published before data collection began. No pre-registration record was found through internet search. Criterion P is not met because there is no evidence of pre-registration in the paper or in available registries.