Success for All: A Summary of Evaluations. ERIC/CUE Digest

Success for All (SFA), a schoolwide research-based reform model developed by Robert Slavin and his associates at Johns Hopkins University, is based on the premise that all students can and must succeed in the early grades (Slavin, Madden, Dolan, & Wasik, 1996). In the belief that reading is fundamental to other skill areas, the program targets students in lower elementary school grades. It provides students with intensive instruction in language arts, extensive professional development to help teachers succeed with every student, and an active family support program. Results from research conducted by the program developers, as well as by external evaluators, have shown the SFA program to be effective in enhancing the reading achievement of economically disadvantaged and non-native English speaking students.

SUCCESS FOR ALL PROGRAM COMPONENTS

The assumption at the heart of the Success for All program design is that preventing the occurrence of early learning problems with immediate comprehensive intervention is more effective than later remediation of academic difficulties (Slavin et al., 1996). The SFA model includes the following components:

*a systematic reading program that emphasizes story telling and retelling (STaR), and language development activities such as phonics, vocabulary building, auditory discrimination, and sound blending using cooperative learning techniques;

*a daily 90-minute reading period with grade 1-3 students regrouped into homogeneous cross-grade ability groups;

*one-on-one tutoring in reading by specially trained certified teachers who work individually with students reading below grade level;

*assessments every eight weeks to determine students' reading progress, adjust reading group placement, and assign tutoring if needed;

*professional development for teachers and tutors, which includes three days of inservice training and guidelines at the start of the school year, and follow-up training throughout the year;

*a family support team designed to provide parenting education, assist families of students experiencing personal or health-related problems, and support family involvement in the school;

*a facilitator who works with teachers and staff in implementing the program; and

*an advisory committee comprised of the principal, facilitator, teacher and parent representatives, and family support staff that meets regularly to review program progress.

First implemented during the 1987-88 school year in five inner-city schools in Baltimore, MD, SFA has expanded to over 475 schools in 31 states across the country. It now serves over one quarter of a million American students.

SFA'S ABILITY TO INCREASE STUDENTS' READING ACHIEVEMENT EVALUATION DESIGN

The basic research design used to determine the effectiveness of SFA has been to match the SFA school with a control school that is similar in terms of poverty level, historical achievement level, ethnicity of students, and other factors (Slavin, Madden, Dolan, Wasik, Ross, Smith, & Dianda, 1996). SFA students are then matched to counterparts at the control schools based on reading pre-test scores. At the end of the school year, the two groups are administered reading tests and their results are compared. Since the SFA students and their counterparts were similar (e.g., academically, socioeconomically, etc.) before the program began, any gains in SFA students' reading achievement at the end of the year can be attributed to the program intervention.

OVERALL FINDINGS

The outcomes at multiple SFA schools have been extensively evaluated since the program's inception. Early evaluations by program developers have shown positive results for all schools on reading measures. Smaller studies have shown consistently positive outcomes in areas such as improved attendance and a reduction in special education placements (Madden, Slavin, Karweit, Dolan, & Wasik, 1993; Slavin, Madden, Dolan, Wasik, Ross et al., 1996).

Slavin and his associates have provided a synthesis of research on SFA over a six-year period from 1988 to 1994 (Slavin, Madden, Dolan, Wasik, Ross, et al., 1996). By combining results of multi-site studies across all years, they were able to show uniform improvement of SFA students in all reading measures as compared to their matched counterparts in non-SFA (control) schools.

SFA'S EFFECTS ON STUDENTS IN THE LOWEST QUARTILE OF THEIR CLASSES

The SFA program developers also found that the greatest benefits of the program are consistently derived by the SFA students who are in the lowest 25 percent of their grade on pre-tests. This means that SFA students with the greatest risk of academic failure perform better and at a much faster pace than their counterparts in non-SFA schools. SFA students in higher quartiles of their classes also outperform their counterparts but not to the same extent. External evaluators have consistently replicated this finding (Slavin, Madden, Dolan, Wasik, Ross et al., 1996). For example, in their comparison of English dominant, bilingual, and English as a Second Language (ESL) SFA students, Dianda and Flaherty (1995a; 1995b) determined that the highest need SFA students outpaced the control students on several reading measures by up to six months. They attribute the success of these students to the daily one-to-one tutoring and the support from the Family Support Team.

SFA'S EFFECTS ON READING ACHIEVEMENT OVER TIME

A key finding by Slavin and his associates in their six-year multi-site study is that the longer a school has implemented SFA, the greater the reading achievement of its students. Slavin proposes that reading outcomes improved as schools gained more experience in implementing SFA. However, he also notes that it is possible that the gains they observed are a result of the lasting effect of participation in the SFA pre-kindergarten and kindergarten programs, since most of the students had started in those grades (Slavin, Madden, Dolan, & Wasik, 1996).

IMPACT OF RACE, ETHNICITY, AND LANGUAGE ON THE READING ACHIEVEMENT OF SFA STUDENTS EFFECTS OF RACE/ETHNICITY

While much research has been conducted comparing SFA and non-SFA children, less is known about the effect of ethnicity or race on the reading achievement of SFA children. One SFA study of a racially integrated school in Fort Wayne, IN, compared the impact of ethnicity on achievement outcomes of SFA students (Ross, Smith, Slavin, & Madden, 1997). In comparing the reading achievement of African American and white students, the researchers found that although African American children scored lower on pre-tests and were more socially disadvantaged than the white students when entering kindergarten, the African American children, when post-tested several grades later, had actually benefited more than the white students, sustaining an advantage over them.

EFFECTS OF SFA ON ENGLISH LANGUAGE LEARNERS

The first SFA program that served English language learners was implemented in an inner-city school in Philadelphia located in a high-poverty neighborhood where 60 percent of the entering students speak Cambodian or other southeast Asian languages. SFA was adapted to meet these children's language needs, and included the integration of ESL teachers into the program. In a study comparing Asian SFA non-native English speaking students with a control group, Slavin and Yampolsky (1991) found that SFA Asian students in all three grades they studied (3-5) performed significantly better than the Asian control students, reading a half-year or more above grade level while the similar non-SFA Asian students were reading a full year below grade level.

The first bilingual version of Success for All, "Lee Conmigo," was also instituted in a low-income inner-city Philadelphia elementary school with a predominantly Latino student population. Spanish-speaking, limited English proficient (LEP) students at both the SFA and the matched non-SFA school were pre-tested at the beginning of kindergarten using a Spanish language picture vocabulary test and tested again at the end of second grade on various Spanish language reading measures. Slavin & Madden (1995) found that while the non-SFA bilingual students at the control school scored far below grade level on all measures, the SFA bilingual students scored near grade level.

These results were also replicated by external evaluators (Dianda & Flaherty, 1995a; 1995b) who examined the effects of SFA on achievement outcomes of English language learners in four types of instructional settings in three different SFA schools in California that served large numbers of bilingual and ESL students. The four groups of SFA students included: English dominant students; Spanish bilingual students taught in Spanish using "Lee Conmigo"; Spanish ESL students taught in English ("sheltered students"); and other ESL students (of various language backgrounds). ESL students all received ESL-adapted versions of SFA. All SFA and control students were pretested in kindergarten and again when they completed first grade using various reading measures. The results for all SFA students were very positive; on average, they were reading several months ahead of their non-SFA counterparts. The greatest gains were made for SFA students who were in the lowest 25 percent of their classes.

THE IMPACT OF SFA ON SPECIAL EDUCATION

A key program goal is to prevent learning deficiencies and ensure the success of all children very early on, thereby reducing the number of children requiring special education services. Research by program developers and others has shown consistently that the most dramatic effects of Success for All occur for children in the lowest 25 percent of their classes--those most at risk for special education placement. Slavin, Madden, Dolan, & Wasik (1996) found in their longitudinal study of SFA students in Baltimore schools that only two percent of the Success for All third graders (including special education students) were reading two years below grade level compared to nine percent of the control group. The Baltimore study also showed a reduction in special education placements of about one-half. Research by Smith and colleagues in a study across four states determined that first grade SFA special education students scored much higher than their counterparts in the control schools (Slavin, 1996).

THE LIMITATIONS OF SUCCESS FOR ALL DURATION OF SFA'S IMPACT ON READING ACHIEVEMENT

In order to determine the long-term impact of SFA, program developers followed the first group of students (who started in kindergarten or first grade) to receive SFA program intervention in Baltimore in 1988 through to sixth and seventh grades. Researchers found that SFA students scored higher than the matched non-SFA students on every reading measure at each grade level. At sixth and seventh grade, SFA students (although no longer in the program) were still scoring higher than their matched counterparts, but to a lesser extent. By eighth and ninth grades, however, neither group of students, those who had been in SFA programs or their counterparts who had not, seemed to be growing much in their reading (Slavin, Madden, Dolan, & Wasik, 1996). Thus, while there is some continuing effect after leaving the program at the end of elementary school, the effect appears to end in the middle grades.

THE GOAL OF READING AT GRADE LEVEL

A key SFA program goal is to ensure that every child reads at grade level by third grade. In order to determine whether this goal has been achieved, Madden et al. (1993) looked at the reading performance of SFA third graders in Baltimore and compared them to a control group of non-SFA third graders. Although SFA third graders performed better than the control group, approximately one-fifth of the program group were performing at least one year below grade level (compared to almost one-half of the control students who were performing at least one year below grade level).

Indeed, the biggest challenge of the Success for All program is to raise the reading level of its students to grade level or above. In Slavin's multi-site study spanning the years 1988 to 1994, the SFA students in the first through third grades did perform at grade level, as did their first and second grade counterparts at the control schools. However, fourth and fifth grade SFA students appeared to be reading below grade level at the end of each school year, though still performing better than their matched counterparts (Slavin, Madden, Dolan, & Wasik, 1996).

VARIATIONS IN FINDINGS ACROSS SFA SCHOOLS

Ross and associates have reported findings from a comparative study on the effects of SFA in schools in four cities (Memphis, TN; Montgomery, AL; Fort Wayne, IN; and Caldwell, ID). They found that although achievement results were positive at three of the schools, the results were inconsistent across all four sites. Reading outcomes at the Idaho school were the most inconsistent, yielding no positive effects for SFA first graders. Researchers speculate that the control school in Idaho might have been changing and becoming more similar to the program school. Nevertheless, the research points to the challenges in replicating successful results across a large number of schools, given differences in training, school conditions, and comparison schools that are used for evaluation purposes (Ross, Smith, Casey, Johnson, & Bond, 1994).

CURRENT RESEARCH ON THE QUALITY OF SFA PROGRAM IMPLEMENTATION

Recent research on SFA has also begun to focus on issues of program implementation in addition to student achievement outcomes. This research involves examining the experiences and perspectives of SFA teachers, facilitators, and principals, as well as incorporating implementation measures into evaluation designs (Ross et al., 1997). In one study which surveyed teachers on their impressions of the effectiveness of various program components, Ross and his colleagues (Ross et al., 1997) found that teachers were very positive about the reading curriculum and tutoring components, but less positive about the Advisory Committee and Family Support Team. Researchers speculate that the components external to the reading curriculum, the centerpiece of the program, probably receive less focus during implementation, and consequently are not perceived as effective. The researchers also found that most of the teachers surveyed expressed the view that student interest and achievement in reading had increased after the implementation of SFA.

SFA researchers have also found it useful to examine the quality of individual program components (e.g., reading curriculum, tutoring, Family Support Team, etc.) in their evaluations. Certified SFA trainers rate the quality of each program component in the SFA school under study in order to provide researchers with more information about the strengths and weaknesses of the components within and between schools (Ross et al., 1997). Using such information, combined with student outcome data, researchers are better able to explain varying levels of success in SFA schools. Results from such studies have shown that in schools where implementation levels are high, that is, in schools where individual components were rated highly by trainers, the effects on students' reading achievement were also strong. On the other hand, in SFA schools where program implementations were weak, the effects on student achievement were also weak (Madden et al., 1993; Ross et al., 1997).

As program developers point out, given the large number of schools adopting SFA, and the exigencies of individual schools, it is unrealistic to expect that all schools will implement the program as designed. This was the case in the Houston Independent School District which adopted SFA in 50 of its schools (20 of which had Spanish bilingual versions). Some adopted just the reading program; others, the reading plus tutoring programs; still others, the entire program as prescribed. As expected, the greatest effects were realized in the schools which implemented the model fully (Ross et al., 1997). In a subsequent study of the same Houston SFA schools, researchers found differences in achievement effects by implementation level according to the race/ethnicity and level of poverty of the SFA student populations (Nunnery, Slavin, Madden, Ross, Smith, Hunter, & Stubbs, 1997). For example, student outcomes in extremely poor and predominantly African American schools "appeared to be much more sensitive to variations in implementation than less impoverished" predominantly Latino schools (p. 5). While both high and medium level implementations produced identical effects in moderately impoverished schools and predominately Latino schools, only in SFA schools with high level implementations (compared to medium level implementations) did African American students gain in reading. Moreover, high implementation African American schools were the only schools where, on average, students met or exceeded grade level on every reading measure, scoring above national norms (Nunnery et al., 1997).

Currently, program developers are conducting research on school context factors that contribute to successful program implementations (Cooper, Slavin, & Madden, 1997). Preliminary findings suggest that successful programs are those in which school principals are committed to school-wide change (as compared to "piecemeal change"), teachers "buy into" the program, SFA facilitators are engaged in a district-wide support network with other SFA facilitators, and the Family Support Team is integrated into the life of the school. Indeed, researchers found that the most salient issue facing most SFA schools is the high level of poverty in the community where the schools are located; it represents the biggest challenge to the successful replication of the program. The extent to which the Family Support Team can meet the multiple needs of the children--physical, emotional, psychological, and academic--will in large measure determine the success of the program.

REFERENCES

Cooper, R., Slavin, R.E., & Madden, N.A. (1997). Success for All: Exploring the technical, normative, political, and socio-cultural dimensions of scaling up. Baltimore: Johns Hopkins University, Center for Research in the Education of Students Placed at Risk. (ED 412 324)

Dianda, M.R., & Flaherty, J.F. (1995a). Report on workstation uses: Effects of Success for All on the reading achievement of first graders in California bilingual programs. Los Alamitos, CA: Southwest Regional Lab. (ED 394 327)

Dianda, M.R., & Flaherty, J.F. (1995b). Promising practices, and program adaptations & successes. Los Alamitos, CA: Southwest Regional Lab (ED 383 793)

Madden, N.A., Slavin, R.E., Karweit, N.L., Dolan, L., & Wasik, B.A. (1993). Success for All: Longitudinal effects of a restructuring program for inner-city elementary schools. American Educational Research Journal, 30(1), 123-148. (EJ 463 408)

Nunnery, J., Slavin, R.E., Madden, N.A., Ross, S., Smith, L., Hunter, P., & Stubbs, J. (1997). Effects of full and partial implementations of Success for All on student reading achievement in English and Spanish. Paper presented at the annual meeting of the American Educational Research Association, Chicago. (ED 408 558)

Ross, S.M., Smith, L.J., Slavin, R.E., & Madden, N.A. (1997). Improving the academic success of disadvantaged children: An examination of Success for All. Psychology in the Schools, 34(2), 171-80. (EJ 551 696)

Ross, S.M., Smith, L.J., Casey, J., Johnson, F., & Bond, C. (1994, April). Using "Success for All" to restructure elementary schools: A tale of four cities. Paper presented at the annual meeting of the American Educational Research Association, New Orleans. (ED 373 456)

Slavin, R.E. (1996, February). Neverstreaming: Preventing learning disabilities. Educational Leadership, 53(5), 4-7. (EJ 517 837)

Slavin, R.E., Madden, N.A., Dolan, L., & Wasik, B.A. (1996). Every child, every school: Success for All. Thousand Oaks, CA: Corwin Press. (ED 397 950)

Slavin, R.E., Madden, N.A., Dolan, L.J., Wasik, B.A., Ross, S.M., Smith, L.J., & Dianda, M. (1996). Success for All: A summary of research. Journal of Education for Students Placed at Risk, 1(1), 41-76. (EJ 530 239)

Slavin, R.L., & Madden, N.A. (1995). Effects of Success for All on the achievement of English language learners. Paper presented at the annual meeting of the American Educational Research Association, San Francisco. (ED 388 050)

Venezky, R.L. (1994). An evaluation of Success for All: Final report to the France and Merrick Foundations. Newark, DE: University of Delaware, Department of Educational Studies.