Schoolwide reforms are improving the performance of students in low-achieving schools, often in high-poverty urban areas, and proving more effective than the piecemeal solutions previously used with such students, according to a new study.

Researchers at Johns Hopkins' Center for Research on the Education of Students Placed at Risk, known as CRESPAR, spent two years evaluating nearly 800 studies of the most widely used comprehensive school reform programs to establish whether there was sufficient evidence to identify the successful programs. The results will be published later this fall in "Comprehensive School Reform and Student Achievement: A Meta-Analysis." Lead author of the report is Geoffrey Borman, who was a researcher at CRESPAR during the time of the study and is now at the University of Wisconsin-Madison.

The overall effects of comprehensive school reform are statistically significant, Borman said.

"Comprehensive school reform and President George W. Bush's No Child Left Behind Act are at the forefront of a national movement to base educational policy and practice on solid research evidence," Borman said. "There is sufficient high-quality evidence to identify several comprehensive school reform programs that do reliably help improve students' test scores, and to establish that the federal Comprehensive School Reform program is, in general, a promising initiative for turning around the nation's underperforming schools. Determining the specific effects of the models, though, requires careful examination of how they are evaluated, by whom and under what circumstances."

In schools where comprehensive reforms were in place, students outperformed approximately 55 percent of their counterparts in nonparticipating schools, earning higher scores on achievement tests. The impact was greater than the effects of traditional Title I programs for low-income students, which typically offer piecemeal solutions instead of sweeping, integrated changes. Although most of the models are used in high-poverty schools, the extent of poverty or lack of it among students does not appear to influence a program's effectiveness.

These findings are ultimately based on a subset of 232 studies of the 29 models that produced evidence of the reforms' effects on student achievement. Three models in particular--Direct Instruction, the Comer School Development Program and Success For All (developed at Johns Hopkins by Robert Slavin, Nancy Madden and a team of researchers)--have amassed a large number of studies and observations from schools and students across the United States. The effectiveness of these three models in improving student achievement is statistically significant, backed by high-quality research and replicated in many different settings, according to the study. Nine other models also appeared promising but required more high-quality research to meet the researchers' highest standard of evidence, Borman said. The effects on student test scores were most profound in schools that followed the models for five years or more.

The characteristics of the studies themselves revealed a great deal about the results, Borman said. He and his colleagues concluded that the potential for bias was high when the reform models were evaluated by their developers and when control groups were not used.

The researchers concluded that new federal education policy should provide clearer research requirements, ample funding for research and development, and a stronger focus on the model's outputs, as only 12 of the models reviewed were supported by five or more studies of their effects on student achievement.

"We need a substantial and long-term federal commitment to educational research and development of replicable programs," Borman said. "This investment could help fuel the transformation of educational research and practice in much the same way that it has helped transform medical research and treatment."

Gina Hewes and Laura Overman, both researchers at the university's Center for Social Organization of Schools, and Shelly Brown, a researcher at the University of North Carolina at Greensboro and formerly of Johns Hopkins, contributed to the report.