U. S. Department of Education Study Questioned;
Private School Advantage Identified
Cambridge, MA -- After re-analyzing the data used by a recent U.S. Department of Education study that found a rough parity in the effects of public and private schools on student achievement, Kennedy School of Government researchers Paul E. Peterson and Elena Llaudet, using what they regard as a preferred methodology, have identified a consistent, statistically significant private school advantage.
Utilizing the same data as the original study but substituting better measures of student characteristics, the Harvard researchers find a private school advantage in 11 out of 12 public-private comparisons. In 8th-grade math, the private school advantage varies between 3 and 7 test points; in reading, it varies between 9 and 13 points. Among 4th graders, in math, parity is observed in one model, but private schools outperform public schools by 2 to 4 points in the other two models; in 4th-grade reading, private schools have an advantage that ranges from 6 to 10 points. All but one of the differences are statistically significant. (See table below for results as well as detailed breakdowns for Catholic, Lutheran, Evangelical Protestant and other private schools.)
The original study, released by the U. S. Department of Education's National Center for Education Statistics (NCES), used information from a nationwide sample of public and private school students collected in 2003 as part of the ongoing National Assessment of Educational Progress (NAEP). Once adjustments for student background characteristics were made, the NCES study reported a 4.5 percentage point advantage for public schools among 4th graders in math, parity between the sectors in 8th grade math and 4th grade reading, and a private-sector advantage of 7.3 points in 8th grade reading.
Peterson and Llaudet contend that key measures of student background characteristics in the NCES study are inaccurate because they are based upon student participation in the following federal programs for disadvantaged students: Title I programs for disadvantaged students; free and reduced-price lunch programs; programs for those with Limited English Proficiency (LEP); and special education services that require an Individualized Education Plan (IEP).
The Harvard researchers provide evidence showing that many private schools do not participate, or participate less fully, in these programs. For example, if a public school has a school-wide Title I program, something that is permitted if 40 percent of its students are eligible for the free lunch program, then every student at the school, regardless of poverty level, is considered a recipient of Title I services. By contrast, private schools must negotiate arrangements with local public school districts, which then provide the Title I services to eligible students. Since many private schools lack the administrative capacity to handle these complex negotiations or do not wish to provide services that they will not administer, private school participation is haphazard. In the 2003-04 school year only 19 percent of private schools participated in Title I, as compared to 54 percent of public schools.
"When you use participation in federal programs as a measure of a student's
family background, you under-count the number of disadvantaged students
in the private sector. Public schools are expected to participate in these
programs, while private schools are not," said Paul E. Peterson, the Henry Lee Shattuck Professor of Government and director of the Program on Education Policy and Governance (PEPG) at Harvard University.
To measure student characteristics, Peterson and Llaudet substituted information supplied by the students themselves. As indicators of student background, they used ethnicity, gender, location of school (regionally as well as by urban, suburban, or rural area), parents education, and the frequency with which English was spoken at home.
Despite their use of what they regard as an improved methodology for measuring family background characteristics, Peterson and Llaudet are reluctant to draw substantive conclusions as to causal relationships based on information from NAEP, the data set from which NCES results are drawn.
"The NAEP data collect information from students at only one point in time, so they are too fragile to be used for purposes of estimating the effects of public and private schools," said Elena Llaudet, a research associate at PEPG. "Our results are not offered as conclusive evidence that private schools outperform public schools but as a demonstration of the dependence of the NCES results on a questionable methodology," she added.
The three models used by Peterson and Llaudet differ in that they gradually delete the NCES variables for reasons discussed above, substituting new variables instead. Model I substitutes parents' education and the location of the school (regionally and by urban or rural area) in place of NCES's Title I and Free Lunch variables. Model II also substitutes the frequency of a language other than English spoken at home for the LEP and IEP variables. In addition to retaining the improvements made to Model II, Model III eliminates variables that may be influenced by the school the child attends -- absenteeism, the availability of a computer in the home, and number of books in the home. Since all three can be affected by school policies and relationships with parents, they are not appropriate for identifying an independent family influence. The attached table includes estimates from each model.
Full report and executive summary: http://www.ksg.harvard.edu/pepg.
The original NCES study:
FOR FURTHER INFORMATION:
Mark D. Linnen (617) 495-6954
Program on Education Policy and Governance (PEPG)
Table 1. The Advantage of Private School Sectors Relative to Public Schools, as
Estimated from NAEP Data, 2003.