Filed Under > Research/Publications
How to Determine ROI for Education Reform
Work by TC's Henry Levin and colleagues is helping to create a new science
Education reform is sometimes assailed for being an inexact science, particularly when it comes to weighing the costs and benefits of different interventions. But over the past decade that picture has been changing, in large measure due to the efforts of Henry Levin, William Heard Kilpatrick Professor Economics and Education, and his colleagues at TC’s Center for Benefit-Cost Studies (CBCSE). And with budgetary austerity, administrators are seeking ways to improve resource allocation in educational activities.
During the early 1970s, at the request of a Congressional committee headed by then-Senator Walter Mondale, Levin conducted the first major analysis of the financial impact on the nation of failure to graduate from high school.
In 2005, armed with vastly more sophisticated tools and methodologies, Levin repeated that exercise, and – together with colleagues at Columbia, Princeton and other institutions – calculated the costs to society, in terms of lost tax revenue and added burden on the health care, welfare and prison systems, associated with failure to graduate.
Since then, Levin and his colleague, Clive Belfield, Professor of Economics at Queens College, have founded the Center for Benefit-Cost Studies of Education and repeatedly demonstrated that significant return on investment could be achieved if proven strategies to boost graduation rates were implemented on a broad scale.
Now, backed by a $500,000 grant from the federal Institute of Education Sciences (IES) to undertake a project entitled “Cost-effectiveness analysis of educational alternatives,” the Center is making its methods widely available for other researchers.
The goal of the IES-funded project is to demonstrate how to conduct cost-effectiveness analysis in education and to promote its use among policymakers to improve the productivity of resource allocation decisions. Eight people are working on the project, including Levin, Belfield,
Fiona Hollands (Associate Director of CBCSE and 2003 alumna of TC Politics and Education Ph.D. program), Henan Cheng (post-doctoral fellow and alumna of TC), doctoral students Brooks Bowden, Robert Shand, Yilin Pan, and Barbara Hanisch-Cerda, a master’s degree student recently accepted into the doctoral program.
The project has proceeded in four stages:
Cost-effectiveness Analysis of Programs that Improve the High School Completion rate. The goal of this study was to conduct cost-effectiveness analysis of dropout prevention programs that have been proven effective and have evaluations accepted by the What Works Clearinghouse. CBCSE completed this report in October 2012. The study reviewed five programs that had a positive impact on the high school completion rate, including one that targeted low-income but academically promising students still in school and four that targeted youth who had already dropped out of school. The former proved to be far less costly (about $3,400 per student and $31,000 to get an extra high school completer above and beyond the number of completers in the control group) than the latter four programs ($14,000 - $22,000 per student, and $70,000 - $195,000 to get an extra high school completer above and beyond the number of completers in the control group).
“While not surprising that dropouts are more expensive to graduate from school than students still in school, the magnitude of difference in cost-effectiveness ratios was surprising,” said Hollands.
CBCSE Cost Tool Kit. CBCSE is developing a set of resources that constitute a Cost Tool Kit to facilitate the collection of cost data for educational programs using the “ingredients method” originally developed by Levin in 1975. The goal of the ingredients method is to identify all resources required to implement an intervention, including detailed information on personnel, facilities, equipment and materials. Currently in alpha version, the Cost Tool Kit includes spreadsheets that allow the user to list all ingredients required for an intervention being evaluated and, if necessary, to make adjustments for inflation, geographical location, and time pattern of investment and returns. A user manual includes an example of cost analysis to illustrate use of the Cost Tool Kit. The Database of Educational Resource Prices can be used to find national prices for the most common ingredients used in educational interventions.
CBCSE is inviting other researchers, graduate students and policymakers to pilot the Cost Tool Kit and provide feedback. The Center also is applying for funding to help create a beta version of the tool that can be disseminated widely to other researchers, program evaluators and education-decision-makers.
Cost-effectiveness analysis of early literacy programs. This study, still in process, involves a cost-effectiveness analysis of seven early literacy programs that have proven effective in addressing one or more early literacy outcomes and that also have had recent evaluations accepted by the What Works Clearinghouse. Professor Michael Kieffer, a faculty member at NYU and former faculty member at TC is serving is assisting in judging literacy outcomes.
Guidance Document. The last product due to be completed in June is a guidance document for researchers and evaluators in how to incorporate cost analysis into evaluation studies in order to facilitate accurate and timely cost-effectiveness analysis that can be used by education decision-makers when making budget allocations across programs.
The CBCSE team will be speaking about their work at conferences throughout the spring. In January, they presented their work to staff from the Education Unit at the federal Office of Management and Budget (OMB) as well as to staff at IES. The presentation was followed by a lively discussion precipitated by Kathy Stack, Head of the Education Unit at OMB, on how to promote the use of cost-effectiveness analysis among government agencies and policymakers. CBCSE has also been invited to participate in a meeting of leading research experts and federal officials to discuss next steps in federal and philanthropic efforts to establish a “common evidence framework,” a common set of evidence definitions, and standards aiming to strengthen investments in evidence-based programs and approaches.
Meanwhile, a soon-to-be released study by Levin and professors at Uppsala University finds that Sweden's privately financed, free public schools (similar to charter schools in the U.S.) risk creating increasing segregation.
Published Monday, Mar. 18, 2013
