Tracking a Return on Investment

"It seems simple enough, when you think about it," stated Susan Berardi, one of the parents spearheading the effort. "When you invest in a company, you want to know how it performs over time, how well it's managed, and what kind of ROI it can provide for you. I look at a school system in much the same way, especially one that I'll be investing in for the next 20 or more years."

The team focused on one essential question: How can we measure, communicate, and ultimately improve the ROI of a public school system? One answer came from the Pearl River School District in New York, the first school to receive the Malcolm Baldrige National Quality Award for education excellence in 2001. After analyzing Pearl River's comprehensive performance indicators, the team realized that by focusing primarily on student test scores as the measure of success, Reading schools and the Massachusetts Department of Education were not telling the whole story.

The team decided to design a "dashboard" for measuring and communicating a diverse set of indicators that better reflect the totality of the school district's performance. The dashboard model, used by many corporations, is a means of assessing an organization's performance and viability. Like the dashboard of a car, it provides the "gauges" needed to determine the health and performance of an organization.

In developing the dashboard, the team chose to adopt the four corporate perspectives outlined in Kaplan and Norton's book, The Balanced Scorecard: Translating Strategy into Action (1996): 1) Financial, 2) Customer, 3) Internal Processes, and 4) Learning and Growth. Together, they identified a list of performance indicators that address these areas from a public, academic model. For example, in the same way a corporation measures its financial performance by ROI and the economic value-added, Reading schools might measure its financial performance by annual cost per pupil, program expenditures as percent of overall budget, and actual spending as a percentage of the Massachusetts foundation budget and compared to other academically successful school districts.

Similarly, where private companies focus on internal processes like quality and response time, Reading might focus on its efficiency with instructional technology, special education, and educational design and delivery. To measure learning and growth, the administration and community could monitor trends in faculty and staff satisfaction and turnover, and professional development.

Perhaps the most challenging task facing the dashboard team in demonstrating ROI was developing performance indicators to reflect the customer perspective. Just as successful private businesses focus on customer satisfaction, customer retention, and market share, Reading schools must define and build strong customer relationships for future growth.

A school's direct customers are parents, citizens, colleges, and businesses because they are the ones who are either paying the bills or recruiting and hiring the graduates - the children of Reading. The performance and reputation of the school system are critical elements in both arenas.

For the last several years, the Reading Public Schools have enjoyed an excellent reputation in the Greater Boston area, with MCAS scores averaging in the top 10% of the state. Students graduating from Reading High are accepted to some of the most prestigious colleges in the country. Reading, however, is highly dependent upon the state to supplement local revenue. In addition, the town is primarily residential, so taxpayers shoulder a large portion of the town budget. As a result, the administration has had to restrict spending over the years, with total pupil spending ranking less than 85% of other Massachusetts districts. However, Reading's average residential tax bill is greater than that in 85% of other communities in the Commonwealth.

It was clear to the team that the people in Reading care about their schools and show their support in many ways. Volunteerism is high and fund raising efforts, based on historical estimates, bring in over $500,000 annually, equating to almost $100 per student. In addition, while many other towns across the state declined to support Prop 2½ override referendums, in 2003 alone, Reading passed three property tax override initiatives supporting the schools, including a $54.3 million high school renovation project. The Reading community volunteers are a significant asset to the school system and they somehow needed to be represented on the dashboard.

As a fifth indicator, the team chose to use "social capital," a term coined by James Coleman, professor of social psychology at the University of Chicago. Coleman studied how differences in family backgrounds affected student achievement. He defined social capital as the resources that reside in the social structure itself, including the norms, social networks, and relationships between adults and children that are valuable for the child's "growing up." While social capital exists within the family, it also resides outside the family, in the community at large. In his landmark studies in the 1980's, social capital was shown to influence and support both the family and the school system in educating the children.

The customer perspective in the Reading dashboard had to be represented by more than standard measures like SAT achievement, diploma rates, and AP course participation, although those are important. It needed to include various measures of Reading's social capital. With demographic statistics (e.g., number of two-parent families, resident college attainment, home owners vs. renters) and new measures in areas like parent volunteer hours, total dollars raised by parent groups, and total dollars spent on enrichment, extra-curricular, or other educational arenas, the dashboard will shed a new, easily quantifiable light on the level of social capital the community contributes to the schools .

Parent satisfaction is also an important part of the Reading dashboard. With no formal process available for collecting parental feedback on the schools, teachers, and curriculum, the team decided to administer a parent satisfaction survey to garner more information. With the generous help from professional surveyor Mike Sokol, of Lokos Consulting, the dashboard group worked to develop an online survey for all of Reading's middle school parents. To develop the survey, volunteer parents from the two middle schools formed a focus group to identify and prioritize key issues, and from that input, developed a customized list of questions.

"By involving parents in the development of the survey questionnaire, we were able to identify the specific issues important to middle school parents and their impact on customer satisfaction," commented Deborah Gilburg, a member of the dashboard team who facilitated the survey focus group. "We also have a customized prototype survey that, in the future, can be replicated for elementary and high school parents as well." Administration at the Reading schools also hopes to solicit satisfaction data from teachers, students, alumni, and college admission officers in the future.

The team decided to restrict the initial number of measures for the Reading dashboard because the effort required to gather, analyze, and present the information in a user-friendly manner is extensive. In addition, the whole project had to be presented in a prototype form to build understanding and support for the project from the school districts' various constituencies. The prototype dashboard was recently made public (June 2004) at ReadingDashboard.org.

"The prototype only offers some indicators for the perspectives we explored," explains James Mulvey, the team's IT specialist. "However, it gives everyone a good idea of how powerful a communication tool it can be." The website also contains two news columns that will change every quarter: Spotlight on Alumni and the Learning Series, where different curriculum-based topics will be reviewed. The team is now collecting feedback. The prototype is merely a first step, and the hope is that the community will respond with questions and suggestions that will help move the project forward in unanticipated and beneficial ways.

There are risks involved in becoming so public about the areas of the school system that are in need of improvement. It might seem more prudent to be private about these things. On the other hand, the dashboard is helping Reading identify its unique competitive advantage. Right now we are ranked 24th in MCAS scores and 242nd out of 351 communities in per pupil spending. We have a town of active parents who give both their time and money, and a community that supports the schools. It is our goal that the dashboard becomes not only a communication tool for the schools, but a data driven reflection of our performance that can lead us to breakthrough achievements.

Dennis Richards is Associate Superintendent with the Reading Public Schools. He is a member of the MASCD Board of Directors and serves on MASCD's Issues and Advocacy Committee. He can be reached by calling 781-944-5800 or by email at drichards@reading.k12.ma.us. He wants to acknowledge all the help he received on this project and article from Susan Berardi, Deborah Gilburg, and James Mulvey, Reading Public School parents.

Coleman, James S.  "Social Capital in the Development of Human Capital:  
     The Ambiguous Position of Private Schools."  February 25, 1988.  New York, NY:  
     Presented at the Annual Conference of the National Association of Independent Schools.

Kaplan, Robert S. and David P. Norton (1996).  The Balanced Scorecard:  
     Translating Strategy into Action.  Cambridge, MA:  Harvard Business School Press.

Pearl River School District.  Pearl River, NY:  2001 Malcolm Baldrige National Quality 
     Award:  Education Application.