Overview: Using Data Effectively

Put simply, data inform decisions. In the current educational context of accountability and school reform, data-informed decisionmaking is increasingly seen as an essential part of the educational leader’s toolkit. Because NCLB requires research-based practices and high performance standards, timely, pertinent, accessible and understandable data is crucial to leaders as they make decisions that will result in increased student achievement. And as leaders consider strategies for school improvement, data are the cornerstone of making decisions related to equity and excellence.

The primary use of data at the school level is to improve teaching and learning. The school is where the” rubber hits the road” for data use. Assessment and other data have to be collated, displayed, analyzed, and linked to actions and interventions that will improve teaching and learning based on student needs. Districts have an important role in supporting schools data use. Schools that use data well have a collaborative culture for looking at data, have basic assessment literacy – teachers and administrators know what the data mean, they can analyze data, discover the “root causes” for low performance, examine instruction and develop action plans that will result in increased student learning. Successful schools have time to collaboratively analyze data and develop a culture of inquiry that is part of the core work of the school.

1. Essential Elements

According to Data Informed Leadership in Education (Knapp et al., 2006), at least five phases of activity connect data to learning improvement:

1. Focusing and (re)framing problems for inquiry. Theories of action help leaders focus attention on problems and frame them to invite inquiry.
2. Accessing or searching for data and evidence. Leaders either use available data or generate data using inquiry or action research. In ideal conditions leaders have easily accessible web-based data.
3. Making sense of data and its implications for action. With data in hand, leaders create occasions for making collective sense of the data and probing the data for possible action implications.
   
4. Taking action and communicating the action in different arenas. Informed by the sense they make of the data, leaders take action and communicate what the data say to stakeholders.
   
5. Data become an integral part of the leaders’ actions and communications, and so a central part of the leaders’ work is “making it public” in ways that are respectful and politically astute.
   

According to a RAND study (Marsh et al., 2006), data are used to make decisions related to:

• Setting and assessing progress toward goals
• Addressing  individual or group needs
• Evaluating the effectiveness of practices
• Assessing whether client needs are being met
• Reallocating resources based on outcomes
• Enhancing processes to improve outcomes.

According to the same RAND study, data are used when:

• Data are readily accessible
• When education leaders believe the data accurately reflect student achievement
• There is a motivation to use data
• The data come in time to make important decisions
• Data users have the training and skill to analyze data and make appropriate adjustments.
• There is strong system or school support to use data and create a culture of data use.

2. Promising Practices

In Datawise in Action (Boudette and Steele, 2007) the story of West Hillsborough Elementary School in California shows how a school can go “deep into data.” Although the principal and teachers had been working with data for years, it was not fully integrated into the core work of the school. And they had not been taking advantage of all the different kinds of data available to them. When they looked at the color coded student achievement results, although many students were meeting or exceeding standards, for many students it wasn’t by much. As a new principal took over the reins of the school, they found themselves ready to “reframe the school’s central challenge not as one of meeting performance goals on the state assessment but as one of helping every child in the school live up to his or her full academic potential.”

The school started by agreeing to narrow the focus of their improvement efforts while broadening their definition of what kinds of data could be used. They already had a school improvement process that involved “identifying student or classroom needs, defining a specific focus to address those needs, setting goals, designing a plan, putting that plan into action, analyzing outcomes, and determining the next steps to be taken.”

School staff identified the “disparate elements” that the school already had in place that could be used more effectively. These included interim assessment data, following a small group of students, and creating personalized education plans for all students. Based on dips in performance on the state assessment in language arts, improving performance became the school wide goal. Grade level teams analyzed their state assessment data and the results of the interim tests. Each grade level identified language arts skills to focus on based on their students’ results. Examining data from the benchmark summary sheets allowed each teach to tailor instruction to the unique needs of his or her students. This included whole classroom instruction, small group work and individual work.

In addition to analyzing interim assessments a second strategy was to identify a few low-performing students in each classroom to track closely as a “barometer” of classroom learning. These students provided a starting point for discussions at school and grade level meetings. They also decided to use data to help students inform their own learning. Personal education plans for students had been developed among the teacher, student and his or her parent. They added a new data element, a student portfolio of work that would help make students more self-directed in their own learning. So, these three strategies, closer scrutiny of student test results on interim assessments, tracking a small number of students as a barometer of classroom student learning, and engaging students and examples of their work in developing their own educational goals may have made a difference. Using SchoolDataDirect.org to check on student achievement in West Hillsborough, 92% of students met proficiency in language arts and 96% met proficiency in mathematics in 2007.

3. Critical Questions

• Do school leaders have the “right data” to analyze?
• Is it easy to access and readily available?  (Is there a district data information system?)
• Do school leaders have data literacy?
• Do teachers have data literacy?
• Does your district support schools by providing data and training in data use?
• Has your district and the school leader created a “culture of inquiry?”
• Do schools have a process to use data (such as that described in West Hillsborough)? A district or state school improvement planning process?
• Is there collaborative time for data analysis and planning?
• Do school leaders and teachers know how to adjust instruction based on data?
• Does your district provide information on research-based interventions that schools can use?

Bernhardt, Victoria L. (1994). The school portfolio: A comprehensive framework for school improvement. Larchmont, NY: Eye on Education. 978-1883001643. Summary: A school portfolio is an effective self-assessment tool that exhibits a school’s goals, progress, achievements, and vision for improvement. This book demonstrates how to develop a school portfolio tailored to a school’s continuous improvement efforts, using examples from schools that have successfully implemented the tool.

Boudett, Kathryn P. and Steele, Jennifer. L. (2007). Data wise in action: Stories of schools using data to improve teaching and learning. Cambridge, MA: Harvard Education Press. Summary: A sequel to 2005’s Data Wise: A Step-by-Step Guide to Using Assessment Results to Improve Teaching and Learning, this book provides concrete examples and suggestions for how to implement the Data Wise process of using assessment results to improve instruction and learning. Case studies of eight schools that have implemented the improvement process offer insight into the challenges faced and strategies used in rolling out the process. The examples demonstrate how these schools adapted the process to fit their particular needs and illustrate how the Data Wise culture can begin with one school leader and spread outward.

Cooley, V.E., Shen, J., Miller, D., Winograd, P.N., Rainey, J.M., Yuan, W., & Ryan, L. (2006). Increasing Leaders’ Capacity in Data-based Decision-making: State Level Initiatives in Ohio, New Mexico and Michigan. Educational Horizons, 85 (1), 57-64.  Summary: This report highlights three state-level initiatives around data-based decisionmaking.

Mason, S. (2002). Turning data into knowledge: Lessons from six Milwaukee public schools. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA. Summary: This paper reviews a two-year project designed to increase the capacity of six Milwaukee Public Schools to use student, classroom, and school data in their decision making processes. The researchers learned that data must become an active part of school planning and improvement strategies, and integrated and accepted into the school culture and organization. The challenges that schools confront as they build their capacity for data-driven decision-making include: cultivating the desire to transform data into knowledge, focusing on the process of data use, committing to the creation and acquisition of data, organizing systems to manage and analyze data, and applying results to improve student achievement.

NEA Foundation for the Improvement of Education. (2003). Using data about classroom practice and student work to improve professional development for educators. Washington, DC: NEA Foundation for the Improvement of Education. Summary:In this paper, the authors examine how data analysis can help guide decision-making about professional development, review the capacity issues that schools and districts must address before they can use data for professional development purposes, and provide a list of resources and guiding questions regarding the appropriate use of data in education.

Schmoker, M. (1999). Results: The key to continuous school improvement (Second Edition). Alexandria, VA: Association for Supervision and Curriculum Development. Summary: This book provides information to support educators in using data to improve student learning, including setting goals, working collaboratively, and keeping track of student-achievement data from many sources. The author includes information on standards and assessments, effective professional development programs, and several examples of successful practices in schools and districts.

Supovitz, J. & Klein, V. (2003, November). Mapping a course for improved student learning: How innovative schools systematically use student performance data to guide improvement. University of Pennsylvania Graduate School of Education: Consortium for Policy Research in Education. Summary: This report develops a framework to help educators build more efficient and effective systems for data collection and analysis, and explains how school leaders and teachers can use student performance data to guide instruction and decision making.

Wiggins, G. P. (1998). Educative assessment: Designing assessments to inform and improve student performance. San Francisco: Jossey-Bass. 978-0787908485. Summary: This book covers all aspects of assessment design, including how to craft performance tasks that meet rigorous educational standards, how to score assessments fairly, and how to structure and judge student portfolios. It also looks at how performance assessment can be used to improve curriculum and instruction, grading and reporting and teacher accountability.

Williams, T., Kirst, M., Haertel, E., et al. (2005). Similar students, different results: Why do some schools do better? A large-scale survey of California elementary schools serving low-income students. Mountain View, CA: EdSource. Summary: Report explains why some California elementary schools, serving mostly low-income students, score significantly higher on the state's academic performance index (API) than schools with similar student populations. In particular, this report indicates that there is a high correlation between districts and school administrators who use student assessment data to improve instruction and learning, and higher-performing schools.

Reeves, Patricia L. and Burt, Walter L. (2006). Challenges in Data-based Decision-making: Voices from Principals. Educational Horizons, v85 n1 p65-71 (ERIC Document Reproduction Service No. EJ750644). Summary: Principals in the information age need to be information driven, committed to shared leadership and relentless about continuous improvement. They must reshape the processes, norms, and behaviors of teaching and learning around aggregating and interpreting shared information. In this article, the authors discuss the role of a principal in shaping the focus of a school and employing data-based decision-making strategies in school. Implications for district support and response in data-based decision-making processes are discussed.