Data-Driven School Improvement. ERIC Digest.
by Johnson, James H.
Effective educators make effective decisions, decisions based on accurate
information. If knowledge is power, then studying the current abilities,
skills, attitudes, and learning styles of students empowers educators to
adjust the curriculum to achieve whatever goals the school and district
When educators study their schools and classes, they seek an answer
to an ageless question: Is it good because we've been doing it for a long
time, or is it good because we have tangible evidence of its worth (James
McLean 1995)? In many instances one must conclude the former because no
evidence exists to support the latter.
One solution is to collect data on student learning that are both timely
CAN DATA USE IMPROVE EDUCATION?
McLean contends that "implementation of a complete program of data collection
and use can lead to the improvement of education as has no other educational
innovation of the last century." Fundamental to this effort is equipping
teachers and administrators with the skills and inclination to ask, "Is
there a better way?"
Traditionally, data collected in schools have been used to assess student
performance. But with the growing decentralization of power in most districts,
educators are more likely to rely on data to help them make better choices
and uncover better ways of serving students and the community.
Educators routinely evaluate reading, writing, and math achievement,
but they rarely assess management skills, individual learning styles, or
other elements that may affect student achievement.
"We do the testing," said Marilyn Olson, of the Lane Education Service
District, "and we get the individual student's scores back, but there's
no concerted effort to interpret the data and work with them" (Johnson
Olson is currently involved with a project to create a database "that
is consistent and accurate enough so that we can use it to make program
changes." The goal is to enable educators to collect accurate information
on students so they can make adjustments to teaching styles or curricula
to gain measurable improvements. Students experiencing difficulty can be
identified and helped earlier than is currently possible.
"Right now," Olson says, "we've got different teachers dealing with
different data and they don't talk to each other, so a student slips through
and no one recognizes the discrepancies in the student's data."
WHAT TYPES OF DATA SHOULD BE COLLECTED?
Data are endemic in educational settings. As Richard Wallace (1996)
reports, "School districts usually gather much more data than they can
effectively use." The challenge is to analyze the information and use it
wisely. Many believe the logical starting point is to make better use of
existing or archival data (Wallace, Emily Calhoun 1994, McLean). These
data include statistics on attendance, grades, referrals, retentions, and
standardized-test results. When compiled and reported on a regular basis,
archival data provide a baseline of school operations and can be used to
make comparisons among similar schools.
With minimal effort, these data can often be disaggregated or broken
down. Often done by grade, gender, race, or socioeconomic background, disaggregation
can reveal previously unrecognized patterns that, in turn, may suggest
areas in need of improvement.
With varying degrees of effort, other classes of data can be collected.
Examples include survey results, interviews, numbers of books read, and
other information on student achievement. These data typically require
development of a means to collect and analyze the information (Calhoun).
Data that may be more difficult to collect and interpret objectively include
evaluations of student work, such as portfolios and exhibitions.
SHOULD TEACHERS SERVE AS RESEARCHERS?
Traditionally, educational researchers have been the ones to proclaim
various educational practices as either effective or ineffective. Principles
of effective instruction are often drawn from large-scale studies.
However, teachers do not work in large-scale, static settings. They
face unique sets of students who respond variably to different strategies.
Hence applying academic research findings to individual classrooms can
One alternative to overreliance on the findings of large-scale studies
is for educators to assume the role of researcher. By studying their students,
teachers can bring the academic findings down to earth, as it were, and
discover what works in their specific classroom.
Susan Black (1996) equates teacher research to the work done by anthropologists
or ethnographers. "They are able to observe the cultural scene closely...and
create a research record of the people, places, events and objects within
it, as well as their own personal interactions and responses." By using
the classroom as laboratory, teachers can, for example, learn whether interdisciplinary
teaching is as effective with their remedial students as with their high
Research conducted by teachers differs from academic research (Black).
First, academic researchers strive for objectivity, keeping the subjects
of their study at arm's length. Teachers, in contrast, maintain a close
relationship with the objects of their study. Academics try to design and
control events under study while teachers observe activities as they occur
in the real world, namely their classes. Finally, traditional researchers
seek to globalize results while teachers know that their findings apply
only to the students in their classes.
Such small-scale studies are often termed action research, because the
teacher-researcher takes action based on the results of his or her findings.
The research may be formal or informal, short- or long-term. But it is
based on the understanding that the results are collected from a set of
students and the action is applied to those same students (Calhoun, Molly
Watt and Daniel Watt 1991).
HOW CAN DATA BE USED EFFECTIVELY?
In any organization, those who possess information typically exercise
a degree of power or control over those who don't. How information is to
be used will affect how data-collection initiatives are perceived and then
Data can be used to judge people's performance and take punitive action
against underachievers, or it can be used to diagnose problems and determine
the efficacy of solutions.
Principals' "willingness to provide opportunities for information acquisition...may
be tempered by their competitive notions of power which only impede the
empowerment of teachers" (Peggy Kirby and Ira Bogotch 1989).
The concept of continuous improvement should be stressed. "Each data-collection
cycle and its results should not be thought of as an activity with a grade...it
should be thought of as information on the progress being made toward attaining
the collective goal and to assist all members of the organization as they
make decisions for current and future action," states Calhoun.
Choosing a specific area of focus is also crucial. Through consultation
with other staff members, whether formally or informally, the researcher
needs to select a well-defined set of objectives. The data-collection cycle
and subsequent action steps should be designed to address these needs.
When goals of the improvement effort are clearly defined, teachers, staff,
and the community will be more likely to understand and support the initiative.
WHAT STEPS CAN SCHOOLS TAKE TO IMPROVE THEIR USE OF DATA?
Most researchers suggest that a logical first step in using data is
to begin making better use of existing data. A systematic analysis of data
that are already being collected can reveal previously unseen patterns
and opportunities for improvement. More inventive data sources require
greater efforts to collect and interpret them.
Outside resources can support efforts to collect and analyze data effectively.
For example, the Northwest Regional Educational Laboratory administers
a variety of data-driven school-improvement programs that have been proven
effective (Robert Blum, Kim Yap, and Jocelyn Butler 1992). One recent innovation
is their "data in a day" program, wherein advisors, staff, and student
volunteers select an area of focus in the morning, collect data, and present
findings later the same day. The immediacy of the results and the involvement
of a range of individuals often provide tangible benefits to everyone involved,
especially the students.
Black, Susan. "Redefining the Teacher's Role." The Executive Educator
(March 1996): 23-26. EJ 519 765.
Blum, Robert E.; Kim O. Yap; and Jocelyn A. Butler. "Onward to Excellence
Impact Study." Paper presented at the annual meeting of the American Educational
Research Association, San Francisco, 1992.
Bredeson, P. "Redefining Leadership and the Roles of School Principals:
Responses to Changes in the Professional Worklife of Teachers." Paper presented
at the annual meeting of the American Educational Research Association,
San Francisco, March 27-31, 1989. ED 304 782.
Calhoun, Emily F. How To Use Action Research in the Self-Renewing School.
Alexandria, Virginia: Association for Supervision and Curriculum Development,
1994. ED 370 205.
Johnson, James H. Data-Driven School Improvement. OSSC Bulletin Series.
Eugene, Oregon: Oregon School Study Council, May 1996. 38 pages.
Kirby, Peggy C., and Ira Bogotch. "Information Utilization in Restructuring
Schools: The Role of the Beginning Principal." Paper presented at the annual
meeting of the American Educational Research Association, Atlanta, April
12-16, 1993. 18 pages. ED 359 646.
McLean, James E. Improving Education Through Action Research: A Guide
for Administrators and Teachers. The Practicing Administrator's Leadership
Series. Roadmaps to Success. Thousand Oaks, California: Corwin Press, 1995.
87 pages. ED 380 884.
Wallace, Richard C., Jr. From Vision to Practice: The Art of Educational
Leadership. Thousand Oaks, California: Corwin Press, 1996. ED 390 174.
Watt, Molly Lynn, and Daniel Lynn Watt. Teacher Research, Action Research.
The LOGO Action Research Collaborative. Report No. 91-4. Newton, Massachusetts:
Education Development Center, Inc., 1991. 23 pages. ED 341 686.