Data Inquiry and Analysis for Educational Reform.
by Wade, Howard H.
Citizens and policymakers alike, as part of the new accountability,
expect schools to justify the value and effectiveness of their programs.
School boards, in turn, routinely ask to see the data administrators use
to guide decision-making in schools. Boards are mindful, too, that allocation
of state and federal funds often necessitates documented evidence that
school programs lead to verifiable improvements in student achievement
Apart from public relations and accountability issues, educators have
come to recognize that they can no longer rely on "intuition, tradition
or convenience" in making decisions about the best strategies to improve
student learning (NCREL 2000). For all these reasons, more schools across
the country are settling on the idea that carefully collected and analyzed
data represent the key to improvement in education.
This Digest outlines the most useful types of data to drive the process
of school improvement, the steps that must be taken to collect and analyze
the data, the role of administrators in guiding the data-driven reform
process, and the results that can be expected.
WHAT ARE THE EDUCATIONAL USES OF DATA?
Statistical data on school programs and student performance provide
educators with their only real evidence of the success or failure of educational
programs. Data "identify the link between teaching practices and student
performance so that high achievement levels can be obtained" (Miller 2000).
When systematically collected and analyzed, data provide an accurate
way of identifying problem areas in school programs. Data reveal strengths
and weaknesses in students' knowledge and skills, and they provide meaningful
guidance on how teaching practices can and should be altered. When acknowledged
and accepted by a faculty, data can lead to the formulation and implementation
of corrective courses of action that can solve problems and meet a school's
goals. Once improvement strategies are under way, educators can continue
to analyze the data to monitor and refine their efforts.
From a wider perspective, data can "provide an honest portrayal of the
district's and school's climate" (NCREL). Data can give a clear profile
of a district and the schools within it: Who are our students, teachers,
and families? What trends, attitudes, abilities, and values do they exhibit?
What outside factors influence them?
WHAT TYPES OF DATA ARE MOST USEFUL?
The profile of a school or an educational program can include at least
four types of data:
* Student assessment data include measurements of student performance,
such as standardized test results, grade point averages, and standard and
other formal assessments, both state and federal.
* Student demographic data describe such things as enrollment, attendance,
grade level, ethnicity, gender, family background, and language proficiency.
They may include facts on student mobility, modes of transportation, parent
involvement, student behavior, and social problems.
* Perceptions data document how a district or school is perceived not
only by its students, teachers, and parents, but by the community at large.
While this type of data is the most subjective-collected by questionnaires,
surveys, interviews, and observations-it can often prove crucial in determining
the direction a school may take in its programs and learning strategies.
* School program data define programs, instructional strategies, and
classroom practices. This type of data is most useful in monitoring, refining,
structuring, and redirecting school programs. In collecting these data,
"educators must systematically examine their practice and student achievement,
making sure both are aligned with specifically defined, desired student
outcomes" (Bernhardt 2000).
HOW CAN A SCHOOL BEGIN USING DATA?
The process of using data to improve school programs must be supported
not just by one or two administrators, but by as many faculty members as
possible. Data inquiry must grow out of a common recognition of the potential
benefits statistical data can have in helping to achieve common goals.
Data analysis can be used at various levels within a school. Individual
teachers can use it to improve teaching strategies in their classrooms.
Groups of faculty can use data analysis to amend areas of concern within
a department. Finally, the entire school can use data analysis to reform
and improve the educational climate.
Although variations exist, depending on personnel and circumstances,
educational reform through data analysis involves four steps.
First, the school staff identifies areas of concern for which improvement
is desired. By gathering and analyzing existing assessment, demographic,
perceptual, and program data, staff members create a profile of the school.
This profile will reveal a school's strengths and weaknesses, and will
allow interested faculty members, administrators, parents, and community
members to ask appropriate questions regarding student achievement, teaching
practices and programs, and school culture. Data collected during profiling
can sometimes be used as baseline information against which subsequent
improvement strategies can be measured.
Second, after the school profile is complete, interested teachers, administrators,
and community participants meet together as a leadership team to prioritize
the areas of concern that the profile has revealed. This is best achieved
before the school year begins, and before faculty become immersed in the
day-to-day challenges of teaching.
Third, once a problem is selected for correction, the leadership team
collects and analyzes any additional data that might elucidate and suggest
solutions to the problem. The team then outlines strategies for improvement
and defines evaluation criteria against which the courses of corrective
action will be measured.
Finally, after plans of action have been implemented and completed,
the team again collects data to measure the success or failure of their
efforts. The corrective strategies may be discontinued or adjusted and
reapplied as needed.
Schools without experience implementing data analysis methods should
start with simple problems, then advance to more complex issues only as
benefits are witnessed and confidence gained. As well, it is important
that the faculty commit to implementing the leadership team's recommendations,
because without such commitment efforts for reform might end before benefits
For many schools, the appointment of a school-based facilitator or "data
analyst" may be desirable. The analyst advises the leadership team in its
collection and analysis of data, and in implementing corrective projects.
In particular, the analyst helps "schools focus on their own particular
problems, set locally appropriate goals, identify a course of action, and
assess progress toward these goals" (Killion and Bellamy 2000).
Selected from the faculty to encourage trust and confidence among staff
for the process, the data analyst must be trained in data analysis, interpretation,
and display processes. Because analysts are on-site, they can provide teachers
and principals with timely responses and assistance (Killion and Bellamy).
HOW CAN ADMINISTRATORS OVERCOME BARRIERS AND HELP THEIR SCHOOLS USE
Despite the arguable benefits of data analysis, some educators remain
ambivalent about its usefulness. No doubt a few educators cling to the
antiquated notion that data collection is unprofessional. It certainly
is true that, as Bernhardt notes, many educators "lack the training, equipment,
and time to develop and carry out complex analyses." Last but not least,
a fear often exists that data, if publicized, might somehow expose incriminating
inadequacies and/or incompetence.
Principals and other administrators can best support data-driven reform
processes by providing vision and leadership. Although they need not be
proficient themselves in sophisticated data-gathering and data analysis
techniques, they should be acquainted with the field, and should respect
and value data analysis as an increasingly important tool in education.
Administrators do not necessarily need to direct the data collection
and data analysis process, but they must support it. They must be able
to assure the more skeptical participants that it is being undertaken not
to blame or accuse, but to give both teachers and students the necessary
tools to succeed.
Because administrators often possess a broader view of the issues that
confront a school or district, they can help select the best, most easily
measurable, results-oriented goals and initiatives. They can ask questions
about student achievement and teaching practice, and can propose menus
of initiatives based on the acquired data. Finally, they can guide staff
development based on which initiatives have worked and which have not,
according to analysis of the data.
WHAT RESULTS CAN BE EXPECTED FROM DATA INQUIRY AND ANALYSIS?
Clearly, data analysis is not a panacea that will solve every problem
in a school. But when properly focused and implemented, data analysis is
one tool that a school's staff can use to help raise educational achievement,
thereby increasing confidence among faculty, students, and the community
regarding the effectiveness of public education.
In learning to incorporate data analysis as a regular part of their
professional activity, teachers become more reflective about their teaching
practices, less reactive, less willing to accept easy answers, and more
open-minded to solutions based on the data they gather. As a whole, the
school assumes a more professional and civil culture of inquiry, in which
"teachers share with each other important questions and ideas related to
teaching and learning" (Feldman and Tung 2001).
Through program improvements brought about by data analysis, a higher
level of achievement can also be expected of students. In some cases, students
have even begun emulating the practices of data inquiry they see their
teachers modeling, conducting their own student surveys and analyses (Feldman
Bernhardt, Victoria L. "New Routes Open When One Type of Data Crosses
Another." Journal of Staff Development 21, 1 (Winter 2000). http://www.nsdc.org/library/jsd/bernhardt211.html
Calhoun, Emily F. How To Use Action Research in the Self-Renewing School.
Alexandria, Virginia: Association for Supervision and Curriculum Development,
Chrispeels, Janet H., and others. "School Leadership Teams: A Process
Model of Team Development." School Effectiveness and School Improvement
11, 1 (March 2000): 20-56. EJ 611 222.
Du, Yi, and Larry Fuglesten. "Beyond Test Scores: Edina Public Schools'
Use of Surveys To Collect School Profile and Accountability Data." ERS
Spectrum (Summer 2001): 20-25.
Feldman, Jay, and Rosann Tung. "Using Data-Based Inquiry and Decision
Making to Improve Instruction." ERS Spectrum (Summer 2001): 10-19.
Holcomb, Edie L. Getting Excited About Data: How To Combine People,
Passion, and Proof. Thousand Oaks, California: Corwin Press, 1999. 162
pages. ED 433 373.
Killion, Joellen, and G. Thomas Bellamy. "On the Job: Data Analysts
Focus School Improvement Efforts." Journal of Staff Development 21, 1 (Winter
McLean, James E. Improving Education Through Action Research: A Guide
for Administrators and Teachers. Thousand Oaks, California: Corwin Press,
1995. 87 pages. ED 380 884.
Miller, A. Christine. "School Reform in Action." Paper presented to
the American Educational Research Association Conference, New Orleans,
April 28, 2000.
North Central Regional Educational Laboratory. Using Data To Bring About
Positive Results in School Improvement Efforts. Oakbrook, Illinois: Author,
Sparks, Dennis. "Results Are the Reason." Journal of Staff Development
21,1 (Winter 2000). http://www.nsdc.org/library/jsd/schmoker211.html