ERIC Identifier: ED334311
Publication Date: 1991-04-00
Author: Darling-Hammond, Linda - Ascher, Carol
Source: ERIC Clearinghouse on Urban Education New York NY.
Accountability Mechanisms in Big City School Systems.
ERIC/CUE Digest No. 71.
Accountability has always been a basic concept in public education,
although ideas about how to accomplish it have changed. In recent years,
the urgent need to improve big city schools has been a powerful incentive
to the adoption of new accountability systems. This digest explores the
strengths and weaknesses of various accountability tools, the use and misuse
of indicators, and ways to create genuine accountability at the school
TYPES OF EDUCATIONAL ACCOUNTABILITY
At least five types of accountability mechanisms may exist alongside
each other (Darling-Hammond, 1989):
Political. School board members and legislators must stand for election.
Legal. Boards and legislatures enact policies, and courts can hear complaints
Bureaucratic. State and district education departments set rules and
regulations to ensure that schools meet standards and follow procedures.
Professional. Teachers and other school staff must acquire specialized
knowledge, pass certification exams, and uphold professional standards
Market. Parents and students may choose the programs or schools they
believe are most appropriate for their needs.
Over the past few decades American schools have relied most heavily
on bureaucratic mechanisms for achieving accountability. Because professional
and market accountability tools are currently being proposed as strategies
for school improvement, it is important to understand what each of these
three offers and what its limits may be.
Bureaucratic accountability is based on the hope of finding the "one
best system" under which all students will be educated. Its strength rests
in its possibility to ensure equal and standardized education. Policies
are made at the top and handed down to administrators who translate them
into procedures for teachers to follow in educating students. This system
of accountability does not hold teachers accountable for meeting the individual
needs of their students; they can be held accountable only for following
Unlike bureaucratic accountability, whose goal is uniformity and standardization,
professional accountability allows practitioners to make their own decisions
about how to meet the educational needs of individual students. This model
seeks to ensure that teachers will be highly knowledgeable, competent,
and committed to good teaching. Consequently, a professional accountability
system must pay particular attention, not only to student progress, but
also to policies governing the preparation, certification, selection, and
evaluation of teachers and other staff.
Magnet schools and other choice plans are based on quasi-market mechanisms.
These are supposed to make schools more accountable in two ways. Because
"customers" choose schools, (1) schools are expected to work harder to
provide services that parents or students want; and (2) problems in undersubscribed
schools are revealed, which policymakers can then address.
Market accountability poses several difficult questions: How can all
citizens be guaranteed access to quality schools? How do students and their
parents make school choices? What information should schools provide so
that students and their parents can be informed consumers of public education?
Other than changing schools, what vehicles for communication and redress
can parents and students use when the schools do not meet their needs?
BALANCING DIFFERENT FORMS OF ACCOUNTABILITY
No form of accountability is sufficient by itself to ensure that all
students are well served. Because each form of accountability has both
strengths and weaknesses, a combination of tools is needed to make schools
responsible and responsive.
In school choice plans, for example, when the most desirable schools
are filled, the remaining schools, desirable or not, must serve the rest
of the students. If no other public policy mechanisms are in place to support
improvements in all schools, choice will not improve education across the
board. A different accountability mechanism is needed to regulate and ensure
equity. Similarly, if bureaucratic mechanisms are de-emphasized, greater
guarantees of staff competence and commitment would need to be provided
through professional accountability mechanisms.
Further, new approaches create new dilemmas that must be resolved. For
example, in cases of interdistrict choice, parents lose electoral accountability
at the district level when students leave the district in which they reside.
Thus, mechanisms must be devised to give parents a voice in their new schools.
Similarly, school-based management proposals call for local ownership
of education and make professionals at the school site primarily accountable,
based on the belief that better decisions will be made by those who are
closest to the situation. But in a system where significant authority devolves
to the school level, how much responsibility should state governments and
local districts still carry for students' treatment and achievement?
In deciding which aspects of education should be relegated to bureaucratic
accountability and which should be left to professional accountability,
it is useful to distinguish between equity and productivity concerns. Wise
and Gendler (1989) argue that equity issues, such as the allocation of
resources and guarantees of equal access, can and should be resolved by
higher units of governance, accountable to a wider public, and should not
be left to the decision of individual teachers or the parents of one school
However, since teaching, at its best, is a highly individualized process,
productivity questions cannot be solved well by bureaucratic regulation.
Uniform state- or district-level policy decisions about teaching methods
and school processes cannot meet the needs of varying school and student
circumstances. Thus, these needs are better addressed using professional
accountability mechanisms. At the same time, the broader goals of the public
require that states have some way of evaluating schools.
USING STATISTICAL INDICATORS
Given the complexities, it is not surprising that policymakers often
try to achieve accountability by the apparently easiest strategy: monitoring
students' test scores and sometimes linking teacher or school rewards and
sanctions to such measures. But accountability also encompasses how a school
or school system hires, evaluates, and supports its staff; how it relates
to students and parents; how it ensures that the best available knowledge
will be acquired and used; how it evaluates its own functioning; and how
it corrects its problems and provides incentives for continual improvement.
Thus, performance indicators do not themselves create accountability.
At best, test scores may provide data for accountability systems that enable
schools to improve and correct problems. At worst, they may deflect attention
from needed school changes. In fact, improperly designed performance indicators
can undermine accountability by creating incorrect assumptions.
CRITERIA FOR SELECTING INDICATORS
Because statistical indicators are used to support broad inferences
about schools and students, policymakers developing indicator systems should
consider carefully what they want to measure, how they can best measure
it, and what other information they need to interpret trends intelligently.
Oakes (1986) and Koretz (1989) maintain that indicators should offer
information that is:
Problem-oriented. Indicators should be able to detect current problems
or potential difficulties.
Relevant to policy. Information should be described in a way that is
amenable to change by policy decisions or that shows policymakers where
to target their efforts.
Reflective of educational outcomes. The data might include graduation
or dropout rates; college attendance rates; voting rates; achievement test
scores; writing samples; and quality of participation in science fairs,
debates, or dramatic productions.
Indicative of student backgrounds. Indicators cannot be interpreted
meaningfully without corollary information on which students are involved.
Furthermore, if a school's student population is highly mobile, test scores
intended to measure student growth at different points in time may not
even be measuring the same students.
Illustrative of school context. Data on how schools are organized and
what they provide are needed to offer clues about why schools achieve the
outcomes that they do.
Oakes (1989) suggests that three interrelated variables are important
to school achievement, and that each can be tapped with a set of indicators.
The first, access to knowledge, is the extent to which schools give students
opportunities to learn various domains of knowledge and skills. The second,
press for achievement, is the mix of incentives--graduation requirements,
program rigor, recognition of accomplishments--that schools provide students
to work and achieve. Third are the professional teaching conditions--salaries,
work load, time for planning--that can empower or constrain teachers as
they create and implement instruction.
CRITERIA FOR INTERPRETING INDICATORS
The greatest danger of indicators is the ease with which they can give
false impressions because they are misunderstood or interpreted in invalid
ways. The most common misunderstandings derive from the use of average
student test scores to indicate the quality of schools or school districts.
The danger is especially relevant when the stakes are high. When rewards
or sanctions are automatically triggered by test scores, incentives are
created not to improve teaching and learning but to push low-scoring students
out of the school, or at least out of the test score count (Darling-Hammond,
1990; Haney & Madaus, 1986).
Validity. If averages hide information about individual students, interpretations
drawn from average test scores will be invalid. Similarly, test items are
meant to be samples of domains of learning, and the assumption that student
performance on the test represents the broader curriculum goals is negated
if teachers "teach to the test" (Koretz, 1989).
Reliability. To be reliable, a test score must be free of random errors
of measurement. That is, if statistics on a school's curricular offerings
are likely to vary, depending on who is collecting the data or how they
were collected, the scores are not a reliable indicator of student progress.
Corruptibility of Indicators. Because the emphasis on basic skills test
scores has prompted teaching to the test, many students spend less time
on untested subjects, such as science, social studies, and the arts. The
scores, therefore, no longer necessarily indicate students' general achievement.
Emphasis on basic skills can also mean that the scores may no longer provide
comprehensive assessments of student' ability, even in the tested subjects,
because classwork narrowly oriented toward a test does not heighten students'
proficiency in aspects of the subject not tested--analysis, complex problem-solving,
and written or oral expression.
Indeed, since about 1970, while test scores on basic skills have improved,
scores on assessments of higher-order thinking skills have declined in
virtually all subject areas. Many observers, including the National Research
Council and the National Councils of Teachers of English and Mathematics,
argue that the overuse of multiple choice basic skills tests has actually
corrupted teaching practices. Aspects of the subjects that are not tested--especially
higher-order thinking and performance skills--are left untaught.
Test scores are not the only indicators that can be corrupted. Indicators
of student course-taking, for example, may prove to mean something other
than what policymakers intended. When pressure is great to meet new standards,
schools with poor resources or shortages of qualified teachers may implement
new science requirements quite differently from wealthy schools. In one
school, three years of science may mean three years of general science
lectures; in another school, it may mean a rigorous sequence of biology,
chemistry, and physics classes replete with laboratory experiences.
MAKING FAIR COMPARISONS
Accountability systems often compare schools and school districts and
increasingly connect rewards and punishments to these comparisons. Thus,
it is important to set up comparisons fairly. This means that school outcomes
cannot be compared without also comparing populations, resources, and educational
goals. For example, schools serving low-income students often have far
fewer highly experienced and qualified teachers and offer many fewer advanced
courses than wealthier schools do. Comparisons of test scores that ignore
these factors hold little promise of directing policymakers' attention
to real problems.
One way to create fair comparisons is to develop measures of how much
each individual student has learned over a period of time. Another approach,
employed in California, is to compare schools with similar student populations
and mobility patterns, and a wide range of other similar characteristics.
CREATING ACCOUNTABILITY AT THE SCHOOL LEVEL
An accountability system is a set of commitments, policies, and practices
increase the use of good educational practices;
reduce the use of harmful or wasteful practices; and
create internal mechanisms to identify, diagnose, and change courses
of action that do not lead to learning.
Accountability is achieved only if a school's policies and practices
work to provide good education and correct problems as they occur. Performance
indicators and ongoing diagnostic processes are needed to evaluate whether
these conditions are being met. These processes must include methods for
changing school practices if they are not working well on behalf of students.
Until now, most indicator systems have been defined by states, and schools
or districts have often had to shift curriculum priorities in order to
succeed according to the states' systems. Often, this has meant de-emphasizing
areas and modes of learning more focused on problem-solving, higher order
thinking, and complex performance in favor of lower level cognitive skills.
If performance measures are to support meaningful accountability, however,
they must assess and encourage valuable kinds of learning for students.
For such tools to be useful at the school level, they must illuminate how
individual students are progressing, and they must be accompanied by methods
for figuring out how changes in practice might improve learning. At the
school level, where good education is defined as meeting the needs of individual
students, arguably the most important form of accountability is professional
accountability, which supports responsible teacher decisionmaking. A major
aspect of professional accountability is the effort to establish an inquiry
ethic and a commitment to collective problem-solving. In a few schools,
this kind of collective questioning and reflection is frequent. More commonly,
teacher isolation, combined with centralized decisionmaking and evaluation,
has removed occasions as well as incentives for this kind of activity at
the school level.
Yet, if schools are to become more responsive and open to change, they
must find ways to make evaluation and assessment part of their everyday
activities. Teachers must have opportunities to profit from their colleagues'
knowledge, experience, and perspectives on behalf of their students.
CREATING INDICES THAT ENHANCE SCHOOL ACCOUNTABILITY
Measures of student learning, as noted above, can be misleading. Aside
from questions of validity and reliability, most standardized tests are
not sensitive to differences in curriculum offerings and thus do not really
assess the opportunities for learning that are offered in various schools
(Madaus et al., 1979). To elicit a better measure of a school's effectiveness,
some states have constructed curriculum-based tests that reflect more precisely
the variations in school teaching.
A number of states and schools in many parts of the country--Vermont,
New York, California, to name a few--are engaged in creating performance-based
assessments of student learning. These approaches include essay examinations,
portfolios of students' best work in various subject areas, and group projects
that require analysis, investigation, experimentation, cooperation, and
written, oral, or graphic presentation of findings. These assessments require
students to think analytically and demonstrate their proficiency as they
would in real-life situations (Archbald & Newmann, 1988). Since these
are new efforts, there is, of course, insufficient experience to pass judgment
on them. Based on the early experience, these approaches appear to be highly
motivating for students.
To inform school decisionmaking and improve practice, measures of school
performance must accompany assessments of student learning. Indicators
of the school's performance--based on systematic observations by faculty,
and information from surveys of students, parents, and staff--should always
start with questions about students: attendance and sense of connection
to school; self-esteem; and civic and social growth, as well as academic
achievement. These indicators should reflect factors that structure students'
experience of schooling. For example:
Who teaches what to whom?
What are the opportunities for each child to succeed and build on strengths?
How many children are able to participate in extra- or co-curricular
What is the likelihood that each child will have an adult advocate at
the school who knows him or her well?
How is class time used? What intellectual material do students encounter?
Do these opportunities vary by classroom, group, or track?
School faculty can use the findings to make well-grounded decisions,
identify areas to change, and establish benchmarks for tracking progress.
GOING WITHIN AND BEYOND THE SCHOOL AS A UNIT
Although school-based analyses can provide rich data, for some educational
issues the school is either too small or too large a unit for analysis.
In schools with tracks and other educational groupings, it is important
to know not only whether a school offers calculus, for example, but which
and how many students can take this course, and what the other math course
options are. In the same vein, while a school may contain computers, their
use and availability may differ depending on the track. To answer questions
of equity, a school's resources must be compared to those of other schools
in the district and in other districts.
INDICATORS AND POLICY
Indicators are powerful political tools. They can generate public support
for educationally beneficial policies, or can persuade the public to endorse
harmful policies. Indicators can neither define nor substitute for decisions
about which policies to implement. Values, as well as knowledge, influence
those decisions. Indicators should help identify areas where further examination
is called for and provide clues to promising lines of effort, but they
should be seen as just one set of inputs into a process of thoughtful school
INDICATORS AS INCENTIVES
Research consistently demonstrates that people will do more in areas
in which they are evaluated. Thus, indicators not only measure reality;
they change it. Depending on the circumstances and the stakes riding on
the outcomes, the indicators may change behaviors, and they may cease to
measure what they were intended to measure (Darling-Hammond, 1988).
Nonetheless, well-conceived indicators, appropriately used, can serve
as incentives for school improvement. If indicators encourage students
and teachers to focus on important skills and abilities, if they help identify
needs or problems so they can be addressed, then they will support responsive
schooling. Similarly, if indicators of school context help faculties and
school communities monitor the quality and the equality of opportunities
available to students, then they will support responsible decisionmaking
and more accountable education.
Archbald, A., & Newmann, F.M. (1988). Beyond standardized testing:
Assessing authentic academic achievement in the secondary school. Reston,
VA: National Association of Secondary School Principals.
Ascher, C. (1990). Can performance-based assessments improve urban schooling?
Digest Number 66. New York: ERIC Clearinghouse on Urban Education, Teachers
College, Columbia University.
Darling-Hammond, L. (1990). Teacher professionalism: Why and how? In
A. Lieberman (Ed.), Schools as collaborative cultures: Creating the future
now. Bristol, PA: Falmer Press, 25-50.
Darling-Hammond, L. (1989, Fall). Accountability for professional practice.
Teachers College Record.
Darling-Hammond, L. (1988). Assessment and incentives: The medium is
the message. The AAHE Assessment Forum. Third National Conference on Assessment
in Higher Education, Chicago, June 8-11.
Haney, W., & Madaus, G. (1986). Effects of standardized testing
and the future of the national assessment of educational progress. Working
paper for the NAEP study group. Chestnut Hill, MA: Center for the Study
of Testing, Evaluation, and Educational Policy.
Koretz, D. (1989). A framework for evaluating and validating indicators
of mathematics and science education. Santa Monica, CA: RAND Corporation.
Madaus, G.F., Kellaghan, T., Rakow, E.A., & King, D.J. (1979). The
sensitivity of measures of school effectiveness. Harvard Educational Review,
49 (2) 207-229.
Oakes, J. (1989, Summer). What educational indicators? The case for
assessing the school context. Educational Evaluation and Policy Analysis,
11 (2), 181-199.
Oakes, J. (1986, October). Educational indicators: A guide for policymakers.
New Brunswick: Rutgers University, Center for Policy Research in Education.
(ED 315 920)
Wise, A.E., & Gendler, T. (1989). Rich schools, poor schools: The
persistence of unequal education. The College Board Review, 151, 12-27.
This digest is based on a monograph. "Creating Accountability in Big
City School Systems," by Linda Darling-Hammond and Carol Ascher. It is
available for $8.00 from the ERIC Clearinghouse on Urban Education or from
the National Center for Restructuring Education, Schools, and Teaching.