Publication Date: 1991-04-00
Author: Darling-Hammond, Linda - Ascher, Carol
Source: ERIC Clearinghouse on Urban Education New York NY.
Accountability Mechanisms in Big City School Systems. ERIC/CUE Digest No. 71.
Accountability has always been a basic concept in public education, although ideas about how to accomplish it have changed. In recent years, the urgent need to improve big city schools has been a powerful incentive to the adoption of new accountability systems. This digest explores the strengths and weaknesses of various accountability tools, the use and misuse of indicators, and ways to create genuine accountability at the school level.
TYPES OF EDUCATIONAL ACCOUNTABILITY
At least five types of accountability mechanisms may exist alongside each other (Darling-Hammond, 1989):
Political. School board members and legislators must stand for election.
Legal. Boards and legislatures enact policies, and courts can hear complaints about schools.
Bureaucratic. State and district education departments set rules and regulations to ensure that schools meet standards and follow procedures.
Professional. Teachers and other school staff must acquire specialized knowledge, pass certification exams, and uphold professional standards of practice.
Market. Parents and students may choose the programs or schools they believe are most appropriate for their needs.
Over the past few decades American schools have relied most heavily on bureaucratic mechanisms for achieving accountability. Because professional and market accountability tools are currently being proposed as strategies for school improvement, it is important to understand what each of these three offers and what its limits may be.
Bureaucratic accountability is based on the hope of finding the "one best system" under which all students will be educated. Its strength rests in its possibility to ensure equal and standardized education. Policies are made at the top and handed down to administrators who translate them into procedures for teachers to follow in educating students. This system of accountability does not hold teachers accountable for meeting the individual needs of their students; they can be held accountable only for following standard procedures.
Unlike bureaucratic accountability, whose goal is uniformity and standardization, professional accountability allows practitioners to make their own decisions about how to meet the educational needs of individual students. This model seeks to ensure that teachers will be highly knowledgeable, competent, and committed to good teaching. Consequently, a professional accountability system must pay particular attention, not only to student progress, but also to policies governing the preparation, certification, selection, and evaluation of teachers and other staff.
Magnet schools and other choice plans are based on quasi-market mechanisms. These are supposed to make schools more accountable in two ways. Because "customers" choose schools, (1) schools are expected to work harder to provide services that parents or students want; and (2) problems in undersubscribed schools are revealed, which policymakers can then address.
Market accountability poses several difficult questions: How can all citizens be guaranteed access to quality schools? How do students and their parents make school choices? What information should schools provide so that students and their parents can be informed consumers of public education? Other than changing schools, what vehicles for communication and redress can parents and students use when the schools do not meet their needs?
BALANCING DIFFERENT FORMS OF ACCOUNTABILITY
No form of accountability is sufficient by itself to ensure that all students are well served. Because each form of accountability has both strengths and weaknesses, a combination of tools is needed to make schools responsible and responsive.
In school choice plans, for example, when the most desirable schools are filled, the remaining schools, desirable or not, must serve the rest of the students. If no other public policy mechanisms are in place to support improvements in all schools, choice will not improve education across the board. A different accountability mechanism is needed to regulate and ensure equity. Similarly, if bureaucratic mechanisms are de-emphasized, greater guarantees of staff competence and commitment would need to be provided through professional accountability mechanisms.
Further, new approaches create new dilemmas that must be resolved. For example, in cases of interdistrict choice, parents lose electoral accountability at the district level when students leave the district in which they reside. Thus, mechanisms must be devised to give parents a voice in their new schools.
Similarly, school-based management proposals call for local ownership of education and make professionals at the school site primarily accountable, based on the belief that better decisions will be made by those who are closest to the situation. But in a system where significant authority devolves to the school level, how much responsibility should state governments and local districts still carry for students' treatment and achievement?
In deciding which aspects of education should be relegated to bureaucratic accountability and which should be left to professional accountability, it is useful to distinguish between equity and productivity concerns. Wise and Gendler (1989) argue that equity issues, such as the allocation of resources and guarantees of equal access, can and should be resolved by higher units of governance, accountable to a wider public, and should not be left to the decision of individual teachers or the parents of one school district.
However, since teaching, at its best, is a highly individualized process, productivity questions cannot be solved well by bureaucratic regulation. Uniform state- or district-level policy decisions about teaching methods and school processes cannot meet the needs of varying school and student circumstances. Thus, these needs are better addressed using professional accountability mechanisms. At the same time, the broader goals of the public require that states have some way of evaluating schools.
USING STATISTICAL INDICATORS
Given the complexities, it is not surprising that policymakers often
try to achieve accountability by the apparently easiest strategy: monitoring
students' test scores and sometimes linking teacher or school rewards and
sanctions to such measures. But accountability also encompasses how a school
or school system hires, evaluates, and supports its staff; how it relates
to students and parents; how it ensures that the best available knowledge
will be acquired and used; how it evaluates its own functioning; and how
it corrects its problems and provides incentives for continual improvement.
CRITERIA FOR SELECTING INDICATORS
Because statistical indicators are used to support broad inferences about schools and students, policymakers developing indicator systems should consider carefully what they want to measure, how they can best measure it, and what other information they need to interpret trends intelligently.
Oakes (1986) and Koretz (1989) maintain that indicators should offer information that is:
Problem-oriented. Indicators should be able to detect current problems or potential difficulties.
Relevant to policy. Information should be described in a way that is amenable to change by policy decisions or that shows policymakers where to target their efforts.
Reflective of educational outcomes. The data might include graduation or dropout rates; college attendance rates; voting rates; achievement test scores; writing samples; and quality of participation in science fairs, debates, or dramatic productions.
Indicative of student backgrounds. Indicators cannot be interpreted meaningfully without corollary information on which students are involved. Furthermore, if a school's student population is highly mobile, test scores intended to measure student growth at different points in time may not even be measuring the same students.
Illustrative of school context. Data on how schools are organized and what they provide are needed to offer clues about why schools achieve the outcomes that they do.
Oakes (1989) suggests that three interrelated variables are important to school achievement, and that each can be tapped with a set of indicators. The first, access to knowledge, is the extent to which schools give students opportunities to learn various domains of knowledge and skills. The second, press for achievement, is the mix of incentives--graduation requirements, program rigor, recognition of accomplishments--that schools provide students to work and achieve. Third are the professional teaching conditions--salaries, work load, time for planning--that can empower or constrain teachers as they create and implement instruction.
CRITERIA FOR INTERPRETING INDICATORS
The greatest danger of indicators is the ease with which they can give false impressions because they are misunderstood or interpreted in invalid ways. The most common misunderstandings derive from the use of average student test scores to indicate the quality of schools or school districts.
The danger is especially relevant when the stakes are high. When rewards or sanctions are automatically triggered by test scores, incentives are created not to improve teaching and learning but to push low-scoring students out of the school, or at least out of the test score count (Darling-Hammond, 1990; Haney & Madaus, 1986).
Validity. If averages hide information about individual students, interpretations drawn from average test scores will be invalid. Similarly, test items are meant to be samples of domains of learning, and the assumption that student performance on the test represents the broader curriculum goals is negated if teachers "teach to the test" (Koretz, 1989).
Reliability. To be reliable, a test score must be free of random errors of measurement. That is, if statistics on a school's curricular offerings are likely to vary, depending on who is collecting the data or how they were collected, the scores are not a reliable indicator of student progress.
Corruptibility of Indicators. Because the emphasis on basic skills test scores has prompted teaching to the test, many students spend less time on untested subjects, such as science, social studies, and the arts. The scores, therefore, no longer necessarily indicate students' general achievement. Emphasis on basic skills can also mean that the scores may no longer provide comprehensive assessments of student' ability, even in the tested subjects, because classwork narrowly oriented toward a test does not heighten students' proficiency in aspects of the subject not tested--analysis, complex problem-solving, and written or oral expression.
Indeed, since about 1970, while test scores on basic skills have improved, scores on assessments of higher-order thinking skills have declined in virtually all subject areas. Many observers, including the National Research Council and the National Councils of Teachers of English and Mathematics, argue that the overuse of multiple choice basic skills tests has actually corrupted teaching practices. Aspects of the subjects that are not tested--especially higher-order thinking and performance skills--are left untaught.
Test scores are not the only indicators that can be corrupted. Indicators of student course-taking, for example, may prove to mean something other than what policymakers intended. When pressure is great to meet new standards, schools with poor resources or shortages of qualified teachers may implement new science requirements quite differently from wealthy schools. In one school, three years of science may mean three years of general science lectures; in another school, it may mean a rigorous sequence of biology, chemistry, and physics classes replete with laboratory experiences.
MAKING FAIR COMPARISONS
Accountability systems often compare schools and school districts and increasingly connect rewards and punishments to these comparisons. Thus, it is important to set up comparisons fairly. This means that school outcomes cannot be compared without also comparing populations, resources, and educational goals. For example, schools serving low-income students often have far fewer highly experienced and qualified teachers and offer many fewer advanced courses than wealthier schools do. Comparisons of test scores that ignore these factors hold little promise of directing policymakers' attention to real problems.
One way to create fair comparisons is to develop measures of how much each individual student has learned over a period of time. Another approach, employed in California, is to compare schools with similar student populations and mobility patterns, and a wide range of other similar characteristics.
CREATING ACCOUNTABILITY AT THE SCHOOL LEVEL
An accountability system is a set of commitments, policies, and practices designed to:
increase the use of good educational practices;
reduce the use of harmful or wasteful practices; and
create internal mechanisms to identify, diagnose, and change courses of action that do not lead to learning.
Accountability is achieved only if a school's policies and practices work to provide good education and correct problems as they occur. Performance indicators and ongoing diagnostic processes are needed to evaluate whether these conditions are being met. These processes must include methods for changing school practices if they are not working well on behalf of students.
Until now, most indicator systems have been defined by states, and schools or districts have often had to shift curriculum priorities in order to succeed according to the states' systems. Often, this has meant de-emphasizing areas and modes of learning more focused on problem-solving, higher order thinking, and complex performance in favor of lower level cognitive skills.
If performance measures are to support meaningful accountability, however, they must assess and encourage valuable kinds of learning for students. For such tools to be useful at the school level, they must illuminate how individual students are progressing, and they must be accompanied by methods for figuring out how changes in practice might improve learning. At the school level, where good education is defined as meeting the needs of individual students, arguably the most important form of accountability is professional accountability, which supports responsible teacher decisionmaking. A major aspect of professional accountability is the effort to establish an inquiry ethic and a commitment to collective problem-solving. In a few schools, this kind of collective questioning and reflection is frequent. More commonly, teacher isolation, combined with centralized decisionmaking and evaluation, has removed occasions as well as incentives for this kind of activity at the school level.
Yet, if schools are to become more responsive and open to change, they must find ways to make evaluation and assessment part of their everyday activities. Teachers must have opportunities to profit from their colleagues' knowledge, experience, and perspectives on behalf of their students.
CREATING INDICES THAT ENHANCE SCHOOL ACCOUNTABILITY
Measures of student learning, as noted above, can be misleading. Aside from questions of validity and reliability, most standardized tests are not sensitive to differences in curriculum offerings and thus do not really assess the opportunities for learning that are offered in various schools (Madaus et al., 1979). To elicit a better measure of a school's effectiveness, some states have constructed curriculum-based tests that reflect more precisely the variations in school teaching.
A number of states and schools in many parts of the country--Vermont, New York, California, to name a few--are engaged in creating performance-based assessments of student learning. These approaches include essay examinations, portfolios of students' best work in various subject areas, and group projects that require analysis, investigation, experimentation, cooperation, and written, oral, or graphic presentation of findings. These assessments require students to think analytically and demonstrate their proficiency as they would in real-life situations (Archbald & Newmann, 1988). Since these are new efforts, there is, of course, insufficient experience to pass judgment on them. Based on the early experience, these approaches appear to be highly motivating for students.
To inform school decisionmaking and improve practice, measures of school performance must accompany assessments of student learning. Indicators of the school's performance--based on systematic observations by faculty, and information from surveys of students, parents, and staff--should always start with questions about students: attendance and sense of connection to school; self-esteem; and civic and social growth, as well as academic achievement. These indicators should reflect factors that structure students' experience of schooling. For example:
Who teaches what to whom?
What are the opportunities for each child to succeed and build on strengths?
How many children are able to participate in extra- or co-curricular activities?
What is the likelihood that each child will have an adult advocate at the school who knows him or her well?
How is class time used? What intellectual material do students encounter? Do these opportunities vary by classroom, group, or track?
School faculty can use the findings to make well-grounded decisions, identify areas to change, and establish benchmarks for tracking progress.
GOING WITHIN AND BEYOND THE SCHOOL AS A UNIT
Although school-based analyses can provide rich data, for some educational issues the school is either too small or too large a unit for analysis. In schools with tracks and other educational groupings, it is important to know not only whether a school offers calculus, for example, but which and how many students can take this course, and what the other math course options are. In the same vein, while a school may contain computers, their use and availability may differ depending on the track. To answer questions of equity, a school's resources must be compared to those of other schools in the district and in other districts.
INDICATORS AND POLICY
Indicators are powerful political tools. They can generate public support for educationally beneficial policies, or can persuade the public to endorse harmful policies. Indicators can neither define nor substitute for decisions about which policies to implement. Values, as well as knowledge, influence those decisions. Indicators should help identify areas where further examination is called for and provide clues to promising lines of effort, but they should be seen as just one set of inputs into a process of thoughtful school decisionmaking.
INDICATORS AS INCENTIVES
Research consistently demonstrates that people will do more in areas in which they are evaluated. Thus, indicators not only measure reality; they change it. Depending on the circumstances and the stakes riding on the outcomes, the indicators may change behaviors, and they may cease to measure what they were intended to measure (Darling-Hammond, 1988).
Nonetheless, well-conceived indicators, appropriately used, can serve as incentives for school improvement. If indicators encourage students and teachers to focus on important skills and abilities, if they help identify needs or problems so they can be addressed, then they will support responsive schooling. Similarly, if indicators of school context help faculties and school communities monitor the quality and the equality of opportunities available to students, then they will support responsible decisionmaking and more accountable education.
Archbald, A., & Newmann, F.M. (1988). Beyond standardized testing: Assessing authentic academic achievement in the secondary school. Reston, VA: National Association of Secondary School Principals.
Ascher, C. (1990). Can performance-based assessments improve urban schooling? Digest Number 66. New York: ERIC Clearinghouse on Urban Education, Teachers College, Columbia University.
Darling-Hammond, L. (1990). Teacher professionalism: Why and how? In A. Lieberman (Ed.), Schools as collaborative cultures: Creating the future now. Bristol, PA: Falmer Press, 25-50.
Darling-Hammond, L. (1989, Fall). Accountability for professional practice. Teachers College Record.
Darling-Hammond, L. (1988). Assessment and incentives: The medium is the message. The AAHE Assessment Forum. Third National Conference on Assessment in Higher Education, Chicago, June 8-11.
Haney, W., & Madaus, G. (1986). Effects of standardized testing and the future of the national assessment of educational progress. Working paper for the NAEP study group. Chestnut Hill, MA: Center for the Study of Testing, Evaluation, and Educational Policy.
Koretz, D. (1989). A framework for evaluating and validating indicators of mathematics and science education. Santa Monica, CA: RAND Corporation.
Madaus, G.F., Kellaghan, T., Rakow, E.A., & King, D.J. (1979). The sensitivity of measures of school effectiveness. Harvard Educational Review, 49 (2) 207-229.
Oakes, J. (1989, Summer). What educational indicators? The case for assessing the school context. Educational Evaluation and Policy Analysis, 11 (2), 181-199.
Oakes, J. (1986, October). Educational indicators: A guide for policymakers. New Brunswick: Rutgers University, Center for Policy Research in Education. (ED 315 920)
Wise, A.E., & Gendler, T. (1989). Rich schools, poor schools: The persistence of unequal education. The College Board Review, 151, 12-27.
This digest is based on a monograph. "Creating Accountability in Big
City School Systems," by Linda Darling-Hammond and Carol Ascher. It is
available for $8.00 from the ERIC Clearinghouse on Urban Education or from
the National Center for Restructuring Education, Schools, and Teaching.
Library Reference Search Web Directory
This site is (c) 2003-2005. All rights reserved.
Please note that this site is privately owned and is in no way related to any Federal agency or ERIC unit. Further, this site is using a privately owned and located server. This is NOT a government sponsored or government sanctioned site. ERIC is a Service Mark of the U.S. Government. This site exists to provide the text of the public domain ERIC Documents previously produced by ERIC. No new content will ever appear here that would in any way challenge the ERIC Service Mark of the U.S. Government.