ERIC Identifier: ED338701
Publication Date: 1991-07-00
Author: Shavelson, Richard J. - And Others
Clearinghouse on Tests Measurement and Evaluation Washington DC.
What Are Educational Indicators and Indicator Systems? ERIC/TM
Educational indicator systems serve similar purposes to indicator systems
that are used to monitor the economy, the criminal justice system, or other
social systems. Statistical indicators are used to monitor complex conditions
that we would probably judge imprecisely or miss altogether in day-to-day
observations. Governments recognize the value of statistics that provide current
information, analyze trends, and forecast impending changes. Consequently, it is
not surprising that policymakers and researchers are seeking better statistical
indicators of education.
The overriding purpose of indicators is to characterize the nature of a
system through its components--how they are related and how they change over
time. This information can then be used to judge progress toward some goal or
standard, against some past benchmark, or by comparison with data from some
other institution or country.
WHAT IS AN INDICATOR?
The term indicator was defined above
as a statistic. After reviewing the literature on social indicators, Jaeger
(1978) concluded that indicators are "anything but clear and consistent. Review
of a dozen definitions has produced much that is contradictory and little that
is concise and illuminating" (p. 285). He recommended that: all variables that
(1) represent the aggregate status or change in status of any group of persons,
objects, institutions, or elements under study, and that (2) are essential to a
report of status or change of status of the entities under study or to an
understanding of the condition of the entities under study, should be termed
indicators. "I would not require that reports of status or change in status be
in quantitative form, for narrative is often a better aid to comprehension and
understanding of phenomena than is a numeric report" (pp. 285-287).
Jaeger's recommendation to leave the definition of an indicator open and to
determine the status of potential indicators on pragmatic rather than strict
definitional grounds is a wise one.
An education system can be conceived as having underlying properties that are
not directly or perfectly measurable. For example, we can talk about the quality
of the teaching force but also recognize that there is no direct way to measure
it. At best, several statistics can be combined into an indicator that gets at
our notion of teacher quality. An indicator of teacher quality might be some
aggregate of years of academic training in the discipline taught; possession (or
lack of) a credential in the subject matter taught; measured subject-matter
knowledge; measured pedagogical knowledge; measured ability to translate
subject-matter knowledge into a form that communicates to students of a given
age, background, and prior knowledge; and so on.
Education indicators are statistics that reflect important aspects of the
education system, but not all statistics about education are indicators.
Statistics qualify as indicators only if they serve as yardsticks. That is, they
must tell a great deal about the entire system by reporting the condition of a
few particularly significant features of it. For example, the number of students
enrolled in schools is an important fact, but it does little to tell us how well
the education system is functioning. However, data on the proportion of
secondary students who have successfully completed advanced study in mathematics
can provide considerable insight into the health of the system, and can be
appropriately considered an indicator.
We propose the following working definition as a heuristic guide: An
indicator is an individual or composite statistic that relates to a basic
construct in education and is useful in a policy context.
WHAT IS AN INDICATOR SYSTEM?
Another central concept in the
discussion of indicators is that of the indicator system. Whether indicators are
single or composite statistics, a single indicator can rarely provide useful
information about such a complex phenomenon as schooling. Indicator systems are
usually designed to generate more and more accurate information about
conditions. However, an indicator system is more than just a collection of
indicator statistics. Ideally, a system of indicators measures distinct
components of the system and also provides information about how the individual
components work together to produce the overall effect. In other words, the
whole of the information provided by a system of indicators is greater than the
sum of its parts.
National indicators should be conceived of as something more comprehensive
than a time series of educational outcomes (e.g., achievement, participation).
Simply monitoring outcomes does not provide explanations for observed trends.
For example, trends might be explained by demographic changes, by educational
improvements, or by some combination of these. Moreover, education policy
indirectly influences outcomes by actions such as increasing standards for
teacher certification or for high school graduation. The direct effects of these
policies will be reflected in changes in teachers' qualifications (e.g., an
increase in teachers with bachelors' degrees in the disciplines they teach, not
degrees in education), in better matches between teachers' subject-matter and
pedagogical training and their teaching assignments, and in the number of
academic courses students take in high school.
National indicators must represent, at least roughly, the important
components of an educational system. In addition to monitoring outcomes,
indicators should reflect the characteristics of students and communities served
by schools, the financial and human resources (especially teachers) available to
the schools, and other educational inputs. Moreover, they should reflect the
adequacy of the curriculum and instruction received by students, the nature of
the school as an organization in pursuit of educational excellence and equity,
and other educational processes. Finally, indicators must be related to one
another so that their relationships, and changes in these relationships, can be
ascertained to suggest possible explanations for observed changes in outcomes.
WHAT ARE REASONABLE EXPECTATIONS FOR AN INDICATOR SYSTEM?
good education indicator system is expected to provide accurate and precise
information to illuminate the condition of education and contribute to its
improvement. The information generated will be neither possible to grasp through
casual observation nor generally available from other efforts to collect,
report, and analyze data about schooling. Indicators are thus expected to assist
policymakers as they formulate schooling goals and translate those goals into
Whenever social indicators have been heralded as a stimulus for reform, their
promise has quickly given way to realism. Promises of policy applications have
been overly optimistic. Indicator systems were, for example, unable to provide
detailed and accurate enough information for evaluating government programs.
These events led to more realistic assessments of what indicators can and cannot
What Indicators Cannot Do.
The literature on social indicators appears to agree on what indicators
Set goals and priorities. The public establishes educational goals and
priorities through its elected representatives. The information generated by an
indicator system can inform those objectives, but it is just one factor among
many in shaping decisions about policy preferences and priorities.
Evaluate programs. Social indicators cannot substitute for well-designed,
in-depth evaluations of social programs. Indicators do not provide the level of
rigor or detail necessary.
Develop a balance sheet. Social indicators lack the common referent available
to economic indicators. Evoking an economic analogy and proposing a parallel
development for social indicators is misleading because education cannot put
each of its constructs on a common dollar metric as can be done, say, for Gross
National Product (GNP). As Rivlin (1973, p. 419) pointed out, "No amount of
disaggregation of inputs...will provide a basis for answering the
how-are-we-doing question in the education sector. As long as cost is used as a
proxy for value there is no way to compare inputs with outputs or to see whether
a given amount of education is being produced with fewer resources." Rivlin also
noted that because students help produce education, it is difficult to
disentangle the quality of the output from the quality of student input.
What Indicators Can Do.
The expectations for social indicators are now quite modest: to describe and
state problems more clearly, to signal new problems more quickly, to obtain
clues about promising educational programs, and the like. The following
statements illustrate the realistic tone currently taken by the social indicator
We will be able to describe the state of the society and its dynamics and
thus improve immensely our ability to state problems in a productive fashion,
obtain clues as to promising lines of endeavor, and ask good questions.
The fruit of these [social indicator] efforts will be more directly a
contribution to policy-makers' cognition than to their decisions. Decisions
emerge from a mosaic of inputs, including valuational and political, as well as
technical components. (Sheldon and Parke, 1975, p 698)
de Neufville, J.I. (1978-79) Validating policy
indicators. Policy Sciences, 10, 171-188.
Jaeger, R. (1978) About educational indicators. In L.S. Shulman (Ed.) Review
of Research in Education, 6, 276-315.
Rivlin, A.M. (1973) Measuring performance in education. In M.Moss (Ed),
Studies in income and wealth. New York: Columbia University Press, 411-437.
Shavelson, R., L.M. McDonnell, & J. Oakes (Eds, 1989) Indicators for
Monitoring Mathematics and Science Education: A Sourcebook. Santa Monica: RAND
Corporation. This Digest was adapted from material appearing in this book.
Sheldon, E.B. & Parke, R. (1975) Social indicators. Science, 188,