ERIC Identifier: ED328603
Publication Date: 1990-12-00
Author: Mullis, Ina V. S.
Source: ERIC Clearinghouse on Tests Measurement and Evaluation
Washington DC., American Institutes for Research Washington DC.
The National Assessment of Educational Progress (NAEP).
Since 1969, the National Assessment of Educational Progress (NAEP) has
been assessing what public and private school students know and can do
in a variety of curriculum areas, including mathematics, reading, science,
writing, U.S. history, and geography. In 1988, Congress added a new dimension
to NAEP by authorizing, on a trial basis, voluntary participation in state-level
assessments in 1990 and 1992. With the President's Summit on Education,
the resultant education goals, and the addition of the state assessment
program, NAEP is playing an increasingly visible role in measuring student
This digest describes how NAEP is organized and what is included in
a typical NAEP assessment. It also looks at how NAEP selects students for
its assessments and how the results from an assessment are used.
HOW IS NAEP ORGANIZED AND MANAGED?
NAEP is a congressionally-mandated project of the National Center for
Education Statistics (NCES), U.S. Department of Education. NCES carries
out the NAEP project through competitive awards to qualified organizations.
NCES awarded the operational contract for conducting the 1990 and 1992
assessments to Educational Testing Service (ETS)--who is responsible for
printing, open-ended scoring, and scanning--and its subcontractor, Westat,
Inc.--who is responsible for data collection. In addition to coordinating
operational activities, ETS develops the assessment instruments, analyzes
the assessment results, and works with NCES staff to prepare the reports
on student achievement.
The National Assessment Governing Board (NAGB) formulates policy guidelines
for NAEP. NAGB's composition is specified by law, and its 24 members include
teachers, curriculum specialists, state legislators, governors, measurement
experts, chief state school officers, state and local school board members,
school superintendents, principals, and representatives from business and
the general public.
NAGB selects the subject areas to be assessed, in addition to those
specified by Congress; develops assessment objectives and specifications;
ensures that all NAEP items are free from racial, gender, or cultural bias;
and identifies appropriate achievement goals for students. NAGB also awards
contracts for technical advice. For example, NAGB awarded a contract to
the Council of Chief State School Officers (CCSSO) to manage the consensus
process to develop NAEP's 1992 reading objectives and item specifications.
WHAT IS COVERED IN A TYPICAL NAEP ASSESSMENT?
The NAEP objectives underlying each assessment typically take the form
of frameworks or matrices delineating the important content and process
areas to be assessed. For example:
o The mathematics framework is a five by three matrix specifying five
content areas--Numbers and Operations; Measurement; Geometry; Data Analysis,
Statistics, and Probability; and Algebra and Functions--and three process
or ability areas--conceptual understanding, procedural knowledge,
and problem solving.
o The reading framework includes reading for three primary purposes--for
literary experience, for information, and for performing a task. The process
dimension includes fluency, constructing meaning (forming an initial understanding
of the text and developing an interpretation of it), and elaborating and
responding critically (reflecting on and responding to the text as well
as demonstrating a critical stance).
As part of the legislatively-mandated consensus process used to regularly
update NAEP objectives and item specifications, the contractors hired by
NAEP indicate the percentages of assessment items that should be devoted
to measuring various aspects of the frameworks. For more recent assessments,
recommendations have emphasized measuring higher order skills and understandings.
The assessment instruments, which are generally administered in group
settings, include a variety of multiple-choice and open-ended items. Here
is a sample of the tasks the students perform:
o During the mathematics assessment, students are allowed to use calculators
and protractors or rulers, and the instrument asks for open-ended responses
to complex problems.
o The reading assessment will break new ground by presenting longer,
naturally occurring passages and increasing the proportion of open-ended
questions to fill nearly half of the assessment. Using the Integrated Reading
Performance Record, NAEP will also assess fluency in oral reading and will
conduct a portfolio study based on interviews of individual fourth-grade
o Based entirely on student writing samples, the writing assessment
includes a variety of prompts addressing different purposes for writing.
Students' responses are evaluated for task accomplishment, overall fluency,
and mechanical correctness.
o The science assessment includes a variety of open-ended questions,
some of which ask students to describe their conceptions of scientific
inquiry and to draw conclusions about scientific phenomena and events.
HOW ARE STUDENTS SELECTED FOR PARTICIPATION IN NAEP?
In 1990, approximately 87,000 students participated in the national
assessment and another 100,000 participated in the state assessments of
eighth grade mathematics. Considering the planned expansion of the state
assessment program, NAEP anticipates that in 1992 the assessments will
involve approximately 419,000 students in 12,000 schools.
The details of the sampling procedures for the national and state assessments
differ in the following respects:
o For the national assessment, NAEP uses a four-stage sampling design:
(1) primary sampling units are identified; (2) schools are enumerated within
the primary sampling units and randomly selected; (3) students are randomly
selected from those schools; (4) those students are assigned to assessments
in different subject areas.
o For the state assessments, the schools in each state are enumerated,
stratified, and randomly selected; then, students are listed and randomly
selected, and assigned to assessment sessions.
HOW DOES NAEP REDUCE THE BURDEN ON PARTICIPATING SCHOOLS AND STUDENTS?
All NAEP data are collected by trained administrators. For the national
assessments, Westat, Inc. trains its own field staff to collect the data,
thus reducing the burden on participating schools. However, according to
the NAEP legislation, each participating state must collect data for the
trial state assessments. Westat achieves uniformity of procedures across
states through training and quality-control monitoring.
NAEP uses matrix sampling to reduce the burden for participating students.
In matrix sampling, the total pool of assessment questions is divided,
and portions are given to different but equivalent samples of students.
Thus, not all students are asked to answer all questions. This system provides
broad coverage of each curriculum area being assessed, while each student
invests only about an hour in the assessment.
HOW ARE NAEP RESULTS ANALYZED AND REPORTED?
NAEP results are known as The Nation's Report Card. NAEP also publishes
results in a series of widely-disseminated reports that summarize achievement
across items and describe relationships between achievement and a variety
of background characteristics. In addition, NAEP provides information about
the percentage of students who give acceptable responses to each item.
NAEP objectives, reports, technical documentation, and complete publications
lists are available from Educational Testing Service, P.O. Box 6710, Princeton,
Johnson, Eugene G. and Rebecca Zwick, The NAEP 1988 Technical Report.
Princeton, NJ: Educational Testing Service, National Assessment of Educational
Mullis, Ina V.S., The NAEP Guide: A Description of the Content and Methods
of the 1990 and 1992 Assessments. Princeton, NJ: Educational Testing Service,
National Assessment of Educational Progress, 1990.