ERIC Identifier: ED410229
Publication Date: 1996-09-00
Author: Roeber, Edward D.
Source: ERIC Clearinghouse on
Assessment and Evaluation Washington DC.
Guidelines for the Development and Management of Performance
Assessments. ERIC Digest.
This digest is based on time proven guidelines developed for use in training
workshops for state and local educators to outline the processes by which
performance assessments could be created, validated, and used in large-scale
assessment. The term performance assessment as used in this digest is reserved
primarily for those assessments that go beyond paper-and-pencil,
group-administered assessments. This type of assessment is an important and
unique tool available for measuring student performance at the state or local
These guidelines are provided to offer guidance to district and state policy
makers and assessment directors concerning some of the issues of managing the
development, administration, and use of performance assessments in large-scale
assessment programs. The following sections present information about how
performance assessments can be developed, administered, scored, and reported.
The paper suggests informal, less costly means to develop and use these types of
measures. However, some of the uses outlined above might require the use of
external contractors or technical advisors.
PRE-ASSESSMENT DEVELOPMENT ACTIVITIES
development can or should occur, several important planning activities must set
the stage for assessment. These steps take place at the outset to insure that
the assessment is developed in a manner that fits the content area to be
assessed and is within the resources available. These steps include the
development of the assessment framework, creation of the assessment plan,
determination of assessment resources, and production of the assessment
The framework for the assessment serves as the guide to the entire
assessment. This consists of the assessment objectives. It is from the framework
that the assessment is developed. There are many ways to determine these
objectives. The traditional manner in which to do this is to gather a group of
content area experts and classroom teachers and ask them to indicate what
students should know and be able to do by the end of certain points of
instruction. However, an alternate procedure is to allow the framework of
expectations to evolve out of research and practical experience of educators on
what outcomes students are capable of at particular times in their school
The assessment plan provides an overview and description of the types of
assessment to be developed and used, as well as the manner in which assessments
will be implemented. The plan serves the useful purpose of describing the types
of assessments that are envisioned and how assessments will be administered,
scored, and reported. The assessment plan should force the agency sponsoring the
assessment development to carefully consider the resources needed and available
in order to make decisions about the assessment at the outset. Once the
assessment plan has been written and the assessment resources have been
determined, it is possible to make any adjustments needed in the assessment
The blueprint will describe the characteristics of an adequate assessment for
each content area of the assessment framework as well as the characteristics of
an adequate assessment for each student outcome. Once completed, this should
guide the development of the assessments that are needed given the resources
available. The final step in the pre-assessment development activities is for
the assessment plan and the assessment blueprint to be approved by the
sponsoring agency and any additional advisory groups or individuals.
ASSESSMENT DEVELOPMENT STEPS
After the blueprint has been
created, it is time to formulate the assessment prompts. Throughout this process
the developer must consider several things. These include: 1) in what manner
should the assessment questions or instructions be presented to the student, 2)
what additional stimulus materials will be needed, 3) how the students will
respond and how such responses will be recorded and scored, 4) what criteria
will be used to judge student responses, 5) the number of scale points for
scoring student responses, and 6) samples of each level of response.
Once the exercises have been written, the next step is to edit them. This is
an important step in assuring that the exercise and assessment administration
processes are understandable, that the exercise warrants the special time and
attention which performance measures require, and that there is consistency
among the performance exercises. The editor also can check to ensure that the
stimulus materials fit with the exercises.
One of the more difficult aspects of performance exercises is writing the
assessment administration directions. Many performance exercises are
administered to students individually or in small groups. These exercises
require the assessment administrator to set up a standard situation to which
each student can respond, as well as, to read a standard set of directions to
each student. Developing standard assessment administration directions that are
complete and accurate is usually the result of trying the exercise out one or
more times and noting areas of students confusion, responses that students
provided which are vague or incomplete, and ways in which some or all of
students responded that were not anticipated. If a majority of student responses
indicate that the assessment instrument is ineffective, considerable
restructuring of the testing tool may be in order before any further testing of
the instrument is completed.
Once the sample student responses are gathered, someone needs to review the
responses and attempt to score them according to the criteria and preliminary
scoring guide developed by the exercise writer. After the initial work on the
scoring guide is complete, an expert panel of judges should be convened. This
panel should be asked to review each exercise, confirm the preliminary judgments
for each score scale point, and discuss those student responses that did not
appear to be scorable according the preliminary scoring guide. This panel may
note changes that needed to be made in the assessment administration process.
Following the institution of these changes, the new test should be retested and
reviewed by the panel for final approval.
PREPARATION FOR ASSESSMENT ADMINISTRATION
important set of activities is getting ready for the actual assessment. This
involves selecting the schools that will participate, preparing the schools for
participation in the assessment, and training the individuals who will gather
the data from students.
Since the assessment materials which have been developed and refined are
intended for large-scale assessment use, it is presumed that the exercises will
be given to some or all of the students at one or more grade or age level. The
original assessment plan will help to determine how and to whom the exercises
will be administered. This information will aid in the decision of which schools
are selected to participate in the standardized performance-based assessment.
Letters of notification should be sent to the school coordinator and to the
district coordinator if applicable. These letters should include such details as
when the assessment will occur, what will be involved in the assessment, what
the schools responsibilities will be, who will administer the test, and so
A major key to the success of the entire performance assessment project is
the quality of the individuals who are chosen to conduct the assessment. After
the selection process for new assessment administrators has taken place,
training should occur. When the assessment administrators are comfortable with
the administration of the tests and the manipulatives and equipment that are
used in them, the focus of the training is to shift to recording and scoring of
responses. Trainees with acceptable scoring prowess will be certified as
assessment administrators; others must be either cycled back through more
training or dismissed from the assessment project.
Once the assessment
administrators are trained, they also should contact the appropriate assessment
coordinators. The purpose of this contact is to schedule the date(s) and times
for the assessment administration to occur, to remind the school coordinator to
have a complete listing of students at the appropriate grade level(s) available
on assessment day, and specify what facilities and equipment will be needed.
After all contacts have been made, the field assessment administrators can
begin the process of assessment administration. This will start with the drawing
of the sample of students to be assessed and a list of alternates. It may be
helpful to have the school assign an aide or a student who can work with the
assessment administrator to locate the students when needed for the assessment
and bring them to the assessment administration site.
It will also be most helpful if the designated contact person and others
associated with the project but not involved directly in the assessment
administration would select some schools in which to observe one or more
students taking part in the performance assessment. In addition to observing the
assessment administration, these people can also discuss the assessment with
some of the students following the assessment. This can provide valuable insight
on why and how students responded and their motivation and interest in the
POST-ASSESSMENT ADMINISTRATION ACTIVITIES
responses are selected by an expert in the area to represent the different types
of responses that students may have given. Such expert judgments are next
confirmed by an expert panel of judges; some of the sample will appear in the
scoring guide prescored and will be used to train the scorers. The others will
be used to judge the accuracy of the scorers following the initial training.
The scoring process requires a number of things to be prearranged for the
scoring to flow smoothly. First, a determination would have to be made about
whether there will be one or two scorers for each response. Second, arrangements
will need to be made to distribute booklets and other materials to be scored,
plus rating sheets, to each of the scorers. Third, routine reliability checks
should be built into the scoring process.
The various scores need to be summarized and prepared for reporting. The
summaries will be most efficient if the individual(s) who will be doing the
reporting and those people directing the scoring discuss the needed and desired
data summarization process before the scoring is conducted. One thing to keep in
mind when reporting the data is that typical audiences are most interested in
knowing how well students performed, why students performed as they did, if the
experts were surprised in any way by the level or types of student performances,
and what the experts believe needs to be done to help students improve.
While performance assessment may be new to some
people, it is not new nor is it untried. National and statewide performance
assessments were successfully conducted in a reliable and cost-efficient manner
decades ago. As this guide has illustrated, performance assessment is feasible
and manageable. Such assessments are vitally needed in the assessment landscape
so that those interested in assessing what students are capable of doing have
access to more complete information on student performance. Although the steps
are more complex and more involved, such assessments are important in the
determination of what skills our students need to have and whether or not they
do in fact have them. Performance assessment is an important adjunct to overall
large-scale assessment strategies.
REFERENCES AND ADDITIONAL READING
Roeber, E. (1995).
Guidelines for the management of performance assessments in large-scale
assessment programs. Oak Brook, IL: North Central Regional Educational
Fairtest, b. (1993). Standardized tests and our children: A guide to testing
reform. Cambridge, MA: Author.
Neill, M., Bursh, P., Schaeffer, B., Thall, C., Yohe, M., & Zappardino,
P.(1994). Implementing Performance Assessments: A guide to classroom, school and
system reform. Cambridge,MA: Fairtest.
Rudner, L.M. & Boston, C. (1994). Performance Based Assessment. ERIC
Review, 3(1), 2-12.