ERIC Identifier: ED447725
Publication Date: 2000-12-00
Author: Gomez, Emily
Source: ERIC Clearinghouse on
Languages and Linguistics Washington DC.
Assessment Portfolios: Including English Language Learners in
Large-Scale Assessments. ERIC Digest.
The current school reform effort emphasizes the need to improve the education
of all students. Assessing the academic achievement of every student is an
essential part of this reform, but one that presents a challenge for most
schools, school districts, and states (North Central Regional Educational
Laboratory and Council of Chief State School Officers, 1996). Traditional
assessment practices in many states and school districts have tended to exclude
students who are learning English as a second language. As a consequence, many
English language learners (ELLs) are denied access to important educational
opportunities that are based on assessment results.
This digest focuses on one type of assessment system, assessment portfolios,
and discusses the advantages and challenges of using an assessment portfolio
system that includes ELLs as a district-wide assessment tool.
WHAT IS AN ASSESMENT PORTFOLIO?
An assessment portfolio is
the systematic collection of student work measured against predetermined scoring
criteria. These criteria may include scoring guides, rubrics, check lists, or
rating scales (O'Malley & Valdez Pierce, 1996). Assessment portfolios can
include performance-based assessments, such as writing samples that illustrate
different genres, solutions to math problems that show problem-solving ability,
lab reports that demonstrate an understanding of a scientific approach, or
social studies research reports that show the ability to use multiple sources.
In addition, district-wide assessment portfolios can include scores on
commercially developed, nationally norm-referenced tests, such as the Iowa Test
of Basic Skills, or results of state assessment measures, such as the Maryland
School Performance Assessment Program, as well as other information pertaining
to students' educational backgrounds.
WHAT ARE THE ADVANTAGES AND DISADVANTAGES OF ASSESSMENT
PORTFOLIO SYSTEMS FOR ENGLISH LANGUAGE LEARNERS?
Inclusion of English language
Unlike other assessment programs, assessment portfolios do not exclude
certain student populations. Many states have policies that restrict English
language learners from taking commercially developed, norm-referenced and
criterion-referenced tests for up to 3 years after their arrival in a U.S.
school system. In contrast, all students can be included in an assessment
school accountability for all students
At the state level, assessment information is often collected to ensure that
the educational system addresses the needs of all students. Inclusion of ELLs in
state and local testing programs is critical to accountability and to providing
accurate data about the achievement of these students (O'Malley &
Valdez-Pierce, 1996). By providing systems with a richer source of information
about school learning, assessment portfolios can help school systems identify
and meet the needs of diverse students, including ELLs.
shared vision of student goals and learning
By developing an assessment portfolio system that includes English language
learners, teachers, administrators, parents, and students can shape a common
vision of what students should know and be able to do as a result of their
course work. By clearly articulating expectations and the criteria upon which to
assess attainment of these expectations, school systems help create a shared
vision of the purpose of education based on the values of the community.
picture of learning
Assessment portfolios can be designed to measure virtually any observable
skill or process or content-area knowledge needed for system-wide assessment
purposes. A wide range of student products can be included in assessment
portfolios as long as predetermined scoring criteria are in place. Portfolios
are designed to be inclusive of all students and to provide an authentic
description of what students can do.
teaching and student learning
Using assessment portfolios that include English language learners not only
provides improved information about student achievement but also makes a
positive impact on teaching and student learning. According to Geoff Hewitt, a
writing assessment consultant, when teachers are trained to use and score
portfolios based on agreed-upon criteria, they tend to move toward a more
learner-centered teaching model, which encourages students to take more
responsibility for their own learning (personal communication, October 11,
1996). Through such training, teachers develop an understanding of the quality
of student work that meets specific achievement levels according to the scoring
of assessment reform
Advocates of assessment reform call for new measures that provide a better
understanding of student achievement, especially for English language learners.
By using assessment portfolios that include ELLs, school systems could reduce
the number of students excluded from system-wide assessment and possibly
increase the number of teachers participating in professional development
WHAT ARE THE CHALLENGES OF ASSESSMENT PORTFOLIOS THAT INCLUDE
ENGLISH LANGUAGE LEARNERS?
Lower comparability and reliability
Many performance-based tests, including some portfolio systems, do not easily
or meaningfully translate into a single score or set of scores. The public has
become accustomed to single scores, such as those used to describe the results
of standardized or norm-referenced tests. Single scores are comparable across
systems and from one year to the next. Because some districts report the
outcomes of performance-based tests in words rather than numbers, some
stakeholders feel the school system is less accountable for individual students.
In addition, it is difficult to implement assessment portfolios that meet the
reliability requirements many school systems want. Achieving a certain degree of
reliability among raters or test evaluators is important (Novack, Herman, &
Gearhart, 1996). Without high inter-rater agreement figures, the usefulness of
the scores as an accountability tool diminishes, because the results cannot be
used to compare scores reliably between schools or over time.
ensuring standardized testing conditions
Some states and districts are prevented by law from implementing
performance-based assessments that include portfolios. Some state legislatures
mandate the use of traditional norm-referenced tests because of the perception
that standardized tests require students to perform under similar circumstances
(Special Issues Analysis Center, 1995). When using portfolio assessments,
performance conditions may vary, and teacher bias can affect students'
performance. For example, the amount of support teachers provide to students,
the amount of time students are allowed to spend on portfolio samples, and the
extent to which student work is augmented by support from external sources have
raised questions about the validity of inferences about student competence based
on portfolio work (Gearhart, Herman, Baker, & Whittaker, 1993).
One hurdle to developing an assessment portfolio system is the ongoing
expense of the program. Designing, implementing, and scoring portfolio items is
labor intensive and therefore costly. Practitioners invest a substantial amount
of time to align the assessment tasks with the curriculum and to develop the
scoring criteria and scoring tools. Persuading state legislatures and school
boards to agree to the additional costs involved in scoring an assessment
portfolio system that includes ELLs can be difficult.
Portfolios pose a difficult scoring dilemma for three reasons. First,
developing and using scoring criteria requires labor intensive staff discussion
and training. When ELLs are included in a portfolio system, the scoring must be
designed to assess content knowledge and language proficiency. Second,
inter-rater agreement among teachers and other scorers requires intensive staff
development. Third, scoring student work using predetermined criteria is more
time consuming than scoring a single-occasion, norm-referenced test.
Fortunately, these same activities also lead to improved teaching and learning.
STEPS TO DEVELOPING AND IMPLEMENTING AN ASSESSMENT PORTFOLIO
SYSTEM THAT INCLUDES ENGLISH LANGUAGE LEARNERS:
"Decide about goals and
content." Stakeholders in the school and school district, including those
responsible for English language learners, decide what assessment information is
needed and how that information can be provided. A group of teachers agree to
lead the program.
"Design the portfolio assessment program." The lead group of teachers,
including English as a second language and bilingual education teachers,
administrators, and parents decide on the range of products to be included in
the portfolio assessment program.
"Develop scoring criteria and standards of performance." The group decides on
common goals for student learning and performance and how they will be assessed,
develops scoring rubrics and checklists, and agrees on standards of performance
to be attained. If possible, benchmarks that exemplify student work are
articulated, including benchmarks for English language learners.
"Align tasks to standards and curriculum." The group aligns the assessment
tasks to the district or state content standards and curriculum frameworks.
"Implement at pilot sites, provide staff development, and analyze results."
Decide on pilot sites and provide staff development on the implementation of
portfolios. Following the implementation at pilot sites for at least one full
school year, score the portfolios from the pilot sites. Assess the effectiveness
of the program and modify the scoring criteria based on feedback from the pilot
site educators and results of the scored portfolios. Study the effects of the
program on English language learners in particular to determine whether improved
information is available as a result of the portfolio implementation.
"Implement at all sites." Once the program has been piloted and found to be
effective, implement the portfolio program at all sites.
"Train teachers to score." Train a team of teachers to score student work
using the portfolio program's scoring criteria and benchmarks. Training should
include discussion of second language proficiency and its impact on student
achievement. Efforts are made to reach an inter-rate reliability level of .7-.8.
"Establish guidelines for administration." Stakeholders develop guidelines
for a standardized collection of student work and decide on the time, place, and
manner in which standardized prompts will be given to assess students throughout
the system. Accommodations for English language learners are delineated.
"Score the portfolios." Teachers score the portfolios based upon
predetermined criteria. This is typically done over several days in a central
location by teachers who have been trained.
"Report the results." All stakeholders receive information about the results
of the portfolio assessment in a timely fashion in ways that make the results
meaningful to everyone, including teachers, students, parents, and other
"Evaluate the program." After one year, evaluate the effectiveness of the
portfolio program and make necessary judgments.
Developing and implementing a large-scale
assessment portfolio program that includes English language learners requires
extensive planning and discussion and considerable resources. It also offers
considerable advantages. Stakeholders within the system have a common vision
about what students should learn and be able to do, how goals will be assessed,
and what criteria will be used. Improved teaching and learning are natural
outcomes of a well-designed, well-implemented assessment portfolio system.
Gearhart, M., Herman, J.L., Baker, E.L., &
Whittaker, A.K. (1993). "Whose work is it? A question for the validity of
large-scale portfolio assessment" (CSE Tech. Rep. No. 363). Los Angeles, CA:
National Center for Research on Evaluation, Standards, and Student Testing.
North Central Regional Educational Laboratory and Council of Chief State
School Officers. (1996). "The status of state student assessment programs in the
United States." Oak Brook, IL: North Central Regional Educational Laboratory.
Novack, J.R., Herman, J.L., & Gearhart, M. (1996). "Issues in portfolio
assessment: The score-ability of narrative collections" (CSE Tech. Rep. No.
410). Los Angeles, CA: National Center for Research on Evaluation, Standards,
and Student Testing.
O'Malley, J.M., & Valdez Pierce, L. (1996). "Authentic assessment for
English language learners: Practical approaches for teachers." New York:
Special Issues Analysis Center. (1995). "Inclusion of limited English
proficient students in state performance standards and assessment." Rosslyn, VA:
This Digest is based on an article published by the Northeast and Islands
Regional Educational Laboratory at Brown University in Perspectives on Policy
and Practice (March 1999): "Creating Large-Scale Assessment Portfolios That
Include English Language Learners" by Emily L. Gomez. The research for the
article was completed in 1997.