ERIC Identifier: ED414518
Publication Date: 1995-00-00
Author: Hutchinson, Nancy L.
Source: ERIC Clearinghouse on Counseling and Student Services Greensboro NC., Canadian Guidance and Counselling Foundation Ottawa (Ontario).
Performance Assessments of Career Development: ERIC Digest.
The emergence of career development programs in Canadian secondary schools enhances self-awareness and career-awareness, while and preparing youth for employment (Conger, 1992). Counselors and teachers who conduct these group interventions often use performance assessments to demonstrate student learning and to guide program improvement. Performance assessments are evaluations of "authentic" student work that reflect central processes and performances within a discipline (Wiggins, 1989). Performance assessments are worthwhile educational experiences which are based in meaningful tasks--tasks that are complex and challenging, consistent with goals for learning, closely related to real-world skills, and allow students to use processes relevant to genuine performance. They may require flexible time frames, open-ended formats, and collaboration with peers.
Norm-referenced appraisals of career development have come under increasing criticism (Healey, 1990). Performance assessments are often referred to as alternate assessment approaches. In performance assessments, the emphasis is on what students can do as well as what students know. Performance assessments are well matched to the way classroom assessments frequently inform students, counselors, teachers, and parents what students have learned, and make day-to-day instructional decisions about the program (Stiggins, 1993).
There are many ways in which performance assessments can be implemented in career development programs:
Student performances can be judged against a set of performance criteria (e.g., completion of an employment application form at levels set by competent adults).
Student performances can be evaluated by individuals other than the students' teachers (e.g., employment interviews conducted by employers from a chosen field).
Portfolios of student work can be used to assess depth and breadth of understanding (e.g., student solutions to problem scenarios that arise on the job).
A number of published career development programs, like Pathways, have implemented performance assessments (Hutchinson & Freeman, 1994). Pathways consists of five instructional modules:
1. Awareness of self and careers.
2. Employment writing.
3. Interview skills.
4. Problem solving on the job.
5. Anger management.
Students work in pairs and small groups. They "think aloud," provide explanations for their choices, and role-play authentic tasks. For example, students learn about employers' uses and expectations of application forms by role-playing as employers who judge completed applications and who make hiring decisions.
Each module in Pathways contains performance assessments. The module on career and self-awareness includes both an assessment measure that provides descriptions of individuals seeking employment and "want ads" containing descriptions of jobs. Students judge whether or not applicants are well matched, on specified criteria, to the advertised positions, and whether they, themselves, are well matched to the positions. The module on employment writing contains an application form as an assessment measure, while in the interview module, one assessment is a simulated employment interview. The module on solving problems on the job and the module on anger management contain assessment measures consisting of realistic scenarios for which students generate a number of solutions and then evaluate those solutions. Scoring criteria are provided for each performance assessment, so that a reliable measure of the students' performance can be obtained. Additionally, in the learning activities within the modules, students generate a portfolio of performances including self-awareness activities, problem-solving scenarios, video-taped role playing, resumes, and other simulated tasks.
CRITICISMS OF PERFORMANCE ASSESSMENTS
"Such simulated tasks are authentic in that they replicate the challenges and standards of real-world performances and are representative of the ways in which knowledge and skills are used in real world contexts, even though they do not simulate all of the complexity of real-world functioning" (Messick, 1994, p.17). Messick has criticized performance assessments for their task-specificity, arguing that educators are less concerned with the particular performance, than in the knowledge and skill that enable a whole range of performances. There are questions to be answered about the generalizability of performance assessments across students and sites, as well as tasks. Messick cautions that performance measures must also be sensitive enough to detect relevant differences between performances and changes in performances.
Proponents of performance assessments admit that they emphasize validity over reliability. As Wiggins asserts, "We must first decide what are the actual performances that we want students to be good at. We must design those performances first and worry about a fair and thorough method of grading them later" (1989, p.705). Data collected in a two-year cohort study (Hutchinson, Freeman, & Fisher, 1993) demonstrated that the performance measures developed for Pathways were sensitive enough to show student changes over time and to distinguish between students who had received intensive instruction and those who had not. There were also indications of adequate reliability in that students maintained their performance improvements five months after intervention in four of the five areas of instruction. Two sets of measures were used: one immediately following instruction of each module, and one for cohort comparisons. Similar scores on the two sets provide preliminary evidence of generalization. The criticisms of performance assessment regarding representativeness, reliability, and sensitivity represent empirical questions that can, and already are, being addressed with data that suggest the criticism may not be solid.
IMPLEMENTATION OF PERFORMANCE ASSESSMENTS
Widespread implementation of performance assessments in career development programs requires that counselors undertake a number of challenging endeavors. At both the national and the local levels they must work toward the following goals:
Articulating the desired outcomes of career development-defining the domain of content, and identifying complex "authentic" performances and processes (Wiggins, 1989).
Stating the specific requirements of performances including the knowledge, skills, and processes that must be exemplified in a performance or collection of performances (Stiggins, 1988).
Developing tests of performances so central to learning that the test is valued and used to further--as well as to demonstrate--learning (Taylor, 1994).
Educating counselors, teachers, and students so they can work together and use performance assessments to assess accurately their own work on a regular basis (Wiggins, 1993).
SUMMARY AND CONCLUSIONS
As career development becomes a more integral part of the secondary school program, the pressure increases to use performance assessments to demonstrate the both effectiveness of programs and the soundness of instructional decisions. With differentiated portraits of student performance within career development, it is possible to envision student assessment as informing, rather than merely measuring, the career development process.
Conger, D. S. (1992). Where to career development and counseling? Career Development Quarterly, 40, 376-377.
Healey, C. C. (1990). Reforming career appraisals to meet the needs of clients in the 1990s. The Counseling Psychologist, 18, 214-226.
Hutchinson, N. L., & Freeman, J. G. (1994). Pathways (5 volumes). Toronto, ON: Nelson Canada.
Hutchinson, N. L., Freeman, J. G., & Fisher, C. (1993). A two-year cohort study: Career development for youth with learning disabilities. Paper presented at the annual meeting of the American Educational Research Association, Atlanta, GA.
Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13-23.
Stiggins, R. J. (1988). The design and development of performance assessments. Educational Measurement: Issues and Practice, 6(3), 33-42.
Stiggins, R. J. (1993). Two disciplines of educational assessment. Measurement and Evaluation in Counseling and Development, 26, 93-104.
Taylor, C. (1994). Assessment for measurement or standards. American Educational Research Journal, 31, 231-262.
Wiggins, G. (1989). Teaching to the (authentic) test. Educational Leadership, April, 41-47.
Library Reference Search
Please note that this site is privately owned and is in no way related to any Federal agency or ERIC unit. Further, this site is using a privately owned and located server. This is NOT a government sponsored or government sanctioned site. ERIC is a Service Mark of the U.S. Government. This site exists to provide the text of the public domain ERIC Documents previously produced by ERIC. No new content will ever appear here that would in any way challenge the ERIC Service Mark of the U.S. Government.