ERIC Identifier: ED391989
Publication Date: 1995-00-00
Author: Kapes, Jerome T.
Source: ERIC Clearinghouse on
Counseling and Student Services Greensboro NC.
Locating and Evaluating Career Assessment Instruments. ERIC
For the purpose of both locating and evaluating career assessment
instruments, there are three primary sources. Best known among these are the
Buros Institute's publication "Tests in Print" and its comparison set of reviews
in the "Mental Measurements Yearbooks (MMY)." A second source, which first
became available in 1983-84, is "Tests and Test Critiques (TC)." Both include a
listing and brief description of most tests commercially available in English
speaking countries (i.e., "Tests in Print" and "Tests") as well as periodically
published volumes of test reviews (i.e., "MMY" & "TC")
The third source, which is published by the National Career Development
Association, is "A Counselor's Guide to Career Assessment Instruments." This
book, first published in 1982 and every six years since, contains reviews of the
most prominent career assessment instruments as well as brief descriptions of
most others commercially available. In addition, this book also includes
chapters on selecting, evaluating, using, and interpreting career relevant
There are a number of other sources that focus on specialized aspects of
career assessment. Also, certain journals publish test reviews or articles that
provide evidence of the quality of specific career assessment instruments. These
qualities typically are evaluated under the categories of norms, reliability and
validity. The American Educational Research Association (AERA), American
Psychological Association (APA) and National Council on Measurement in Education
(NCME) jointly publish "Standards for Educational and Psychological Testing,"
which provides guidance for both test publishers and users of all types of tests
(AERA, APA & NCME, 1985). In addition, the American Counseling Association
(formerly the American Association for Counseling and Development, 1989) has
produced its own guidelines, "Responsibilities of Users of Standardized Tests,"
which provides additional help for users in counseling situations.
Much information is available to help locate and evaluate career assessment
instruments, but it is the users who must employ these resources to make their
own judgments about the appropriateness of a particular instrument for a
specific situation. The purpose of this digest is to help users locate and
organize information that will improve their evaluations.
Although "Tests in Print IV" (1994)
and "Tests" (1991) are the most comprehensive listings of all tests, those
wishing to locate a career assessment instrument may find it more useful to
consult "A Counselor's Guide to Career Assessment Instruments" - 3rd Edition
(Kapes, Mastie and Whitfield, 1994). This recent edition contains reviews of 52
prominent instruments along with an Additional Instruments chapter which briefly
describes an additional 245 instruments. The entire 297 instruments are listed
alphabetically in a User's Matrix that categorizes each entry by Characteristics
(achievement, aptitude, interest, values/satisfaction/environments, career
development/maturity, personality) and Use level (elementary, junior high/middle
school, senior high, 2- or 4-year college, adult education/ training, business
and industry/employment, disabled or disadvantaged). Those interested in
locating a test for a specific purpose can use this matrix to identify
instruments that may be appropriate.
If the instruments selected in the initial search are among the 52 with a
complete review, the user can consult the reviews to further narrow the choice.
Each entry includes a section of publisher-provided information that includes
target population, statement of purpose, titles of subtests, scales and scores,
forms and levels, date of most recent edition, languages available, time, norm
groups, results reported, format, scoring, computer software, costs, comments,
and published reviews. The review section is divided into the following
headings: Description, Use in Counseling, Technical Considerations,
Computer-Based Version (if available), Overall Critique, and References. If the
instruments on which further information is needed are not among the 52
reviewed, the Additional Instruments chapter can be consulted to obtain the
publisher, date and intended population on any of the 245 additional
instruments. In addition, citations for all reviews published in the "Mental
Measurements Yearbooks," "Test Critiques," previous editions of "A Counselor's
Guide" as well as several other sources are listed. A brief description for each
test is also included.
For those instruments not reviewed in "A Counselor's Guide" (3rd edition) or
for a second opinion, the reader can consult one or more reviews cited in either
the Additional Instruments chapter or at the end of the publisher information.
If a "Mental Measurements Yearbook (MMY)" is to be consulted, it is necessary to
know both the volume (Sixth, 1965 through Eleventh, 1992) and test number. The
"MMY" entries typically contain brief publisher information along with two
independent reviews. The reviews themselves are not divided into sections and
tend to focus primarily on psychometric characteristics. Additional references
are also provided.
To access "Test Critiques (TC)," it is also necessary to know the volume
(Volume 1, 1984 through Volume 10, 1994). Each of these entries are written by a
single reviewer and are divided into five categories (Introduction, Practical
Applications/Use, Technical Aspects, Critique, and References). In addition to
"MMY" and "TC" it may also be useful to consult the first (1982) and second
(1988) editions of "A Counselor's Guide" or any of the other sources listed in
the chapter on Sources of Information about Testing and Career Assessment in the
third edition, which is an annotated bibliography of sources.
Many sources exist that could aid a
user to evaluate the potential usefulness of a career assessment instrument. The
previously mentioned AERA, APA and NCME (1985) document provides guidance for
both test publishers and users in the form of essential, conditional and
secondary standards. The standards, for example, call for a technical manual to
be made available by the publisher so that any user can obtain information about
the norms, reliability and validity of the instrument as well as other relevant
topics. It should be pointed out that, although the publisher typically provides
evidence of this type from studies conducted with subjects for whom the
instrument is intended, it may be necessary for the user to obtain data from
other sources that better reflect the use intended for a particular application.
This can be done from studies published in the literature (e.g., in the "Measurement and Evaluation in Counseling and Development" journal or the
"Career Development Quarterly") or from studies conducted on data collected by
In addition to the technical qualities of norms, reliability and validity,
there are many other qualities of a career assessment instrument that need to be
evaluated before selection for a particular use. In his chapter in "A
Counselor's Guide" on "Selecting a Career Assessment Instrument," Mehrens covers
many of these, including types of scores and interpretation materials,
appropriateness for various groups and a host of practical issues (e.g.,
qualifications of users, time, costs, and publisher support). In addition, he
provides a "Test Evaluation Outline" that is included here to assist the user to
systematically identify and collect all information necessary to conduct an
adequate evaluation (Mehrens, 1994, p. 29):
1. State your purpose for testing
2. Describe the group that will be tested (for example, age or grade)
3. Name of test
6. Copyright date(s)
7. Purpose and recommended use as stated in the manual
8. Grade/age levels for which the instrument was constructed
9. Forms: Are equivalent forms available? What evidence is presented on the
equivalence of the forms?
Format: comment on legibility, attractiveness, and convenience
Content of test and types of items used
Administration and timing requirements
Scoring processes available (e.g., machine scoring)
Types of derived scores available
Types and quality of norms
Adequacy of reliability evidence presented in the manual
General quality of administrative, interpretative and technical manuals
Comments about the instrument by outside reviewers
There are many sources to aide in locating and
evaluating career assessment instruments. The primary sources are "A Counselor's
Guide to Career Assessment Instruments," the "Mental Measurement Yearbooks," and
"Test Critiques." The AERA, APA and NCME Standards provide guidance to
publishers and users on the qualities of norms, reliability, and validity as
well as many other considerations that affect test use. However, the bottom line
is that the user is responsible to make the final judgment about the
appropriateness of a particular instrument for a specific use.
American Association for Counseling and
Development (1989). Responsibilities of users of standardized tests (AACD/AMECD
Policy Statement). Alexandria, VA: Author.
American Educational Research Association, American Psychological
Association, National Council on Measurement in Education. (1985). Standards for
educational and psychological testing. Washington, D.C.: American Psychological
Kapes, J.T., Mastie, M.M., & Whitfield, E.A. (Eds.). (1994). A
counselor's guide to career assessment instruments (3rd edition). Alexandria,
VA: National Career Development Association.
Keyser, J.D., & Sweetland, R.C. (1984-94). Test critiques (Vols. 1-10).
Austin, TX: PRO-ED. Kramer, J.J., & Conoley, J.C. (Eds.). (1992). Eleventh
mental measurements yearbook. Lincoln, NE: Buros Institute of Mental
Mehrens, W.A. (1994). Selecting a career assessment instrument. In J.T.
Kapes, M.M. Mastie, & E.A. Whitfield (Eds.), A counselor's guide to career
assessment instruments (3rd edition). Alexandria, VA: National Career
Murphy, L.L., Conoley, J.C., Impara, J.C., & Mitchell, J.V. (Eds.).
(1994). Tests in print IV. Lincoln, NE: Buros Institute of Mental Measurement.
Sweetland, R.C., & Keyser, D.J. (Eds.). (1991). Tests: A comprehensive
reference for assessments in psychology, education, and business (3rd edition).
Austin, TX: PRO-ED.