ERIC Identifier: ED391983
Publication Date: 1995-00-00
Author: Sampson, James P., Jr.
Source: ERIC Clearinghouse
on Counseling and Student Services Greensboro NC.
Computer-Assisted Testing in Counseling and Therapy. ERIC
Computer-assisted testing (CAT) in counseling and therapy is becoming
increasingly common due to dramatic improvements in cost-effectiveness and
increased counselor familiarity with computer applications. The assumption
underlying the use of CAT is that the effectiveness of counseling is improved by
allocating repetitive computational and instructional tasks to the computer,
thus allowing counselors to more fully focus on interpersonal tasks, such as
helping clients understand the role of testing in counseling and helping clients
integrate the self-knowledge obtained in testing into a concrete plan for
behavior change. The potential benefits of CAT, however, need to be evaluated
against the potential limitations of this technology.
COMPUTER-ASSISTED TESTING OPTIONS
The following options
exist for using computer-assisted testing in counseling and therapy:
Test administration via: a) keyboard input by the client from items presented on
the computer display, with alternative input options available for physically
disabled clients; or b) client completion of a specially prepared test answer
sheet that is then optically scanned for computer input; or c) client completion
of a traditional test answer sheet with keyboard input by a clerical staff
Test scoring via the computer (local or remote).
Test score profile generation via the computer.
Narrative interpretive report generation via the computer with reports available
for both the client and practitioner if appropriate (the narrative report may
also include the test profile).
Videodisc-based generalized test interpretation provided to the client
immediately following test administration (Sampson, 1990a, p452-453).
POTENTIAL BENEFITS OF COMPUTER-ASSISTED
Computer-assisted testing can enhance test administration, scoring,
interpretation, and integration. Test administration and scoring may be enhanced
due to the standardization inherent in computer functioning. Each test taker
receives an identical presentation of test items and response sets (with the
exception of adaptive testing where each test taker receives a unique minimum
selection of items necessary to achieve a valid result). Greater standardization
of item presentation eliminates errors caused when a test taker gets out of sync
between the answer sheet and a printed test item (Byers, 1981). The availability
of adaptive devices allows persons with a disability to complete tests with
minimal staff assistance (Sampson, 1990b). Test results can be more valid since
staff members have less of an opportunity to influence client responses. Test
scoring is enhanced due to reduced computational errors.
Test interpretation may be enhanced by providing the counselor with an
expanded and consistent knowledge base to assist in the interpretation of test
data. Computer-based test interpretation (CBTI) is typically based on research
data and clinical experience. Roid and Gorsuch (1984) described four approaches
to CBTI: 1) descriptive interpretations; 2) clinician-modeled interpretations
(renowned clinician type); 3) clinician-modeled interpretations (statistical
model type); and 4) clinical actuarial interpretations. Counselors can use CBTI
to support or challenge their judgments about the nature of client problems and
potentially effective intervention strategies.
Test integration may be enhanced by including computer-assisted instruction
as part of CAT. Clients can be better prepared to use their test results by
being more aware of basic concepts and the general nature of their scores.
Relieved of presenting repetitive test interpretation information, counselors
have more time to explore clients' perceptions of their test data and the
implications of the test data for behavior change. The computer can be used to
deliver both text-based and interactive video-based instruction (Sampson,
POTENTIAL LIMITATIONS OF COMPUTER-ASSISTED
Computer-assisted testing can limit, as well as enhance, test
administration and interpretation. Although paper-and-pencil and computer
administration of tests often produce equivalent results, variations in results
have sometimes been found to exist. French (1986) recommended that the
equivalency of results from different types of administration modes needs to be
established for each instrument. Establishing equivalency will reduce the
likelihood that computer administration is influencing the nature of test
results. Scoring errors represent another potential limitation for
computer-assisted test administration. Most (1987) noted that, "The computer
itself does not contribute error, but the complex nature of computer programming
and the difficulty involved in reading computer programs or code makes it easy
to make program errors which are difficult to find" (p. 377).
Concerns have been raised about the validity of computer-based test
interpretation. Eyde and Kowal (1987) found differences in CBTI reports
generated from a single set of scores from one instrument. Differences also were
noted in their study between the CBTI reports and the judgments of a clinician.
Eyde and Kowal (1987) stated, "Buyers should be aware of the limitations of
computer products and remind themselves that computer output is only as good as
the data behind the decision rules used to produce the interpretation" (p. 407).
Ethical concerns also exist about counselor misuse of CBTI. Unqualified
counselors may be more likely to use CBTI reports to compensate for a lack of
training and experience. By using CBTI to replace rather than supplement
counselor judgment, counselors become more dependent on the potentially dubious
validity of some CBTI software and are less likely to integrate data from valid
CBTI reports effectively with other sources of client data due to their lack of
Counselors should become familiar with
existing CAT applications (see Krug, 1993; Walz, Bleuer, & Maze, 1989) and
the various professional standards that relate to CAT. Counselors then should
carefully select and effectively implement valid software that is subsequently
evaluated in terms of service delivery impact.
The use of CAT can either enhance or limit the
effectiveness of testing in counseling and therapy. Having an open mind about
the potential of this technology and a willingness to change needs to be matched
with good critical thinking skills and a healthy skepticism for any innovation
promising substantial benefits from minimal efforts. It is the responsibility of
counselors to guide the design and use of this technology.
Byers, A. P. (1981). Psychological evaluation by
means of an on-line computer. Behavior Research Methods & Instrumentation,
Eyde, L. D., & Kowal, D. M. (1987). Computerised test interpretation
services: Ethical and professional concerns regarding U.S. producers and users.
Applied Psychology: An International Review, 36(3/4), 401-417.
French, C. F. (1986). Microcomputers and psychometric assessment. British
Journal of Guidance and Counselling, 14, 33-45.
Krug, S. E. (1993). PSYCHWARE SOURCEBOOK: A resource guide to computer based
products for assessment in psychology, education, and business (4th Ed.).
Champaign, IL: Metritech.
Most, R. (1987). Levels of error in computerized psychological inventories.
Applied Psychology: An International Review, 36(3/4), 375-383.
Roid, G. H., & Gorsuch, R. L. (1984). Development and clinical use of
test-interpretive programs on microcomputers. In M. D. Schwartz (Ed.), Using
computers in clinical practice: Psychotherapy and mental health applications
(pp. 141-149). New York: Haworth Press.
Sampson, J. P., Jr. (1990a). Computer applications and issues in using tests
in counseling. In C. E. Watkins, Jr. & V. L. Campbell (Eds.), Testing in
counseling practice (pp. 451-474). Hillsdale, NJ: Lawrence Erlbaum Associates,
Sampson, J. P., Jr. (1990b). Computer-assisted testing and the goals of
counseling psychology. The Counseling Psychologist, 18, 227-239.
Walz, G. R., Bleuer, J. C., & Maze, M. (Eds). (1989). Counseling software
guide: A resource for the guidance and human development professions.
Alexandria, VA: American Association for Counseling and Development.