ERIC Identifier: ED391983
Publication Date: 1995-00-00
Author: Sampson, James P., Jr.
Source: ERIC Clearinghouse on Counseling and Student Services Greensboro NC.
Computer-Assisted Testing in Counseling and Therapy. ERIC Digest.
Computer-assisted testing (CAT) in counseling and therapy is becoming increasingly common due to dramatic improvements in cost-effectiveness and increased counselor familiarity with computer applications. The assumption underlying the use of CAT is that the effectiveness of counseling is improved by allocating repetitive computational and instructional tasks to the computer, thus allowing counselors to more fully focus on interpersonal tasks, such as helping clients understand the role of testing in counseling and helping clients integrate the self-knowledge obtained in testing into a concrete plan for behavior change. The potential benefits of CAT, however, need to be evaluated against the potential limitations of this technology.
COMPUTER-ASSISTED TESTING OPTIONS
The following options exist for using computer-assisted testing in counseling and therapy:
1. Test administration via: a) keyboard input by the client from items presented on the computer display, with alternative input options available for physically disabled clients; or b) client completion of a specially prepared test answer sheet that is then optically scanned for computer input; or c) client completion of a traditional test answer sheet with keyboard input by a clerical staff person.
2. Test scoring via the computer (local or remote).
3. Test score profile generation via the computer.
4. Narrative interpretive report generation via the computer with reports available for both the client and practitioner if appropriate (the narrative report may also include the test profile).
5. Videodisc-based generalized test interpretation provided to the client immediately following test administration (Sampson, 1990a, p452-453).
POTENTIAL BENEFITS OF COMPUTER-ASSISTED TESTING
Computer-assisted testing can enhance test administration, scoring, interpretation, and integration. Test administration and scoring may be enhanced due to the standardization inherent in computer functioning. Each test taker receives an identical presentation of test items and response sets (with the exception of adaptive testing where each test taker receives a unique minimum selection of items necessary to achieve a valid result). Greater standardization of item presentation eliminates errors caused when a test taker gets out of sync between the answer sheet and a printed test item (Byers, 1981). The availability of adaptive devices allows persons with a disability to complete tests with minimal staff assistance (Sampson, 1990b). Test results can be more valid since staff members have less of an opportunity to influence client responses. Test scoring is enhanced due to reduced computational errors.
Test interpretation may be enhanced by providing the counselor with an expanded and consistent knowledge base to assist in the interpretation of test data. Computer-based test interpretation (CBTI) is typically based on research data and clinical experience. Roid and Gorsuch (1984) described four approaches to CBTI: 1) descriptive interpretations; 2) clinician-modeled interpretations (renowned clinician type); 3) clinician-modeled interpretations (statistical model type); and 4) clinical actuarial interpretations. Counselors can use CBTI to support or challenge their judgments about the nature of client problems and potentially effective intervention strategies.
Test integration may be enhanced by including computer-assisted instruction as part of CAT. Clients can be better prepared to use their test results by being more aware of basic concepts and the general nature of their scores. Relieved of presenting repetitive test interpretation information, counselors have more time to explore clients' perceptions of their test data and the implications of the test data for behavior change. The computer can be used to deliver both text-based and interactive video-based instruction (Sampson, 1990a).
POTENTIAL LIMITATIONS OF COMPUTER-ASSISTED TESTING
Computer-assisted testing can limit, as well as enhance, test administration and interpretation. Although paper-and-pencil and computer administration of tests often produce equivalent results, variations in results have sometimes been found to exist. French (1986) recommended that the equivalency of results from different types of administration modes needs to be established for each instrument. Establishing equivalency will reduce the likelihood that computer administration is influencing the nature of test results. Scoring errors represent another potential limitation for computer-assisted test administration. Most (1987) noted that, "The computer itself does not contribute error, but the complex nature of computer programming and the difficulty involved in reading computer programs or code makes it easy to make program errors which are difficult to find" (p. 377).
Concerns have been raised about the validity of computer-based test interpretation. Eyde and Kowal (1987) found differences in CBTI reports generated from a single set of scores from one instrument. Differences also were noted in their study between the CBTI reports and the judgments of a clinician. Eyde and Kowal (1987) stated, "Buyers should be aware of the limitations of computer products and remind themselves that computer output is only as good as the data behind the decision rules used to produce the interpretation" (p. 407). Ethical concerns also exist about counselor misuse of CBTI. Unqualified counselors may be more likely to use CBTI reports to compensate for a lack of training and experience. By using CBTI to replace rather than supplement counselor judgment, counselors become more dependent on the potentially dubious validity of some CBTI software and are less likely to integrate data from valid CBTI reports effectively with other sources of client data due to their lack of background knowledge.
Counselors should become familiar with existing CAT applications (see Krug, 1993; Walz, Bleuer, & Maze, 1989) and the various professional standards that relate to CAT. Counselors then should carefully select and effectively implement valid software that is subsequently evaluated in terms of service delivery impact.
The use of CAT can either enhance or limit the effectiveness of testing in counseling and therapy. Having an open mind about the potential of this technology and a willingness to change needs to be matched with good critical thinking skills and a healthy skepticism for any innovation promising substantial benefits from minimal efforts. It is the responsibility of counselors to guide the design and use of this technology.
Byers, A. P. (1981). Psychological evaluation by means of an on-line computer. Behavior Research Methods & Instrumentation, 13, 585-587.
Eyde, L. D., & Kowal, D. M. (1987). Computerised test interpretation services: Ethical and professional concerns regarding U.S. producers and users. Applied Psychology: An International Review, 36(3/4), 401-417.
French, C. F. (1986). Microcomputers and psychometric assessment. British Journal of Guidance and Counselling, 14, 33-45.
Krug, S. E. (1993). PSYCHWARE SOURCEBOOK: A resource guide to computer based products for assessment in psychology, education, and business (4th Ed.). Champaign, IL: Metritech.
Most, R. (1987). Levels of error in computerized psychological inventories. Applied Psychology: An International Review, 36(3/4), 375-383.
Roid, G. H., & Gorsuch, R. L. (1984). Development and clinical use of test-interpretive programs on microcomputers. In M. D. Schwartz (Ed.), Using computers in clinical practice: Psychotherapy and mental health applications (pp. 141-149). New York: Haworth Press.
Sampson, J. P., Jr. (1990a). Computer applications and issues in using tests in counseling. In C. E. Watkins, Jr. & V. L. Campbell (Eds.), Testing in counseling practice (pp. 451-474). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
Sampson, J. P., Jr. (1990b). Computer-assisted testing and the goals of counseling psychology. The Counseling Psychologist, 18, 227-239.
Walz, G. R., Bleuer, J. C., & Maze, M. (Eds). (1989). Counseling software guide: A resource for the guidance and human development professions. Alexandria, VA: American Association for Counseling and Development.
Library Reference Search
Please note that this site is privately owned and is in no way related to any Federal agency or ERIC unit. Further, this site is using a privately owned and located server. This is NOT a government sponsored or government sanctioned site. ERIC is a Service Mark of the U.S. Government. This site exists to provide the text of the public domain ERIC Documents previously produced by ERIC. No new content will ever appear here that would in any way challenge the ERIC Service Mark of the U.S. Government.