ERIC Identifier: ED414519
Publication Date: 1995-00-00
Author: Hiebert, Bryan
Source: ERIC Clearinghouse on
Counseling and Student Services Greensboro NC., Canadian Guidance and
Counselling Foundation Ottawa (Ontario).
A Changing Focus in Evaluation: Linking Process and Outline:
Career counselors continually evaluate their work; they draw conclusions and
develop action plans based on numerous client activities: homework completion,
client engagement, acquisition of interview skills or relaxation skills, number
of employers contacted, and so forth. Both counselors and clients typically know
when counseling is successful. Unfortunately, the evidence used to gauge success
often is not considered evaluation, is not documented, and therefore cannot be
used to back up claims that counseling has been successful.
Conger, Hiebert, and Hong-Farrell (1993) found that counseling is rarely
evaluated in Canada. In some sectors, 40% of counselors reported never formally
evaluating their work. This is unfortunate, since research has shown that career
guidance and counseling consistently produce positive results. (See Killeen
& Kidd, 1991; Oliver & Spokane, 1988.) In fact, Killeen and Kidd found
that 90% of the studies recorded some positive counseling effects. Given these
findings, it is surprising that counselors are not eager to evaluate such a
predictably beneficial experience. Perhaps a new approach to evaluation is
needed; one that counselors see as relevant, practical, and capable of embracing
the informal observations that counselors and clients make about counseling
AN ALTERNATIVE FRAMEWORK
Counseling is interactive.
Client-need determines counseling intervention. The intervention is tailored to
client characteristics and desired outcome. As counseling progresses, counselors
adjust the approach to fit the client's changing situation. Evaluation models
must accommodate this interplay between counselor and client and between process
Figure 1 depicts a framework connecting long-term global impacts of
counseling (e.g., job satisfaction, employability, career maturity); immediate
outcomes associated with counseling (changes in client attitudes, knowledge,
skills); client engagement in the counseling process; and, counselor approaches.
The loops in the framework illustrate the interactive nature of counseling.
Process and outcome interact in a circular fashion. Certain processes foster
particular types of learning and in turn, create certain types of global impact.
Reciprocally, the types of client skill, knowledge, and attitude necessary to
achieve a certain kind of global impact can be identified, and the process
needed to facilitate that learning can also be identified.
============================ FIGURE 1
What is the counselor doing:
Skills, Strategies, Tools.
What is the client doing?
Skill Practice, Homework, Self-Examination.
What did the client learn?
Skills, Knowledge, Attitudes.
What difference did the learning
make? Precursors, Presenting problem,
Evaluating the counseling process requires detailed information which links
counselor activities and client reactions. This helps identify the processes
that promote client change and that aid the development of alternative plans
when sufficient progress is lacking. Client documentation might include
engagement in the counseling process, homework completion, client openness and
honesty, and client follow-through. Such data demonstrate that clients are doing
their part in counseling. Evidence depicting the counselor as an indispensable
part of the process might include the pattern of micro-skills used, the focus in
a counseling session (both content and process), and data showing that an
acceptable procedure for an intervention followed. These factors can be assessed
from counselor case notes, client check-lists, semi-structured interviews, or
Counselors need to be clear about the legitimate outcomes (effects or
products) of counseling (Hiebert 1989, 1994; Killeen & Kidd, 1991; Killeen
White, & Watts, 1993). In Figure 1, counseling outcomes have two major
(1) Learning outcomes: the knowledge, skills, and attitudes that are directly
linked to counseling. These are the legitimate outcomes of counseling:
about self, the labor market, job descriptions, entrance qualifications, and how
to overcome barriers
for decision making, job interviews, self-management, making transition, and
overcoming barriers like anxiety, substance abuse, poor financial planning
towards being planful, belief in self, motivation to look for work, self-esteem,
increased optimism Learning outcomes arise both from counseling and from
learning to apply existing skills to new contexts (e.g., using communication
skills in conflict resolution). These are the legitimate outcomes of counseling,
outcomes that counseling can reasonably hope to influence.
(2) Global outcomes reflect counseling's larger impact on the client's life.
in client-presenting problem
along better with co-workers
stress during job interviews
of time to job offer
Administrators and sponsors often focus on global impact outcomes, but
outcomes are influenced greatly by factors over which counseling has little
control (e.g., the number of jobs available, corporate climate, etc.).
Ultimately, it is important to demonstrate that counseling has affected these
variables, but it is also important to refrain from promising that counseling
can facilitate change in areas over which it has little control.
In addition to the above factors, program administrators and sponsors often
are interested in factors such as adherence to mandates, types of clients, types
of client problems, and client satisfaction with service. These variables are
important to the functioning of the agency, but they are not indicators of the
effectiveness of counseling per se. To emphasize this difference, they are
called "system requirements." System requirements are part of the policies and
procedures governing an agency and departures from these requirements should be
negotiated between the counselor and the agency manager or sponsor.
Although standardized assessment is
emphasized in counseling evaluation, there is frequent incongruence between the
aims of standardized tests and the needs of clients (Killeen & Kidd, 1991).
Implicit in Figure 1 is a far greater emphasis on informal procedures that
document the judgments counselors and clients make about counseling progress.
These include checklists for homework completion, skill mastery, skill
implementation, steps completed in a program; subjective ratings of affective
state (e.g., depression, motivation, stress level, job satisfaction); use of job
interview skills during job interviews (see Hiebert (1991, 1994a, 1994b) for
Currently, these informal measures are in their infancy. Better ways must be
developed to track variables that are part of the counseling process and which
have an influence on client change. For example, "planful" attitude is an
important prerequisite (or co-requisite) to developing a career-action plan.
Therefore, it is important to have a trustworthy and easy-to-use procedure to
track changes in such attitudes.
SUMMARY: A CALL FOR ACTION
Two points underline the main
arguments in this paper.
Counselors, program administrators, and sponsors need to reformulate their view
of evaluation so that it provides an essential link between process and outcome;
and, is an integral part of counseling, coequal with relationship building and
Evaluation needs to be planned and implemented alongside client-change
intervention-not conducted at the end of a program by an external expert.
The scope of evaluation (what constitutes acceptable evidence) needs to be
expanded to include the sorts of data that counselors and clients already
collect on a regular basis. This includes client self-monitoring data, homework
data, quantification of counselor observations in case notes, documentation of
client in-session skill practice, goal attainment scaling, and performance
An evaluation model should assess the informal observations counselors and
clients use to indicate whether they are on the right track, the amount of
progress they are making, and the achieving of desired outcomes. It should
encourage counselors to develop creative ways for documenting and quantifying
those observations, and it should create non-quantified ways of portraying the
evidence that clients, managers and supervisors find acceptable. This will
ensure that evaluation needs are seen as relevant by all concerned: clients,
counselors, agency managers, district supervisors, and funders.
Conger, D. S., Hiebert, B., & Hong-Farrell,
E. (1993). Career and employment counselling in Canada. Ottawa, ON: Canadian
Labour Force Development Board.
Hiebert, B. (1989). Meeting the accountability challenge. In B. O'Byrne,
(Ed.), Natcon 15 (pp. 239-255). Toronto, ON: Ontario College Counsellor's
Hiebert, B., (1994a). A Framework For Quality Control, Accountability, And
Evaluation: Being Clear About The Legitimate Outcomes of Career Counseling.
Canadian Journal of Counselling, 24.
Hiebert, B. (1994b). Moving to the future: Outcome-based comprehensive
guidance and counselling in Alberta schools. Edmonton, AB: Alberta Education.
Killeen, J., & Kidd, J. (1991). Learning outcomes of guidance: A review
of recent research. London, England: National Institute for Careers Education
Killeen, J., White, M., & Watts, A. G. (1993). The Economic Value of
Careers Guidance. London, England: Policy Studies Institute.
Oliver, L. W., & Spokane, A. R. (1988). Career intervention outcome: What
contributes to client gain? Journal of Counseling Psychology, 35, 447-462.