ERIC Identifier: ED388887
Publication Date: 1995-01-30
Author: Gysbers, Norman C.
Source: ERIC Clearinghouse on
Counseling and Student Services Greensboro NC.
Evaluating School Guidance Programs. ERIC Digest.
"Demonstrating accountability through the measured effectiveness of the
delivery of the guidance program and the performance of the guidance staff helps
ensure that students, parents, teachers, administrators, and the general public
will continue to benefit from quality comprehensive guidance programs" (Gysbers
& Henderson, 1994, p. 362). To achieve accountability, evaluation is needed
concerning the nature, structure, organization and implementation of school
district/building guidance programs; the school counselors and other personnel
who are implementing the programs; and the impact the programs are having on
students, the schools where they learn, and the communities in which they live.
Thus, the overall evaluation of school district/building guidance programs needs
to be approached from three perspectives: program evaluation, personnel
evaluation, and results evaluation (Gysbers & Henderson, 1994).
GUIDANCE PROGRAM EVALUATION
Guidance program evaluation
asks two questions. First, is there a written guidance program in the school
district? And second, is the written guidance program the actual implemented
program in the buildings of the district? Discrepancies between the written
program and the implemented program, if present, will come into sharp focus as
the program evaluation process unfolds.
To conduct program evaluation, program standards are required. Program
standards are acknowledged measures of comparison or the criteria used to make
judgments about the adequacy of the nature and structure of the program as well
as the degree to which the program is in place. For example, here is a program
The school district is able to demonstrate that all students are
provided the opportunity to gain knowledge, skills, values, and
attitudes that lead to a self-sufficient, socially responsible life.
(Gysbers & Henderson, 1994, p. 481)
To make judgments about guidance programs using standards, evidence is needed
concerning whether or not the standards are being met. In program evaluation
such evidence is called documentation. Using the standard listed above, evidence
that the standard is in place might include the following:
1. A developmentally appropriate guidance curriculum
that teaches all students the knowledge and skills they need to be
self-sufficient and lead socially responsible lives.
Yearly schedule that incorporates the classroom guidance plan (Gysbers &
Henderson, 1994, p. 482).
Documentation of such evidence could include:
guidance curriculum guides
teachers' and counselors' unit and lesson plans
yearly master calendar for the guidance program
curriculum materials (Gysbers & Henderson, 1994, p. 482)
Sometimes the program evaluation process is called a program audit. The
American School Counselor Association, for example, uses the "term" audit in its
program evaluation materials. The Association has developed guidelines for a
program audit for secondary schools (ASCA, 1986), for middle/junior high schools
(ASCA, 1990b), and for elementary schools (ASCA, 1990a).
GUIDANCE PROGRAM PERSONNEL EVALUATION
begins with the organizational structure and activities of the guidance program
in a school district. A major first step is the development of job descriptions
that are based directly on the structure and activities of a school district's
Using the Missouri Comprehensive Guidance Program framework for example, the
job description of a school counselor would include the following key duties:
implementing the guidance curriculum; counseling individuals and small groups
concerning their educational and occupational plans; counseling individuals and
small groups with immediate needs and specific problems; consulting with parents
and teachers; referring students to appropriate community agencies;
coordinating, conducting, and being involved with activities that improve the
operation of the school; evaluating and updating the guidance program; and
continuing professional development (Starr & Gysbers, 1993). (For examples
of job descriptions of other guidance personnel including director of guidance,
career guidance center technician, and high school registrar, see Gysbers &
Henderson, 1994, 422-428).
Guidance program personnel evaluation is based directly on their job task
descriptions and usually has two parts: a formative part (supervision) and a
summative part (evaluation). The job task description identifies the performance
areas to be supervised and evaluated. Gysbers and Henderson (1994) have
developed an extensive listing of job task descriptors for school counselors
grouped under the basic guidance program components of guidance curriculum,
individual planning, responsive services, and system support plus the areas of
professional relationships and professional responsibilities.
PROGRAM RESULTS EVALUATION
Having established that a
guidance program is operating in a school district through program evaluation,
and having established through personnel evaluation that school counselors and
other guidance program personnel are carrying out the duties listed on their job
descriptions 100% of the time, it now is possible to evaluate the results of the
program. Johnson (1991) suggested that there are long-range, intermediate,
immediate, and unplanned-for results that need consideration. According to
Johnson, long-range results focus on how programs affect students after they
have left school. Usually long-range results are gathered using follow-up
studies. Intermediate results focus on the knowledge and skills all students may
gain by graduation from participating in the guidance program. Immediate results
are the knowledge and skills students may gain from participating in specific
guidance activities. Finally, the possibility of unplanned-for results that may
occur as a consequence of guidance activities conducted as a part of the
guidance program also need to be taken into account.
For the purposes of this digest, illustrations of immediate and intermediate
results evaluation using the structure of the Missouri Comprehensive Guidance
Program Model (Starr & Gysbers, 1993) are presented in the form of two
research questions. First, do students master guidance competencies as a result
of their participation in the Guidance Curriculum Component of the Model
(immediate evaluation)? Second, do students develop and use career plans as a
result of their participation in the Individual Planning Component of the Model
IMMEDIATE EVALUATION--GUIDANCE COMPETENCY MASTERY
students master guidance competencies? Johnson (1991) outlined the following
procedures to answer this question for immediate results. First the competencies
to be mastered need to be identified. Second what results (what students should
be able to write, what they may be able to talk about, or what they may be able
to do) are specified. Then who will conduct the evaluation is decided. This is
followed by when the evaluation is done. Then criteria are established so that
judgments can be made about students' mastery of guidance competencies. Finally,
how all of this is done is specified.
Do students master guidance competencies? Another way to conduct immediate
evaluation, to measure mastery of guidance competencies, is the use of a
confidence survey. In this format, guidance competencies are listed and students
are asked to rate how confident they are that they have mastered these
competencies. The confidence survey can then be used as a pre-post measure. Gain
scores can be obtained and related to such measures as academic achievement and
vocational identity. (Gysbers, Hughey, Starr, & Lapan, 1992; Gysbers, Lapan,
Multon, & Lukin, 1992; Lapan, Gysbers, Hughey, & Arni, 1993).
INTERMEDIATE EVALUATION--CAREER PLANS
Do students develop
and use career plans? In making judgments concerning the career plans of
students, criteria need to be identified as to what makes good plans. Four
criteria are recommended; plans need to be comprehensive, developmental,
student-centered and student-directed, and competency based.
Based on these criteria, one way to evaluate students' career plans is to
judge the extent to which the activities included in the Individual Planning
Component of the guidance program lead to the development of plans that meet
these criteria. A second way is to make judgments about the adequacy of the plan
contents. Finally, a third way is to judge their use. Do students actually use
their career plans in planning for the future?
In order to fully evaluate comprehensive school
guidance programs, three forms of evaluation are required. First, the program
must be reviewed using program standards, evidence, and documentation to
establish that there is a written guidance program in a school district and/or
building and that the written program is the implemented program. Second,
guidance program personnel need job descriptions derived directly from the
program so that evaluation forms can be developed and used for formative and
summative personnel evaluation. Third, results evaluation that focuses on the
impact of the guidance and counseling activities in the guidance curriculum,
individual planning, responsive services, and system support components of a
comprehensive guidance program is mandatory.
American School Counselor Association (1986).
Professional development guidelines for secondary school counselors: a self
audit. Alexandria, VA: Author
American School Counselor Association (1990a). Professional development
guidelines for elementary school counselors: a self audit. Alexandria, VA:
American School Counselor Association (1990b). Professional development
guidelines for middle/junior high school counselors: a self audit. Alexandria,
Gysbers, N.C., Lapan, R.T., Multon, K.D., & Lukin, L. (1992). Missouri
guidance competency evaluation surveys, grades 4-6, 6-9, and 9-12. Columbia, MO:
Center for Educational Assessment
Gysbers, N.C., & Henderson, P. (1994). Developing and managing your
school guidance program (2nd ed.). Alexandria, VA: American Counseling
Johnson, C.D. (1991). Assessing results. In S.K. Johnson & E.A. Whitfield
(Eds.), Evaluating guidance programs. Iowa City, IA: American College Testing
Lapan, R.T., Gysbers, N., Hughey, K., & Arni, T.J. (1993). Evaluating a
guidance and language arts unit for high school juniors. Journal of Counseling
and Development, 71, 444-451
Starr, M., & Gysbers, N.C. (1993). Missouri comprehensive guidance: a
model for program development, implementation, and evaluation (Rev. ed.).
Jefferson City, MO: Missouri Department of Elementary and Secondary Education