ERIC Identifier: ED385310
Publication Date: 1994-06-00
Author: Cohen, Arthur M.
Source: ERIC Clearinghouse for Community Colleges Los Angeles CA.
Indicators of Institutional Effectiveness. ERIC Digest.
Within the last few years, American schools have moved toward routinely measuring the outcomes of their educational programs. Two major national associations, the American Association of Community Colleges and the League for Innovation in the Community College, have not only addressed the importance of institutional assessment in community colleges, but also displayed indicators and definitions to estimate effectiveness in the two-year sector (Community College Roundtable, 1990; Doucette and Hughes, 1994). The reports flowing into the ERIC system from individual colleges, researchers, and state systems show a trend toward assessing outcomes. This Digest discusses gains made by community colleges in using the more quantitative indicators to assess outcomes.
The trend toward using indicators is encouraging for those who understand the value of institutional measures that document outcomes, successes, and effectiveness in the various programmatic efforts that community colleges undertake. The arguments in favor of assessment typically boil down to the need to document institutional effects so that students, the public, and the professional community understand how the institutions use their resources in fulfilling their missions.
But the millennium has not yet arrived; pockets of resistance still exist. While some colleges employ exemplary practices, others make "minimalist" efforts at assessment and cite the inability to produce evidence of effective performance, uneven institutional responses, and poor communication as hurdles (Hudgins, 1993). Some of the arguments against assessment are well-founded; for example, a common complaint is that available assessment technologies are fairly primitive. Other arguments are delusive, betraying a reluctance to consider outcomes on the part of educators whose professional life has been dedicated to process, not product. Still other justifications are offered by those who don't understand its usefulness or who shrink from the effort of obtaining reliable data. Some of the antagonists are practical, contending that the political arena in which institutional support is generated and sustained thrives on image, good news, and successes; therefore, the validity of the research methods (if any) used to derive that information are irrelevant.
Another assessment concern involves qualitative indicators. While headway has been made in using quantitative indicators, considerably less progress has been made in employing more abstract indicators of institutional effectiveness, such as general education outcomes. The reason can be attributed in part to the community colleges' diverse clientele, pointing to a disjuncture of methods and motives (Palmer, 1993).
Nevertheless, the move toward assessing institutional effectiveness seems steady and we can anticipate more reports, more concise definitions, more attempts to display institutional value through providing evidence of student progress.
The League for Innovation monograph describes five major missions: transfer; career preparation; basic skills; continuing education and community service; and access; and further breaks down the five missions into 69 sub-sets. The authors suggest assessing the transfer mission variously by measuring student knowledge, the college's transfer rate, the grades that students earn when they transfer to universities, and the number of college credits that universities accept (Doucette and Hughes, 1990). The Community College Roundtable document describes seven missions and thirteen indicators: three listed under student progress; two under career preparation; two under transfer; one in developmental education; two in general education; one under customized education; and two in community development (1994).
The important characteristic of the works cited is that each of them offers an operational definition for each indicator. The League document lists the questions to be asked and the data sources from which answers can be derived for each of the indicators that it sets forth. For example, under the transfer mission, they suggest that the college's transfer rate can be reasonably measured by using "enrollment reports and student profile data; transcripts; student surveys; reports from transfer institutions" (p. 11). The Community College Roundtable document offers two indicators for the transfer mission: "the number and rate who transfer" and "performance after transfer" (p. 11). Cohen (1992) further refined a transfer indicator by specifying the method of calculation within the definition itself: "A transfer rate can be calculated by counting the number of students who enter in a given term with no prior college experience and who receive a minimum of twelve units in the college [divided into the number who] matriculate at a university within the next four years" (p. 45). Thus, the validity of the indicators is suggested by the extent to which they reflect the colleges' basic missions; they provide a continual report on institutional effects.
The indicators that have been articulated also display certain other valuable characteristics. They are readily understandable by members of the college community and the lay public--an important characteristic because an indicator in which the measurement is couched in arcane terms, buried under masses of data, and reliant on esoteric statistical manipulations is not likely to convince anyone that the college is doing a good job. A second important characteristic of indicators is that they must be defined so that the data necessary for measurement are available or can be gathered at minimal expense; few colleges have the resources to conduct involved research studies. And while it is inevitable and unfortunate that outside agencies will use some data comparatively, it is essential that institutions treat indicators as measures of institutional effectiveness so that faculty will buy into the process. The indicators are for the use of the college community itself in estimating how it is doing year after year in each of its major missions.
THE METHODS IN USE
A review of ERIC documents processed for the period between 1982 and March 1994 found 48 studies that included data useful in assessing institutional effectiveness. The follow-up survey of students who had recently graduated, dropped out, or transferred was the predominant form of data receipt. Such surveys typically asked about student satisfaction, jobs obtained, earnings, and further schooling involvements. The contemporary interest in transfer rates was exhibited by a number of studies that calculated the rate of transfer from the colleges to the neighboring universities. Several of these studies also reported student performance at the university. Interestingly, student performance at the university as a criterion of college success and the use of follow-up surveys to estimate student satisfaction and earnings are the two measures that have been traditionally used by community colleges in the past to assess outcomes. Other measures that appeared repeatedly were occupational education outcomes; pre- and post-measures of student learning in college courses and programs; and a few public-image studies in which members of the community were asked what they thought of the college or students were asked if they were satisfied.
A new category of assessment that has achieved considerable interest in recent years includes studies done in response to state or accreditation association mandates. Here, the colleges follow the directives of these agencies and produce documentation which often centers on basic numbers of students according to ethnicity and gender, and patterns of enrollment, attrition, graduation, job placement, and transfer.
The basis for defining institutional effectiveness is readily at hand. Since the community college is a school it is expected to effect learning. But it is also expected to have an impact on the students' successive enrollments in institutions of higher learning and attaining jobs that they might not have received had they not been associated with the community college. Most of the indicators of institutional effectiveness center on one or another of the colleges' basic missions: career upgrading; general education; job entry; literacy development; personal interest; and transfer. As long as indicators are not used to rank colleges normatively, their routine acceptance seems assured.
Cohen, Arthur M. (1992) "The Fear of Knowing and the Ethics of Ignoring," in Vaughan (ed.) Dilemmas of Leadership. San Francisco: Jossey-Bass.
Community College Roundtable. (1994) Community Colleges: Core Indicators of Effectiveness. Washington, D.C.: AACC. (ED 367 411)
Doucette, Don and Hughes, Billie (eds.). (1990) "Assessing Institutional Effectiveness in Community Colleges." Laguna Hills, CA: League for Innovation in the Community College. (ED 324 072)
Heaney, Barbara. (July 1990) "The Assessment of Educational Outcomes." ERIC Digest. Los Angeles: ERIC Clearinghouse for Community Colleges. (ED 321 834)
Hudgins, James L. (1993) "Institutional Effectiveness: A Maturing Movement. Where Do We Go from Here?" Paper presented at the 5th Summer Institute of the Community College Consortium, Madison, WI, June 21, 1993. (ED 358 891)
Palmer, James C. (Fall 1993) "Institutional Accreditation, Student Outcomes
Assessment, and the Open-Ended Institution." In Prager (ed.) New Directions for
Community Colleges, No. 83. San Francisco: Jossey-Bass.
Library Reference Search
Please note that this site is privately owned and is in no way related to any Federal agency or ERIC unit. Further, this site is using a privately owned and located server. This is NOT a government sponsored or government sanctioned site. ERIC is a Service Mark of the U.S. Government. This site exists to provide the text of the public domain ERIC Documents previously produced by ERIC. No new content will ever appear here that would in any way challenge the ERIC Service Mark of the U.S. Government.