ERIC Identifier: ED355250
Publication Date: 1992-08-00
Author: Rudner, Lawrence - Farris, Michael P.
Clearinghouse on Tests Measurement and Evaluation Washington DC.
A Precedent for Test Validation. ERIC/TM Digest.
In December 1991, the South Carolina Supreme Court ordered the state Board of
Education to stop using the Education Entrance Examination (EEE) for licensing
parents who want to teach their children at home. That ruling struck down a
statute requiring parents with only high school diplomas to pass the EEE before
being approved as home schoolers.
In ruling that the test's validation process did not meet a standard of
reasonableness, the Court established a significant precedent for test
validation. This digest discusses the South Carolina Supreme Court's decision
and its implications for future validation efforts.
South Carolina's home-schooling statute required
the Board of Education to evaluate the suitability of the EEE, a test designed
as an admissions test for teacher education programs, as a test for licensing
home schoolers. Through the state Department of Education, the Board of
Education contracted a traditional content-based validation study. A panel of
judges was asked to evaluate each item on the EEE for task-relatedness and bias.
Of the 33 panelists, 17 were home schoolers, and the remaining 16 were public
school and college teachers. Panelists who were not home schoolers received no
description of the requirements for successful home schooling.
For task-relatedness, the panelists had to decide whether the knowledge or
skill needed to answer the particular item was "a necessary prerequisite" for
home schoolers. To evaluate bias, the panelists considered whether the test
items would offend or unfairly penalize any group of home schoolers because of
gender, ethnicity, religion, or socio-economic status.
The contractor scored the entire panel's responses for the items on the EEE.
For the entire panel, the ratings were:
presence of bias
%The contractor reported to the Department of Education that these scores
were good enough to validate the EEE's use for home schoolers. The Department
then accepted the EEE as provided in the statute.
The plaintiffs, the Home School Legal Defense
Association, brought a class action suit challenging the validation process.
They convinced the Court that the validation was defective for two main reasons:
panelists were not given a "job analysis" or description of successful home
the panel was not qualified to make the judgments expected of them.
Since 16 of the 33 panelists knew nothing about home schooling and received
no information about its prerequisites, the Court ruled that relying on their
evaluations of task-relatedness in validating the EEE was unreasonable.
Reanalyzing the contractor's data, the plaintiffs found large differences in
the task-relatedness ratings for the subgroup of home schoolers compared with
the entire panel. The subgroup ratings were:
presence of bias
%The Court noted that the task-relatedness scores of teacher panelists were
20% to 44% higher than the scores of home-schooling panelists. Panelists with no
knowledge of home schooling found the EEE items more task-related to home
schooling than did panelists familiar with home schooling.
THE STATE'S POSITION
The Board of Education relied heavily
on two main arguments to justify using the EEE for home schoolers. First, the
board argued that the EEE is designed to test basic literacy, an underlying
qualification for teaching in either the public school or the home. This
argument relied on the theory of validity generalization, which means that the
validity of a test established for one task may be assumed or generalized for
another task if the tasks are reasonably similar. The Court dismissed this
argument because it overlooks a specific requirement of the statute--validation
of the EEE for home schoolers. The state legislature considered the tasks of
home schooling and public school teaching very different.
The Board of Education also argued that home schoolers who have taken the EEE
have a high pass rate: 258 of 310 test takers have received a passing score. The
Court found that just because home schoolers can pass the EEE does not mean it
is reasonable to require them to do so. At best, the high pass rate indicates
that the EEE requirement has infringed little on parents' entitlement to home
schooling; it does not, however, justify imposing such a requirement.
IMPLICATIONS FOR TEST VALIDATION
The Court did not accept
the argument that the validity of the EEE generalizes from its original purpose,
as an admissions test for teacher-education programs, to its use as a licensure
examination for home schoolers. Educators need to base arguments for validity
generalization on the accumulated research.
If the EEE or any other test were valid as a licensure examination for home
schoolers in 30 other states, then we could reasonably expect it to be valid in
South Carolina as well. But here, the state's purpose differed from the original
purpose of the test. The Board of Education presented no evidence that the
skills needed for home schooling are similar to the skills needed to succeed in
Finally, we cannot generalize from an unvalidated test. The Board of
Education failed to present any evidence that the EEE was valid for any purpose.
Passing rates alone do not prove validity. The Board of Education argued that
the test was not an imposition for home schoolers because of high passing rates.
But the Court found that argument irrelevant. People who can effectively teach
their children should be permitted to do so, regardless of their numbers. The
passing rate only indicates a test's difficulty; it has nothing to do with
validity. A perfectly valid test can have a zero pass rate. And a perfectly
invalid test can have a 100% pass rate.
To judge the validity of a test for a particular use, we need to know which
skills we're testing. With no description of home schooling, the plaintiffs
argued, the panelists who were not home schoolers were not necessarily qualified
to rate the test items. If the contractor had given them a job analysis,
however, the Board of Education could have argued that those panelists were
familiar with home-schooling tasks and thus qualified to judge the relevance of
the test items.
Further, we need to clearly establish the expertise of the judges. In this
case, the panelists were chosen to be representative college staff, public
school teachers, and home schoolers. Still, they were not necessarily qualified.
No evidence indicated that any of the judges, including those representing home
schoolers, were qualified to make the types of judgments expected of them.
For more detail on this case, see Opinion No.
23526, by South Carolina Supreme Court Judge the Honorable C. J. Gregory,
December 9, 1991. (ED 338 719)
American Psychological Association (1986). Standards for Educational and
Psychological Tests and Manuals. Washington, DC: APA.
Equal Employment Opportunity Commission (1978). Uniform Guidelines on
Employee Selection Procedures, Federal Register 43, 116, 38295-38309.
Lawrence v South Carolina State Board of Education, 412 s.e.2d 394, 71 Ed.
Law Reptr. 1226 (S.C. 1991).
Rafilson, F. (1991). The Case for Validity Generalization, ERIC/TM Digest,