ERIC Identifier: ED468593
Publication Date: 2002-09-00
Author: Stoltman, Joseph P.
Source: ERIC Clearinghouse for
Social Studies/Social Science Education Bloomington IN.
The 2001 National Assessment of Educational Progress in
Geography. ERIC Digest.
The National Assessment of Educational Progress (NAEP) in geography is a
periodic survey of geographic knowledge and skill of students at grades four,
eight, and twelve. NAEP is administered by the National Center for Educational
Statistics (NCES) in the United States Department of Education's Office of
Educational Research and Improvement (OERI). The first NAEP in geography was
conducted in 1994 and the second in 2001. Like the first assessment, the second
assessment probed students' ability to recall, understand, analyze, and
interpret geographic information. Students were also required to apply geography
content to the completion of various practical tasks.
Educators and researchers can evaluate progress in geography education by
comparing the 2001 data with that from the 1994 NAEP in geography. Such
information is important for several reasons, two of which are paramount. First,
by comparing the 1994 and 2001 results, educators and researchers can evaluate
the changes in geography education between the first and second administration
of the assessment. This longitudinal information is important for both teachers
and policy makers as they evaluate the effects of school reform on student
progress in geography. Second, the 2001 administration is a snapshot of
performance in geography at a single point in time by a national sample of
students in grades four, eight, and twelve. This information will enable
educators and researchers to evaluate whether or not children in the United
States are developing the geographic skills and knowledge essential for
effective participation in the economic and political activities of the nation.
This Digest discusses: (1) the framework of the 2001 NAEP in geography, (2)
the findings, (3) comparisons of the 1994 and 2001 national assessments in
geography, and (4) conclusions about the significance and usefulness of the
national assessment for the teaching of geography.
A single comprehensive framework guided the
structure of the geography assessment in both 1994 and 2001. Three sub-content
areas of geography constituted the framework. First, space and place: knowledge
of geography as it relates to particular places on Earth, to spatial patterns on
Earth's surface, and to physical and human patterns that shape such spatial
patterns. Second, environment and society. Third, spatial dynamics and
connections: knowledge of geography as it relates to spatial connections among
people, places, and regions.
The nature of the framework necessitated that the assessment include both
multiple-choice questions and constructed-response questions for which the
students wrote their own responses. Assessment items measured three cognitive
categories: (1) knowing; (2) understanding; and (3) applying. The first category
of questions asked students to observe and recall information. The second
category asked students to attribute meaning to an observation and to explain
events. The third category asked students to hypothesize, use reasons, and solve
problems. Fourth grade students were asked more knowing types of questions; the
assessment included more applying questions in the eighth grade version, and
even more in the twelfth grade version. In most respects the assessment required
students to apply knowledge to higher-level cognitive operations rather than to
recall information. The constructed response questions challenged students to
write answers ranging in length from a few words or sentences to several
REPORT OF STUDENT PERFORMANCE.
Results for each grade --
fourth, eighth, and twelfth -- are reported according to three achievement
levels: Basic, Proficient, and Advanced. These achievement levels were
determined by expert judgments about what students should know and be able to do
in geography at each of the grade levels. The Basic level indicates partial
mastery of knowledge and skills fundamental for proficient work at each grade. A
score of Basic means additional knowledge and skills are necessary for competent
performance in geography. The Proficient level represents solid academic
performance and competencies in working with challenging subject matter. The
Advanced level signifies superior performance, demonstrating excellence in
knowing and using geography. Two percent of fourth graders, 4 percent of eighth
graders, and 1 percent of twelfth graders achieved the Advanced level. Nineteen
percent of fourth graders, 26 percent of eighth graders, and 23 percent of
twelfth graders attained the Proficient level. Fifty-three percent of fourth
graders, 44 percent of eighth graders, and 47 percent of twelfth graders reached
the Basic level. Twenty-six percent of fourth graders, 26 percent of eighth
graders, and 29 percent of twelfth-graders scored below the Basic level.
The assessment revealed descriptive as well as statistically significant
differences on performance between major subgroups of the population. For
example, males at each grade scored higher on average than female students. At
grades four and eight, white and Asian/Pacific Islander students scored higher
on average than black, Hispanic, and Native American students. White,
Asian/Pacific Islander, and Native American students scored higher in the
twelfth grade than black and Hispanic students. In grades eight and twelve, the
more education that students reported their parents had attained, the higher the
student performance on the assessment. At all three grades, students attending
non-public schools performed at a higher level than did students attending
public schools. Students attending central city schools scored lower than
students in urban fringe areas, large towns, rural areas, or small towns.
Students eligible for free or reduced price lunch (meeting poverty guidelines)
scored lower on average than ineligible students.
COMPARISON OF THE 1994 AND 2001 GEOGRAPHY ASSESSMENTS.
The overall geography scores for students at the fourth and eighth grades
were higher in 2001 than in 1994. The difference in performances of twelfth
grade students in 2001 and 1994 was statistically insignificant. At both fourth
and eighth grades, the improvements in performance occurred among the lowest
performing students. The proportion of students who performed at or above the
Basic level increased within the fourth and eighth grades between 1994 and 2001.
There were no statistically significant changes from 1994 to 2001 in the
percentage of students at any grade at or above the Proficient level.
In addition to collecting data about student performance, NAEP also collected
information about the classroom context of teaching and learning. A higher
percentage of fourth grade teachers indicated that they were very prepared to
teach geography than in 1994. Forty-four percent of eighth grade teachers
reported that they were very prepared to teach geography.
Instructional time was greater in 2001 than in 1994 for the following
geography topics: map and globe studies at the eighth grade; the study of
natural resources at the eighth and twelfth grades; and countries and cultures
at the eighth grade.
The amount of classroom instruction as suggested by geography courses taken
revealed the following patterns: a higher percentage of eighth grade students
reported studying geography in grades six, seven, and eight in 2001 than in
1994, and eighth grade students who took more geography had higher average
scores than those who took it for fewer years. The percentage of twelfth grade
students reporting geography courses in each of the high school years was
greater in 2001 than in 1994, but students who took one year or less of
geography scored higher on average than those who took three or four years of
Students at fourth, eighth, and twelfth grades who used Internet or CD
materials to a small or moderate extent scored higher than students who did not
use those tools.
The 2001 geography test also included special needs students and reported no
significant differences at the fourth and twelfth grade levels when special
accommodations were provided for administering the test.
While student performance in geography since
1994 has generally improved, a large proportion of students in 2001 did not
reach either the Basic or Proficient levels and did not demonstrate achievement
in the essential content and skills in geography judged necessary for
responsible citizenship. Although more teachers believe they are very prepared
to teach geography, and students at the eighth and twelfth grades are taking
more geography courses, performance remains low.
The review of the released items suggests that many students in the early
grades do not know basic information such as the name and location of the state
where they live. Similarly, students at grades eight and twelve do somewhat
better with definitional information, but a large proportion were unable to
analyze the information related to an environmental issue, provide reasons for
or consequences emanating from the issue, or suggest a possible solution to the
geographic issue or problem. Items that require those steps are rigorous and
challenging. To answer them successfully, students must regularly have
opportunities to apply content and skills to issues of geographic significance.
Also, the alignment between the NAEP Geography Framework and the National
Content Standards in Geography is vague. Geography experts can conceptualize the
linkages, but they are not readily apparent to many people. The national content
standards influence instruction, student materials, and teacher preparation and
professional development. The alignment between the two frameworks should be
improved to enhance the usefulness of both for teachers, students, and parents.
SOURCES OF INFORMATION ABOUT THE NAEP IN
Information in this Digest is from The Nation's Report Card:
Geography 2001. To order this publication or any other NAEP-related products,
contact Education Publications Center (ED Pubs), U.S. Department of Education,
P.O. Box 1398, Jessup, MD 20794-1398; toll-free 877-433-7827; FAX 301-470-1244.
This publication is also available from the ERIC Document Reproduction Service
(EDRS), 7420 Fullerton Road, Suite 110, Springfield, VA 22153-2852; toll-free
The NAEP Web site contains information about the NAEP in geography and
general information about assessment, publications, and analysis tools used by
various NAEP projects <http://nces.ed.gov/nationsreportcard>.
REFERENCES AND ERIC RESOURCES.
The following list of
resources includes references used to prepare this Digest. The items followed by
an ED number are available in microfiche and/or paper copies from the ERIC
Document Reproduction Service (EDRS). For information about prices, contact
EDRS, 7420 Fullerton Road, Suite 110, Springfield, Virginia 22153-2852;
telephone numbers are (703) 440-1400 and (800) 443-3742. Entries followed by an
EJ number, annotated monthly in CURRENT INDEX TO JOURNALS IN EDUCATION (CIJE),
are not available through EDRS. However, they can be located in the journal
section of most larger libraries by using the bibliographic information
provided, requested through Interlibrary Loan, or ordered from commercial
Bednarz, Sarah Witham, and Others. GEOGRAPHY FOR LIFE: NATIONAL GEOGRAPHY
STANDARDS. Washington, DC: National Geographic Society, 1994. ED 375 073.
Fromboluti, Carol Sue. HELPING YOUR CHILD LEARN GEOGRAPHY. Washington, DC:
U.S. Department of Education, 1990. ED 313 316.
GEOGRAPHY FRAMEWORK FOR THE 1994 NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS.
NAEP GEOGRAPHY CONSENSUS PROJECT. Washington, DC: National Assessment Governing
Board, 1994. ED 373 019.
Jones, Lyle V. "A History of the National Assessment of Educational Progress
and Some Questions about Its Future." EDUCATIONAL RESEARCHER 25 (October 1996):
15-22. EJ 537 026.
Persky, Hilary R., and Others. NAEP 1994 GEOGRAPHY REPORT CARD: FINDINGS FROM
THE NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS. Washington, DC: National Center
for Education Statistics, 1996. ED 398 145.
U.S. Department of Education. Office of Educational Research and Improvement.
National Center for Education Statistics. THE NATION'S REPORT CARD: GEOGRAPHY
2001. NCES 2002-484, by A. R. Weiss, A. D. Lutkus, B. S. Hildebrant, and M. S.
Johnson. Washington, DC: 2002.