ERIC Identifier: ED468593
Publication Date: 2002-09-00
Author: Stoltman, Joseph P.
Source: ERIC Clearinghouse for Social Studies/Social Science Education Bloomington IN.
The 2001 National Assessment of Educational Progress in Geography. ERIC Digest.
The National Assessment of Educational Progress (NAEP) in geography is a periodic survey of geographic knowledge and skill of students at grades four, eight, and twelve. NAEP is administered by the National Center for Educational Statistics (NCES) in the United States Department of Education's Office of Educational Research and Improvement (OERI). The first NAEP in geography was conducted in 1994 and the second in 2001. Like the first assessment, the second assessment probed students' ability to recall, understand, analyze, and interpret geographic information. Students were also required to apply geography content to the completion of various practical tasks.
Educators and researchers can evaluate progress in geography education by comparing the 2001 data with that from the 1994 NAEP in geography. Such information is important for several reasons, two of which are paramount. First, by comparing the 1994 and 2001 results, educators and researchers can evaluate the changes in geography education between the first and second administration of the assessment. This longitudinal information is important for both teachers and policy makers as they evaluate the effects of school reform on student progress in geography. Second, the 2001 administration is a snapshot of performance in geography at a single point in time by a national sample of students in grades four, eight, and twelve. This information will enable educators and researchers to evaluate whether or not children in the United States are developing the geographic skills and knowledge essential for effective participation in the economic and political activities of the nation.
This Digest discusses: (1) the framework of the 2001 NAEP in geography, (2) the findings, (3) comparisons of the 1994 and 2001 national assessments in geography, and (4) conclusions about the significance and usefulness of the national assessment for the teaching of geography.
A single comprehensive framework guided the structure of the geography assessment in both 1994 and 2001. Three sub-content areas of geography constituted the framework. First, space and place: knowledge of geography as it relates to particular places on Earth, to spatial patterns on Earth's surface, and to physical and human patterns that shape such spatial patterns. Second, environment and society. Third, spatial dynamics and connections: knowledge of geography as it relates to spatial connections among people, places, and regions.
The nature of the framework necessitated that the assessment include both multiple-choice questions and constructed-response questions for which the students wrote their own responses. Assessment items measured three cognitive categories: (1) knowing; (2) understanding; and (3) applying. The first category of questions asked students to observe and recall information. The second category asked students to attribute meaning to an observation and to explain events. The third category asked students to hypothesize, use reasons, and solve problems. Fourth grade students were asked more knowing types of questions; the assessment included more applying questions in the eighth grade version, and even more in the twelfth grade version. In most respects the assessment required students to apply knowledge to higher-level cognitive operations rather than to recall information. The constructed response questions challenged students to write answers ranging in length from a few words or sentences to several paragraphs.
REPORT OF STUDENT PERFORMANCE.
Results for each grade -- fourth, eighth, and twelfth -- are reported according to three achievement levels: Basic, Proficient, and Advanced. These achievement levels were determined by expert judgments about what students should know and be able to do in geography at each of the grade levels. The Basic level indicates partial mastery of knowledge and skills fundamental for proficient work at each grade. A score of Basic means additional knowledge and skills are necessary for competent performance in geography. The Proficient level represents solid academic performance and competencies in working with challenging subject matter. The Advanced level signifies superior performance, demonstrating excellence in knowing and using geography. Two percent of fourth graders, 4 percent of eighth graders, and 1 percent of twelfth graders achieved the Advanced level. Nineteen percent of fourth graders, 26 percent of eighth graders, and 23 percent of twelfth graders attained the Proficient level. Fifty-three percent of fourth graders, 44 percent of eighth graders, and 47 percent of twelfth graders reached the Basic level. Twenty-six percent of fourth graders, 26 percent of eighth graders, and 29 percent of twelfth-graders scored below the Basic level.
The assessment revealed descriptive as well as statistically significant differences on performance between major subgroups of the population. For example, males at each grade scored higher on average than female students. At grades four and eight, white and Asian/Pacific Islander students scored higher on average than black, Hispanic, and Native American students. White, Asian/Pacific Islander, and Native American students scored higher in the twelfth grade than black and Hispanic students. In grades eight and twelve, the more education that students reported their parents had attained, the higher the student performance on the assessment. At all three grades, students attending non-public schools performed at a higher level than did students attending public schools. Students attending central city schools scored lower than students in urban fringe areas, large towns, rural areas, or small towns. Students eligible for free or reduced price lunch (meeting poverty guidelines) scored lower on average than ineligible students.
COMPARISON OF THE 1994 AND 2001 GEOGRAPHY ASSESSMENTS.
The overall geography scores for students at the fourth and eighth grades were higher in 2001 than in 1994. The difference in performances of twelfth grade students in 2001 and 1994 was statistically insignificant. At both fourth and eighth grades, the improvements in performance occurred among the lowest performing students. The proportion of students who performed at or above the Basic level increased within the fourth and eighth grades between 1994 and 2001. There were no statistically significant changes from 1994 to 2001 in the percentage of students at any grade at or above the Proficient level.
In addition to collecting data about student performance, NAEP also collected information about the classroom context of teaching and learning. A higher percentage of fourth grade teachers indicated that they were very prepared to teach geography than in 1994. Forty-four percent of eighth grade teachers reported that they were very prepared to teach geography.
Instructional time was greater in 2001 than in 1994 for the following geography topics: map and globe studies at the eighth grade; the study of natural resources at the eighth and twelfth grades; and countries and cultures at the eighth grade.
The amount of classroom instruction as suggested by geography courses taken revealed the following patterns: a higher percentage of eighth grade students reported studying geography in grades six, seven, and eight in 2001 than in 1994, and eighth grade students who took more geography had higher average scores than those who took it for fewer years. The percentage of twelfth grade students reporting geography courses in each of the high school years was greater in 2001 than in 1994, but students who took one year or less of geography scored higher on average than those who took three or four years of geography.
Students at fourth, eighth, and twelfth grades who used Internet or CD materials to a small or moderate extent scored higher than students who did not use those tools.
The 2001 geography test also included special needs students and reported no significant differences at the fourth and twelfth grade levels when special accommodations were provided for administering the test.
While student performance in geography since 1994 has generally improved, a large proportion of students in 2001 did not reach either the Basic or Proficient levels and did not demonstrate achievement in the essential content and skills in geography judged necessary for responsible citizenship. Although more teachers believe they are very prepared to teach geography, and students at the eighth and twelfth grades are taking more geography courses, performance remains low.
The review of the released items suggests that many students in the early grades do not know basic information such as the name and location of the state where they live. Similarly, students at grades eight and twelve do somewhat better with definitional information, but a large proportion were unable to analyze the information related to an environmental issue, provide reasons for or consequences emanating from the issue, or suggest a possible solution to the geographic issue or problem. Items that require those steps are rigorous and challenging. To answer them successfully, students must regularly have opportunities to apply content and skills to issues of geographic significance.
Also, the alignment between the NAEP Geography Framework and the National Content Standards in Geography is vague. Geography experts can conceptualize the linkages, but they are not readily apparent to many people. The national content standards influence instruction, student materials, and teacher preparation and professional development. The alignment between the two frameworks should be improved to enhance the usefulness of both for teachers, students, and parents.
SOURCES OF INFORMATION ABOUT THE NAEP IN GEOGRAPHY.
Information in this Digest is from The Nation's Report Card: Geography 2001. To order this publication or any other NAEP-related products, contact Education Publications Center (ED Pubs), U.S. Department of Education, P.O. Box 1398, Jessup, MD 20794-1398; toll-free 877-433-7827; FAX 301-470-1244. This publication is also available from the ERIC Document Reproduction Service (EDRS), 7420 Fullerton Road, Suite 110, Springfield, VA 22153-2852; toll-free 800-443-3742.
The NAEP Web site contains information about the NAEP in geography and general information about assessment, publications, and analysis tools used by various NAEP projects <http://nces.ed.gov/nationsreportcard>.
REFERENCES AND ERIC RESOURCES.
The following list of resources includes references used to prepare this Digest. The items followed by an ED number are available in microfiche and/or paper copies from the ERIC Document Reproduction Service (EDRS). For information about prices, contact EDRS, 7420 Fullerton Road, Suite 110, Springfield, Virginia 22153-2852; telephone numbers are (703) 440-1400 and (800) 443-3742. Entries followed by an EJ number, annotated monthly in CURRENT INDEX TO JOURNALS IN EDUCATION (CIJE), are not available through EDRS. However, they can be located in the journal section of most larger libraries by using the bibliographic information provided, requested through Interlibrary Loan, or ordered from commercial reprint services.
Bednarz, Sarah Witham, and Others. GEOGRAPHY FOR LIFE: NATIONAL GEOGRAPHY STANDARDS. Washington, DC: National Geographic Society, 1994. ED 375 073.
Fromboluti, Carol Sue. HELPING YOUR CHILD LEARN GEOGRAPHY. Washington, DC: U.S. Department of Education, 1990. ED 313 316.
GEOGRAPHY FRAMEWORK FOR THE 1994 NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS. NAEP GEOGRAPHY CONSENSUS PROJECT. Washington, DC: National Assessment Governing Board, 1994. ED 373 019.
Jones, Lyle V. "A History of the National Assessment of Educational Progress and Some Questions about Its Future." EDUCATIONAL RESEARCHER 25 (October 1996): 15-22. EJ 537 026.
Persky, Hilary R., and Others. NAEP 1994 GEOGRAPHY REPORT CARD: FINDINGS FROM THE NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS. Washington, DC: National Center for Education Statistics, 1996. ED 398 145.
U.S. Department of Education. Office of Educational Research and Improvement. National Center for Education Statistics. THE NATION'S REPORT CARD: GEOGRAPHY 2001. NCES 2002-484, by A. R. Weiss, A. D. Lutkus, B. S. Hildebrant, and M. S. Johnson. Washington, DC: 2002.
Library Reference Search
Please note that this site is privately owned and is in no way related to any Federal agency or ERIC unit. Further, this site is using a privately owned and located server. This is NOT a government sponsored or government sanctioned site. ERIC is a Service Mark of the U.S. Government. This site exists to provide the text of the public domain ERIC Documents previously produced by ERIC. No new content will ever appear here that would in any way challenge the ERIC Service Mark of the U.S. Government.