by McCann, Wendy Sherman
Over half a million students from more than 15,000 schools worldwide took part in the Third International Mathematics and Science Study (TIMSS) (Mullis, 1997). All 41 TIMSS countries tested thirteen-year-old students, and could elect to test nine-year-olds and students in the last year of secondary school. The study itself consisted of five components: assessments of student knowledge in mathematics and science; surveys of student and teacher educational practices and beliefs about mathematics and science; analysis of curriculum guides and textbooks from participating countries; videotape analyses of mathematics classrooms in the U.S., Germany, and Japan; and ethnographic investigations of classrooms and policies in the United States, Germany, and Japan. Design and field testing occurred from 1991-1994, data were collected for the curriculum study in 1992-1993, and the main assessments took place during the 1995-1996 school year.
TIMSS ACHIEVEMENT RESULTS
U.S. students in the fourth and eighth grades performed above the international average in the science assessments. Out of 26 nations, U.S. fourth-graders scored significantly lower than students of only one other nation, and U.S. scores were statistically equivalent to five other nations. U.S. scores were significantly higher than those of nineteen countries. All 41 countries participated in the eighth- grade study, with students of nine countries scoring significantly higher than U.S. students. The students of sixteen countries had scores which were significantly lower than U.S. students.
Results from the twelfth-grade assessment were not as positive. U.S. students scored below the international average of 21 participating countries for general science knowledge, and eleven countries significantly outperformed the U.S. The students of only two countries had scores significantly lower than those of the United States. Additionally, sixteen countries took part in a physics assessment, and U.S. twelve-graders scored sixteenth. Students in fourteen of the countries scored significantly higher than U.S. students.
Although it is tempting to compare the raw score rankings of countries participating in TIMSS, the comparisons are not particularly meaningful. "A simple ranking of student scores like a horse race or an Olympic event is not valid. The scores must be read...in broad bands of similar-performing countries" (NSTA, no date given), since each country's score is estimated from the sample of students who took the test, and is accurate over a particular range (Peak, 1996).
Studies like TIMSS are often criticized for comparing "apples and oranges." Baker (1997) contends, however, that international schooling practices are basically the same, so "differences" in the organizational and cultural aspects of educational systems can be studied to explain differences in achievement; he says, "there may be both apples and oranges in the bag, but they are all fruit, and they can be compared and contrasted as such." Furthermore, TIMSS was designed to be as fair and accurate as possible, so students were randomly selected for each country's representative sample to ensure against comparing "all of our students with other nations' best" (OERI, 1997). Finally, the tests were designed in English and rigorously translated into 30 languages, and persons responsible for data collection and scoring were thoroughly trained (Mullis, 1997).
RECOMMENDATIONS FOR CURRICULUM
TIMSS researchers responsible for the curriculum study argue that in the United States, mathematics and science "curricula, textbooks, and teaching are all 'a mile wide and an inch deep'" (USNRC, no date given); since U.S. schools tend to teach a little bit about a lot of topics, rather than a great deal about relatively few topics. Results from the study show that, except for physical science, the curriculum in science is more focused than the curriculum in mathematics. However, the science topics presented in particular grade levels vary widely from state to state. Textbooks compound the problem by including many topics in order to cater to the broadest possible audience; and often teachers feel pressured to cover most or all of the textbook material (USNRC, no date given).
The National Science Teachers Association "is pleased that the TIMSS curriculum study stresses the need for a common vision for science education. That vision exists in the National Science Education Standards" (NSTA, no date given).
Since the National Science Education Standards were not published until December 1995, their impact on U.S. science curricula is not reflected in the TIMSS report. Although the NSTA strongly supports the Standards, the organization cautions against viewing the Standards as a "quick fix" for improving student achievement in science education as a whole (NSTA, no date given).
Others dispute the idea that a "splintered" U. S. curriculum is truly the problem. Biddle (1997) contends that changing standards alone will not change achievement scores; he found that differences in school funding and levels of child poverty between districts could account for more than 25% of the variance of differences in mathematics achievement in the Second International Mathematics Study (SIMS), and were the major predictors of mathematics achievement averages on data from the 1996 National Assessment of Educational Progress (NAEP) exams. Further, Atkin and Black (1997) point out that although eight of the countries with the top ten TIMSS scores have nationally centralized curricula, so do eight of the ten lowest-scoring countries. They claim that a more useful exercise would be to study what other countries are doing to reform their mathematics and science practices, since achievement scores are one aspect of the overall educational picture. In fact, one such study of 13 nations (all TIMSS participants) found that every country was engaged in serious reform efforts, and that virtually all countries were moving toward a curriculum geared to practical application and integration of disciplinary content (Atkin & Black, 1997).
FINDINGS ON TEACHER PRACTICES
Findings from the study of teacher beliefs and practices indicate that teacher preparation in the U.S. differs from most countries in lacking an examination requirement for certification (Mullis, 1997). Close comparison with Japan and Germany showed that those countries require lengthy apprenticeships with mentor teachers before certification, a practice that is uncommon in the U.S. (Peak, 1996). Also, in Japan, practicing teachers have more formal and informal opportunities to interact with their colleagues than do teachers in Germany or the United States (Peak, 1996).
Teachers in the U.S., Japan and Germany reported that uninterested students, students of mixed-ability, and too many students per class were challenges faced in their daily work, and science teachers included the problem of limited demonstration and instructional equipment (Peak, 1996). Findings from the TIMSS videotape study of eighth-grade mathematics classes in the U.S., Germany, and Japan indicate that U.S. and German teachers emphasize skills, whereas Japanese instructors emphasize understanding. Further, U.S. teachers do not develop concepts to the same extent that their German and Japanese counterparts do (Peak, 1996). It is not clear that these findings are applicable to science teaching, but science teachers may be interested in viewing the representative videotaped lessons of mathematics classes from the three countries. The videotapes are available as part of the TIMSS Resource Kit. More detailed information about the teacher and student questionnaires and ethnographic studies can also be found in the Kit (see information below).
WHERE TO FIND MORE INFORMATION
U.S. science teachers will undoubtedly hear more about the Third International Mathematics and Science Study. Information is readily available, and more reports are scheduled for future distribution. ERIC/CSMEE provides links to TIMSS information on the Internet at through the website at http://www.ericse.org. The "TIMSS Resource Kit" [ED 410 122] includes many of the publications listed in the Reference section of this digest, as well as other documents. It is available for downloading at http://nces.ed.gov/timss ; or can be ordered by contacting Lois Peak, TIMSS Project Officer, National Center for Education Statistics, U.S. Department of Education, 555 New Jersey Ave. NW, Washington, D.C. 20208-5574; (202) 219-1333.
Atkin, J. M. & Black, P. (1997). Policy perils of international comparisons: The TIMSS case. "Phi Delta Kappan," 79(1), 22-28. [EJ 550 557]
Baker, D. P. (1997). Surviving TIMSS: Or, everything you blissfully forgot about international comparisons. "Phi Delta Kappan," 79(1), 22-28. [EJ 556 845]
Biddle, B. J. (1997). Foolishness, dangerous nonsense, and real correlates of state differences in achievement. "Phi Delta Kappan," 79(1), 8-13. [EJ 550 555]
Mullis, I. V. S. (1997). "Benchmarking to international achievement: TIMSS as a starting point to examine student achievement" (ORAD-97-1025). Washington, D.C.: Office of Educational Research and Improvement. [ED 410 124]
National Science Teachers' Association. Third international science and mathematics study: A Q&A prepared by the National Science Teachers Association. NSTA press release, retrieved from the world wide web October, 1998. http://www.nsta.org/pressrel/timssqa.htm
Office of Educational Research and Improvement. (1997). "Introduction to TIMSS: The Third International Mathematics and Science Study. TIMSS as a starting point to examine U.S. Education" (ORAD-97-1016). Washington, D.C.: Office of Educational Research and Improvement. [ED 410 123]
Peak, L. (1996). "Pursuing excellence: A study of U.S. eighth-grade mathematics and science teaching, learning, curriculum, and achievement in international context. Initial findings from the Third International Mathematics and Science Study" (NCES-97-198). Washington D.C.: National Center for Education Statistics, Office of Educational Research and Improvement. [ED 400 209] http://www.ed.gov/NCES/timss
U.S. National Research Center for TIMSS. A splintered vision: An investigation of U.S. science and mathematics education. Executive summary. Retrieved from the world wide web October, 1998. http://www.col-ed.org/smcnws/timss/splintrd.html