Mathematics Benchmarking Report TIMSS 1999–Eighth Grade

Table of Contents

Executive Summary
Major Findings

 

 

 

Executive Summary

TIMSS 1999, a successor to the acclaimed 1995 Third International Mathematics and Science Study (TIMSS), focused on the mathematics and science achievement of eighth-grade students. Thirty-eight countries including the United States participated in TIMSS 1999 (also known as TIMSS-Repeat or TIMSS-R).(1) Even more significantly for the United States, however, TIMSS 1999 included a voluntary Benchmarking Study. Twenty-seven jurisdictions from all across the nation, including 13 states and 14 districts or consortia (see below), participated in the Benchmarking Study.

TIMSS 1999 Benchmarking Participants

States Districts and Consortia
Connecticut
Idaho
Illinois
Indiana
Maryland
Massachusetts
Michigan
Missouri
North Carolina
Oregon
Pennsylvania
South Carolina
Texas
Academy School District #20, Colorado Springs, CO
Chicago Public Schools, IL
Delaware Science Coalition, DE
First in the World Consortium, IL
Fremont/Lincoln/Westside Public Schools, NE
Guilford County, NC
Jersey City Public Schools, NJ
Miami-Dade County Public Schools, FL
Michigan Invitational Group, MI
Montgomery County, MD
Naperville School District #203, IL
Project SMART Consortium, OH
Rochester City School District, NY
Southwest Pennsylvania Math and Science Collaborative, PA

Each jurisdiction had its own reasonsfor taking part in the TIMSS 1999 Benchmarking Study. In general, participation provided an unprecedented opportunity for jurisdictions to assess the comparative international standing of their students’ achievement and to evaluate their mathematics and science programs in an international context. Participants were also able to compare their achievement with that of the United States as a whole,(2) and in the cases where they both participated, school districts could compare with the performance of their states.

Each participating entity invested valuable resources in this effort, primarily for data collection and team building, but also for staff development to facilitate use of the TIMSS 1999 results as an effective tool for school improvement. Despite each participant’s deep commitment to educational improvement by virtue of its participation in such a venture, it took courage and initiative to join such a high profile enterprise as the TIMSS 1999 Benchmarking Study. Whether students’ achievement fell at the top, middle, or bottom of the range of results for countries internationally, each participant will be asked to explain the results to its parents and communities.

This report provides a preliminary overview of the results for the Benchmarking Study in mathematics. The real work will take place as each participating entity begins to examine its curriculum, teaching force, instructional approaches, and school environment in an international context. As those working on school improvement know full well, there is no “silver bullet” or single factor that is the answer to higher achievement in mathematics or any other school subject. Making strides in raising student achievement requires tireless diligence, as policy makers, administrators, teachers, and communities work to make improvements in a number of important areas related to educational quality.

Unlike in many countries around the world where educational decision making is highly centralized, in the United States the opportunities to learn mathematics derive from an educational system that operates through states and districts, allocating opportunities through schools and then through classrooms. Improving students’ opportunities to learn requires examining every step of the educational system, including the curriculum, teacher quality, availability and appropriateness of resources, student motivation, instructional effectiveness, parental support, and school safety.

Particularly since A Nation at Risk (3) was issued eighteen years ago, many states and school districts have been working on the arduous task of improving education in their jurisdictions. During the past decade, content-driven systemic school reform has emerged as a promising model for school improvement.(4) That is, curriculum frameworks establishing what students should know and be able to do provide a coherent direction for improving the quality of instruction. Teacher preparation, instructional materials, and other aspects of the system are then aligned to reßect the content of the frameworks in an integrated way to reinforce and sustain high-quality teaching and learning in schools and classrooms.

There has been concerted effort across the nation in writing and revising academic standards that has very much included attention to mathematics. All states except Iowa (which as a matter of policy publishes no state standards) now have content or curriculum standards in mathematics, and many educational jurisdictions have worked successfully to improve their initial standards in clarity and content.(5) Forty-three states also have some type of criterion-referenced mathematics assessment aligned to state standards.(6) Much of this effort has been based on work done at the national level over the past decade to develop standards aimed at increasing the mathematics competencies of all students. Since 1989, when the National Council of Teachers of Mathematics (NCTM) published Curriculum and Education Standards for School Mathematics, the mathematics education community has had the benefit of a unified set of goals for mathematics teaching and learning. The nctm standards have been a springboard for state and local efforts to focus and improve mathematics education.(7)

Despite considerable energy devoted to educational improvement, achievement in mathematics has shown only modest gains since 1983.(8) The TIMSS results show little change in eighth-grade mathematics achievement between 1995 and 1999. In 1999, the U.S. eighth graders performed significantly above the TIMSS international average in mathematics, but about in the middle of the achievement distribution of the 38 participating countries (above 17 countries, similar to 6, and below 14). In TIMSS 1999, the world class performance levels in mathematics were set essentially by five Asian countries. Singapore, the Republic of Korea, Chinese Taipei, and Hong Kong SAR had the highest average performance, with Singapore and Korea having significantly higher achievement than all other participating countries. Japan, the fifth, also performed very well, as did Belgium (Flemish) (9) (see Exhibits 1.1 and 1.2 in Chapter 1).

Major Findings from the TIMSS 1999 Benchmarking Study

Average mathematics performance for the 13 Benchmarking states was clustered in the middle of the international distribution of results for the 38 countries. All of the Benchmarking states performed either significantly above or similar to the international average, yet significantly below the high-performing Asian countries.

The Benchmarking Study underscores the extreme importance of looking beyond the averages to the range of performance found across the nation. Performance across the participating school districts and consortia reßected nearly the full range of achievement internationally. Although achievement was not as high as Singapore, Korea, and Chinese Taipei, the top-performing Benchmarking jurisdictions of the Naperville School District and the First in the World Consortium (both in Illinois) performed similarly to Hong Kong, Japan, Belgium (Flemish), and the Netherlands. At the other end of the continuum, urban districts with high percentages of students from low-income families, such as the Chicago Public Schools, the Rochester City School District, and the Miami-Dade County Public Schools, performed more similarly to lower-performing countries such as Thailand, Macedonia, and Iran, respectively, but significantly higher than the lowest-scoring countries.

The TIMSS 1999 Benchmarking Study provides evidence that some schools in the U.S. are among the best in the world, but that a world-class education is not available to all children across the nation. The TIMSS index of home educational resources (based on books in the home, availability of study aids, and parents’ education level) shows that students with more home resources have higher mathematics achievement. Furthermore, the Benchmarking jurisdictions with the greatest percentages of students with high levels of home resources were among the top-performing jurisdictions, and those with the lowest achievement were four urban districts that also had the lowest percentages of students with high levels of home resources. These and other TIMSS 1999 Benchmarking results support research indicating that students in urban districts with a high proportion of low-income families and minorities often attend schools with fewer resources than in non-urban districts, including less experienced teachers, fewer appropriate instructional materials, more emphasis on lower-level content, less access to gifted and talented programs, higher absenteeism, more inadequate buildings, and more discipline problems.

It is good news that in mathematics at the eighth grade, the TIMSS 1999 Benchmarking Study shows relatively equivalent average achievement for girls and boys in each of the Benchmarking jurisdictions. This follows the national and international pattern where the United States was one of 34 countries in 1999 with girls and boys performing similarly.

Of the five mathematics content areas assessed by TIMSS, U.S. eighth graders performed higher than the international average in fractions and number sense; data representation, analysis, and probability; and algebra; but only at the international average in measurement and geometry. Despite the major differences among the Benchmarking participants geographically, economically, and culturally, most to some extent followed the national pattern. It will be important, however, for each participant to determine its specific relative strengths and weaknesses in mathematics achievement.

The Benchmarking results indicate that students’ relatively lower achievement in geometry is most likely related to less coverage of geometry topics in mathematics classrooms. Teachers also expressed the least confidence in their preparation to teach geometry.

The content area emphasis differed dramatically from jurisdiction to jurisdiction, however. For example, teachers in Naperville reported emphasizing algebra for nearly all their students (91 percent), and those in the Academy School District, the Michigan Invitational Group, and Montgomery County for about half. In contrast, about 70 percent of the students in Jersey City and Rochester received a combined emphasis on algebra, geometry, number, etc., and nearly half the students in Chicago had an emphasis mainly on number.

Research shows that higher achievement in mathematics is associated with teachers having a bachelor’s and/or master’s degree in mathematics.(10) According to their teachers, however, U.S. eighth-grade students were less likely than those in other countries to be taught mathematics by teachers with a major area of study in mathematics (41 percent in the U.S. compared with 71 percent internationally, on average). Among the Benchmarking jurisdictions, the percentages of students taught by teachers with mathematics as a major area of study varied dramatically from 70 to 73 percent in the First in the World Consortium, Naperville, and Rochester, to less than one-quarter in the Delaware Science Coalition and Jersey City.

In general, teachers in many Benchmarking entities and in the United States overall may be overconfident about their preparation to teach eighth-grade mathematics. More teachers in the Benchmarking jurisdictions and in the U.S. nationally reported feeling very well prepared to teach mathematics compared with their counterparts in other countries. In half the Benchmarking jurisdictions, 90 percent of the students had teachers who felt “very well prepared” to teach across a range of 12 general mathematics topics covered by TIMSS. Across the Benchmarking entities, the smallest percentage of students with teachers highly confident in their preparation to teach mathematics was 75 percent, which was higher than the international average of 63 percent. The comparable figure for the U.S. was 87 percent.

Since entering teachers make up a relatively small percentage of the teaching force, improving teacher quality depends on providing opportunities for professional development. Across the Benchmarking participants, there was considerable variation in the type of professional development that teachers engaged in. For example, only in the First in the World Consortium and Montgomery County did more than half the students have mathematics teachers who reported both observing and being observed by other teachers. In many of the Benchmarking entities, half or more of the students had teachers who reported that their professional development activities emphasized curriculum, but only about one-quarter had teachers who reported that their professional development activities emphasized content knowledge.

The choices teachers make determine, to a large extent, what students learn. In effective teaching, worthwhile mathematical problems are used to introduce important ideas and engage students’ thinking. The Benchmarking results show that higher achievement is related to the emphasis that teachers place on reasoning and problem-solving activities. This finding is consistent with the video study component of TIMSS conducted in 1995.(11) Analyses of videotapes of mathematics classes revealed that in the typical mathematics lesson in Japan students worked on developing solution procedures to report to the class that were often expected to be original constructions. In contrast, in the typical U.S. lesson students essentially practiced procedures that had been demonstrated by the teacher.

In TIMSS 1999, about half the Japanese students had teachers who reported a high degree of emphasis on reasoning activities in their mathematics classes, more than in any other country. The degree of emphasis on reasoning and problem-solving varied dramatically among Benchmarking participants. At the top end, between 41 and 46 percent of the students in Jersey City, the First in the World Consortium, and the Michigan Invitational Group had teachers who reported a high degree of emphasis on mathematics reasoning and problem-solving. Oregon and Chicago had the smallest percentages of students (eight and nine percent, respectively) with teachers reporting this degree of emphasis.

In general, the TIMSS 1999 data reveal that in most mathematics classes teachers do not focus on mathematics reasoning. Just as was found in the 1995 videotapes, it appears that usually the teacher states the problem, demonstrates the solution, and then asks the students to practice. Ninety-four percent of U.S. eighth graders reported that their teachers showed them how to do mathematics problems almost always or pretty often during mathematics lessons, and 86 percent reported working from worksheets or textbooks on their own this frequently. According to U.S. mathematics teachers, class time is spent as follows: 15 percent on homework review; 20 percent on lecture style teacher presentation; 35 percent on teacher-guided or independent student practice; 12 percent on re-teaching and clarification; 11 percent on tests and quizzes, six percent on administrative tasks; and four percent on other activities.

The TIMSS 1999 data indicate that the instructional time for learning mathematics, beyond being spent primarily on demonstrations of procedures and repeated practice, becomes further eroded by non-instructional tasks. In Japan and Korea, more than half the students were in classes that never had interruptions for announcements or administrative tasks. Among the Benchmarking participants, the results ranged from 22 percent of the eighth graders in such classes in Naperville to only five percent in Jersey City. Also, 74 percent of the U.S. students reported that they began their mathematics homework during class almost always or pretty often, well above the international average of 42 percent. In most Benchmarking jurisdictions, the results followed the national pattern, although the percentage varied from 43 to 90 percent.

The Benchmarking Study shows that students in schools that are well-resourced have higher mathematics achievement. Among the Benchmarking participants, three-fourths or more of the students in the Academy School District, the First in the World Consortium, and Naperville were in schools where the capacity to provide mathematics instruction was largely unaffected by shortages or inadequacies in instructional materials, supplies, buildings, space, computers and computer software, calculators, library materials and audio-visual resources. These high percentages exceeded those of all the TIMSS 1999 countries, with the highest percentages (about 50 percent) reported by Belgium (Flemish), Singapore, and the Czech Republic.

Discipline that maintains a safe and orderly atmosphere conducive to learning is very important to school quality, and research indicates that urban schools have conditions less conducive to learning than non-urban schools.12 For example, urban schools report more crime against students and teachers at school and that physical conßict among students is a serious or moderate problem. Among the Benchmarking participants there was considerable variation in principals’ reports about the seriousness of a variety of potential discipline problems. In several of the urban districts, however, 10 percent or more of the students were in schools where absenteeism, classroom disturbances, and physical injury to students were felt to be serious problems. Also in several of these districts, 20 percent or more of the students were in schools where intimidation or verbal abuse among students was a serious problem.

Among the 27 participants in the TIMSS 1999 Benchmarking Study, there was particularly extreme variation in mathematics achievement among the school districts and consortia, but less among the states. Several districts in relatively wealthy communities had comparatively high achievement in mathematics, while others in urban areas with high percentages of students from low-income families had relatively low achievement, compared with the TIMSS 1999 results internationally. Regardless of its performance, however, each state, district, and consortium now has a better idea of the challenges ahead and access to a rich array of data about various facets of its educational system. The TIMSS 1999 data provide an excellent basis for examining how best to move from developing a curriculum framework or standards in mathematics to meeting the extraordinary challenge of actually implementing the standards in schools and classrooms often characterized by considerable cultural, social, and experiential diversity.

 

next section >


Footnotes

1 IEA’s International Study Center at Boston College reported the international results for TIMSS 1999 as well as trends between 1995 and 1999 in two companion volumes – the TIMSS 1999 International Mathematics Report and the TIMSS 1999 International Science Report. Performance in the United States relative to that of other nations was reported by the U.S. National Center for Education Statistics in Pursuing Excellence: Comparisons of International Eighth-Grade Mathematics and Science Achievement from a U.S. Perspective, 1995 and 1999. (See the Introduction for full citations.)
2 For the most part, the U.S. TIMSS national sample was separate from the students assessed in each of the Benchmarking jurisdictions. Each Benchmarking participant had its own sample to provide comparisons to each of the TIMSS 1999 countries including the United States. Collectively, the Benchmarking participants are not representative of the United States even though the effort was substantial
in scope.
3 A Nation at Risk: The Imperative for Education Reform (1983), Washington, DC: National Commission on Excellence in Education.
4 O’Day, J.A. and Smith, M.S. (1993), “Systemic Reform and Educational Opportunity” in S.H. Fuhrman (ed.), Designing Coherent Education Policy: Improving the System, San Francisco, CA: Jossey-Bass, Inc.
5 Raimi, R.A. (2000), “The State of State Standards in Mathematics” in C.E. Finn and M.J. Petrilli (eds.), The State of State Standards, Washington, DC: Thomas B. Fordham Foundation; Glidden, H. (1999), Making Standards Matter 1999, Washington, DC: American Federation of Teachers.
6 Orlofsky, G.F. and Olson, L. (2001), “The State of the States” in Quality Counts 2001, A Better Balance: Standards, Tests, and the Tools to Succeed, Education Week, 20(17).
7 Kelly, D.L., Mullis, I.V.S., and Martin, M.O. (2000), Profiles of Student Achievement in Mathematics at the TIMSS International Benchmarks: U.S. Performance and Standards in an International Context, Chestnut Hill, MA: Boston College.
8 Campbell, J.R., Hombo, C.M., and Mazzeo, J. (2000), NAEP 1999 Trends in Academic Progress: Three Decades of Student Performance, NCES 2000-469, Washington, DC: National Center for Education Statistics.
9 Belgium has two separate educational systems, Flemish and French. The Flemish system participated in TIMSS 1999.
10 Goldhaber, D.D. and Brewer, D.J. (1997), “Evaluating the Effect of Teacher Degree Level on Educational Performance” in W. Fowler (ed.), Developments in School Finance, 1996, NCES 97-535, Washington DC: National Center for Education Statistics; Darling-Hammond, L. (2000), Teacher Quality and Student Achievement: A Review of State Policy Evidence, Education Policy Analysis Archives, 8(1).
11 Stigler, J.W., Gonzales, P., Kawanaka, T., Knoll S., and Serrano, A. (1999), The TIMSS Videotape Classroom Study: Methods and Findings from an Exploratory Research Project on Eighth-Grade Mathematics Instruction in Germany, Japan, and the United States, NCES 1999-074, Washington, DC: National Center for Education Statistics.12 Mayer, D.P., Mullens, J.E., and Moore, M.T. (2000), Monitoring School Quality: An Indicators Report, NCES 2001-030, Washington, DC: National Center for Education Statistics; Kaufman, P., Chen, X., Choy, S.P., Ruddy, S.A., Miller, A.K., Fleury, J.K., Chandler, K.A., Rand, M.R., Klaus, P., and Planty, M.G. (2000), Indicators of School Crime and Safety, 2000, NCES 2001-017/NCJ-184176, Washington, DC: U.S. Departments of Education and Justice.

Click here to return to the ISC homepage

TIMSS 1999 is a project of the International Study Center
Boston College, Lynch School of Education