|
|
TIMSS
1999, a successor to the acclaimed 1995
Third International Mathematics and Science Study (TIMSS), focused
on the mathematics and science achievement of eighth-grade students.
Thirty-eight countries including the United States participated in TIMSS
1999 (also known as TIMSS-Repeat or TIMSS-R).(1)
Even more significantly for the United States, however, TIMSS 1999 included
a voluntary Benchmarking Study. Twenty-seven jurisdictions from all
across the nation, including 13 states and 14 districts or consortia
(see below), participated in the Benchmarking Study.
TIMSS 1999 Benchmarking Participants
States |
Districts and Consortia |
Connecticut
Idaho
Illinois
Indiana
Maryland
Massachusetts
Michigan
Missouri
North Carolina
Oregon
Pennsylvania
South Carolina
Texas
|
Academy School District #20,
Colorado Springs, CO
Chicago Public Schools, IL
Delaware Science Coalition, DE
First in the World Consortium, IL
Fremont/Lincoln/Westside Public Schools, NE
Guilford County, NC
Jersey City Public Schools, NJ
Miami-Dade County Public Schools, FL
Michigan Invitational Group, MI
Montgomery County, MD
Naperville School District #203, IL
Project SMART Consortium, OH
Rochester City School District, NY
Southwest Pennsylvania Math and Science Collaborative, PA |
|
Each jurisdiction had its own reasonsfor taking part in the TIMSS 1999
Benchmarking Study. In general, participation provided an unprecedented
opportunity for jurisdictions to assess the comparative international
standing of their students achievement and to evaluate their mathematics
and science programs in an international context. Participants
were also able to compare their achievement with that of the United
States as a whole,(2)
and in the cases where they both participated, school districts could
compare with the performance of their states.
Each participating entity invested valuable resources in this effort,
primarily for data collection and team building, but also for staff
development to facilitate use of the TIMSS 1999 results as an effective
tool for school improvement. Despite each participants deep commitment
to educational improvement by virtue of its participation in such a
venture, it took courage and initiative to join such a high profile enterprise
as the TIMSS 1999 Benchmarking Study. Whether students achievement
fell at the top, middle, or bottom of the range of results for countries
internationally, each participant will be asked to explain the results
to its parents and communities.
This report provides a preliminary overview of the results for the
Benchmarking Study in mathematics. The real work will take place as
each participating entity begins to examine its curriculum, teaching
force, instructional approaches, and school environment in an international
context. As those working on school improvement know full well, there
is no silver bullet or single factor that is the answer
to higher achievement in mathematics or any other school subject. Making
strides in raising student achievement requires tireless diligence,
as policy makers, administrators, teachers, and communities work to
make improvements in a number of important areas related to educational
quality.
Unlike in many countries around the world where educational decision
making is highly centralized, in the United States the opportunities
to learn mathematics derive from an educational system that operates
through states and districts, allocating opportunities through schools
and then through classrooms. Improving students opportunities
to learn requires examining every step of the educational system, including
the curriculum, teacher quality, availability and appropriateness of
resources, student motivation, instructional effectiveness, parental
support, and school safety.
Particularly since A Nation at Risk (3)
was issued eighteen years ago, many states and school districts have
been working on the arduous task of improving education in their jurisdictions.
During the past decade, content-driven systemic school reform has emerged
as a promising model for school improvement.(4)
That is, curriculum frameworks establishing what students should know
and be able to do provide a coherent direction for improving the quality
of instruction. Teacher preparation, instructional materials, and other
aspects of the system are then aligned to reßect the content of the
frameworks in an integrated way to reinforce and sustain high-quality
teaching and learning in schools and classrooms.
There has been
concerted effort across the nation in writing and revising academic
standards that has very much included attention to mathematics. All
states except Iowa (which as a matter of policy publishes no state standards)
now have content or curriculum standards in mathematics, and many educational
jurisdictions have worked successfully to improve their initial standards
in clarity and content.(5)
Forty-three states
also have some type of criterion-referenced mathematics assessment aligned
to state standards.(6)
Much of this effort has been based on work done at the national level
over the past decade to develop standards aimed at increasing the mathematics
competencies of all students. Since
1989, when the National Council of Teachers of Mathematics (NCTM) published
Curriculum and Education Standards for School Mathematics, the mathematics
education community has had the benefit of a unified set of goals for
mathematics teaching and learning. The nctm standards have been a springboard
for state and local efforts to focus and improve mathematics education.(7)
Despite considerable
energy devoted to educational improvement, achievement in mathematics
has shown only modest gains since 1983.(8)
The TIMSS results show little change in eighth-grade mathematics achievement
between 1995 and 1999. In 1999, the U.S. eighth graders performed significantly
above the TIMSS international average in mathematics, but about in the
middle of the achievement distribution of the 38 participating countries
(above 17 countries, similar to 6, and below 14). In TIMSS 1999, the
world class performance levels in mathematics were set essentially by
five Asian countries. Singapore,
the Republic of Korea, Chinese Taipei, and Hong Kong SAR had the highest
average performance, with Singapore and Korea having significantly higher
achievement than all other participating countries. Japan, the fifth,
also performed very well, as did Belgium (Flemish) (9)
(see Exhibits 1.1
and 1.2
in Chapter 1).
Major Findings from the TIMSS 1999 Benchmarking Study
|
Average mathematics performance for the 13 Benchmarking states
was clustered in the middle of the international distribution
of results for the 38 countries. All of the Benchmarking states
performed either significantly above or similar to the international
average, yet significantly below the high-performing Asian countries.
|
|
The Benchmarking Study underscores the extreme importance of
looking beyond the averages to the range of performance found
across the nation. Performance across the participating school
districts and consortia reßected nearly the full range of achievement
internationally. Although achievement was not as high as Singapore,
Korea, and Chinese Taipei, the top-performing Benchmarking jurisdictions
of the Naperville School District and the First in the World Consortium
(both in Illinois) performed similarly to Hong Kong, Japan, Belgium
(Flemish), and the Netherlands. At the other end of the continuum,
urban districts with high percentages of students from low-income
families, such as the Chicago Public Schools, the Rochester City
School District, and the Miami-Dade County Public Schools, performed
more similarly to lower-performing countries such as Thailand,
Macedonia, and Iran, respectively, but significantly higher than
the lowest-scoring countries.
|
|
The TIMSS 1999 Benchmarking Study provides evidence that some
schools in the U.S. are among the best in the world, but that
a world-class education is not available to all children across
the nation. The TIMSS index of home educational resources (based
on books in the home, availability of study aids, and parents
education level) shows that students with more home resources
have higher mathematics achievement. Furthermore, the Benchmarking
jurisdictions with the greatest percentages of students with high
levels of home resources were among the top-performing jurisdictions,
and those with the lowest achievement were four urban districts
that also had the lowest percentages of students with high levels
of home resources. These and other TIMSS 1999 Benchmarking results
support research indicating that students in urban districts with
a high proportion of low-income families and minorities often
attend schools with fewer resources than in non-urban districts,
including less experienced teachers, fewer appropriate instructional
materials, more emphasis on lower-level content, less access to
gifted and talented programs, higher absenteeism, more inadequate
buildings, and more discipline problems.
|
|
It is good news that in mathematics at the eighth grade, the
TIMSS 1999 Benchmarking Study shows relatively equivalent average
achievement for girls and boys in each of the Benchmarking jurisdictions.
This follows the national and international pattern where the
United States was one of 34 countries in 1999 with girls and boys
performing similarly.
|
|
Of the five mathematics content areas assessed by TIMSS, U.S.
eighth graders performed higher than the international average
in fractions and number sense; data representation, analysis,
and probability; and algebra; but only at the international average
in measurement and geometry. Despite the major differences among
the Benchmarking participants geographically, economically, and
culturally, most to some extent followed the national pattern.
It will be important, however, for each participant to determine
its specific relative strengths and weaknesses in mathematics
achievement.
|
|
The Benchmarking results indicate that students relatively
lower achievement in geometry is most likely related to less coverage
of geometry topics in mathematics classrooms. Teachers also expressed
the least confidence in their preparation to teach geometry.
|
|
The content area emphasis differed dramatically from jurisdiction
to jurisdiction, however. For example, teachers in Naperville
reported emphasizing algebra for nearly all their students (91
percent), and those in the Academy School District, the Michigan
Invitational Group, and Montgomery County for about half. In contrast,
about 70 percent of the students in Jersey City and Rochester
received a combined emphasis on algebra, geometry, number, etc.,
and nearly half the students in Chicago had an emphasis mainly
on number.
|
|
Research
shows that higher achievement in mathematics is associated with
teachers having a bachelors and/or masters degree
in mathematics.(10)
According to their teachers, however, U.S. eighth-grade students
were less likely than those in other countries to be taught mathematics
by teachers with a major area of study in mathematics (41 percent
in the U.S. compared with 71 percent internationally, on average).
Among the Benchmarking jurisdictions, the percentages of students
taught by teachers with mathematics as a major area of study varied
dramatically from 70 to 73 percent in the First in the World Consortium,
Naperville, and Rochester, to less than one-quarter in the Delaware
Science Coalition and Jersey City.
|
|
In general, teachers in many Benchmarking entities and in the
United States overall may be overconfident about their preparation
to teach eighth-grade mathematics. More teachers in the Benchmarking
jurisdictions and in the U.S. nationally reported feeling very
well prepared to teach mathematics compared with their counterparts
in other countries. In half the Benchmarking jurisdictions, 90
percent of the students had teachers who felt very well
prepared to teach across a range of 12 general mathematics
topics covered by TIMSS. Across the Benchmarking entities, the
smallest percentage of students with teachers highly confident
in their preparation to teach mathematics was 75 percent, which
was higher than the international average of 63 percent. The comparable
figure for the U.S. was 87 percent.
|
|
Since entering teachers make up a relatively small percentage
of the teaching force, improving teacher quality depends on providing
opportunities for professional development. Across the Benchmarking
participants, there was considerable variation in the type of
professional development that teachers engaged in. For example,
only in the First in the World Consortium and Montgomery County
did more than half the students have mathematics teachers who
reported both observing and being observed by other teachers.
In many of the Benchmarking entities, half or more of the students
had teachers who reported that their professional development
activities emphasized curriculum, but only about one-quarter had
teachers who reported that their professional development activities
emphasized content knowledge.
|
|
The choices teachers make determine, to a large extent, what
students learn. In effective teaching, worthwhile mathematical
problems are used to introduce important ideas and engage students
thinking. The Benchmarking results show that higher achievement
is related to the emphasis that teachers place on reasoning and
problem-solving activities. This
finding is consistent with the video study component of TIMSS
conducted in 1995.(11)
Analyses of videotapes of mathematics classes revealed that in
the typical mathematics lesson in Japan students worked on developing
solution procedures to report to the class that were often expected
to be original constructions. In contrast, in the typical U.S.
lesson students essentially practiced procedures that had been
demonstrated by the teacher.
|
|
In TIMSS 1999, about half the Japanese students had teachers
who reported a high degree of emphasis on reasoning activities
in their mathematics classes, more than in any other country.
The degree of emphasis on reasoning and problem-solving varied
dramatically among Benchmarking participants. At the top end,
between 41 and 46 percent of the students in Jersey City, the
First in the World Consortium, and the Michigan Invitational Group
had teachers who reported a high degree of emphasis on mathematics
reasoning and problem-solving. Oregon and Chicago had the smallest
percentages of students (eight and nine percent, respectively)
with teachers reporting this degree of emphasis.
|
|
In general, the TIMSS 1999 data reveal that in most mathematics
classes teachers do not focus on mathematics reasoning. Just as
was found in the 1995 videotapes, it appears that usually the
teacher states the problem, demonstrates the solution, and then
asks the students to practice. Ninety-four percent of U.S. eighth
graders reported that their teachers showed them how to do mathematics
problems almost always or pretty often during mathematics lessons,
and 86 percent reported working from worksheets or textbooks on
their own this frequently. According to U.S. mathematics teachers,
class time is spent as follows: 15 percent on homework review;
20 percent on lecture style teacher presentation; 35 percent on
teacher-guided or independent student practice; 12 percent on
re-teaching and clarification; 11 percent on tests and quizzes,
six percent on administrative tasks; and four percent on other
activities.
|
|
The TIMSS 1999 data indicate that the instructional time for
learning mathematics, beyond being spent primarily on demonstrations
of procedures and repeated practice, becomes further eroded by
non-instructional tasks. In Japan and Korea, more than half the
students were in classes that never had interruptions for announcements
or administrative tasks. Among the Benchmarking participants,
the results ranged from 22 percent of the eighth graders in such
classes in Naperville to only five percent in Jersey City. Also,
74 percent of the U.S. students reported that they began their
mathematics homework during class almost always or pretty often,
well above the international average of 42 percent. In most Benchmarking
jurisdictions, the results followed the national pattern, although
the percentage varied from 43 to 90 percent.
|
|
The Benchmarking Study shows that students in schools that are
well-resourced have higher mathematics achievement. Among the
Benchmarking participants, three-fourths or more of the students
in the Academy School District, the First in the World Consortium,
and Naperville were in schools where the capacity to provide mathematics
instruction was largely unaffected by shortages or inadequacies
in instructional materials, supplies, buildings, space, computers
and computer software, calculators, library materials and audio-visual
resources. These high percentages exceeded those of all the TIMSS
1999 countries, with the highest percentages (about 50 percent)
reported by Belgium (Flemish), Singapore, and the Czech Republic.
|
|
Discipline that maintains a safe and orderly atmosphere conducive
to learning is very important to school quality, and research
indicates that urban schools have conditions less conducive to
learning than non-urban schools.12 For example, urban schools
report more crime against students and teachers at school and
that physical conßict among students is a serious or moderate
problem. Among the Benchmarking participants there was considerable
variation in principals reports about the seriousness of
a variety of potential discipline problems. In several of the
urban districts, however, 10 percent or more of the students were
in schools where absenteeism, classroom disturbances, and physical
injury to students were felt to be serious problems. Also in several
of these districts, 20 percent or more of the students were in
schools where intimidation or verbal abuse among students was
a serious problem.
|
Among the 27 participants in the TIMSS 1999 Benchmarking Study, there
was particularly extreme variation in mathematics achievement among
the school districts and consortia, but less among the states. Several
districts in relatively wealthy communities had comparatively high achievement
in mathematics, while others in urban areas with high percentages of
students from low-income families had relatively low achievement, compared
with the TIMSS 1999 results internationally. Regardless of its performance,
however, each state, district, and consortium now has a better idea
of the challenges ahead and access to a rich array of data about various
facets of its educational system. The TIMSS 1999 data provide an excellent
basis for examining how best to move from developing a curriculum framework
or standards in mathematics to meeting the extraordinary challenge of
actually implementing the standards in schools and classrooms often
characterized by considerable cultural, social, and experiential diversity.
Footnotes
1 |
IEAs International Study Center
at Boston College reported the international results for TIMSS 1999
as well as trends between 1995 and 1999 in two companion volumes
the TIMSS 1999 International Mathematics Report and the TIMSS
1999 International Science Report. Performance in the United States
relative to that of other nations was reported by the U.S. National
Center for Education Statistics in Pursuing Excellence: Comparisons
of International Eighth-Grade Mathematics and Science Achievement
from a U.S. Perspective, 1995 and 1999. (See the Introduction for
full citations.) |
2 |
For the most part, the U.S. TIMSS national
sample was separate from the students assessed in each of the Benchmarking
jurisdictions. Each Benchmarking participant had its own sample
to provide comparisons to each of the TIMSS 1999 countries including
the United States. Collectively, the Benchmarking participants are
not representative of the United States even though the effort was
substantial
in scope. |
3 |
A Nation at Risk: The Imperative for
Education Reform (1983), Washington, DC: National Commission on
Excellence in Education. |
4 |
ODay, J.A. and Smith, M.S. (1993),
Systemic Reform and Educational Opportunity in S.H.
Fuhrman (ed.), Designing Coherent Education Policy: Improving the
System, San Francisco, CA: Jossey-Bass, Inc. |
5 |
Raimi, R.A. (2000), The State
of State Standards in Mathematics in C.E. Finn and M.J. Petrilli
(eds.), The State of State Standards, Washington, DC: Thomas B.
Fordham Foundation; Glidden, H. (1999), Making Standards Matter
1999, Washington, DC: American Federation of Teachers. |
6 |
Orlofsky, G.F. and Olson, L. (2001),
The State of the States in Quality Counts 2001, A Better
Balance: Standards, Tests, and the Tools to Succeed, Education Week,
20(17). |
7 |
Kelly, D.L., Mullis, I.V.S., and Martin,
M.O. (2000), Profiles of Student Achievement in Mathematics at the
TIMSS International Benchmarks: U.S. Performance and Standards in
an International Context, Chestnut Hill, MA: Boston College. |
8 |
Campbell, J.R., Hombo, C.M., and Mazzeo,
J. (2000), NAEP 1999 Trends in Academic Progress: Three Decades
of Student Performance, NCES 2000-469, Washington, DC: National
Center for Education Statistics. |
9 |
Belgium has two separate educational
systems, Flemish and French. The Flemish system participated in
TIMSS 1999. |
10 |
Goldhaber, D.D. and Brewer, D.J. (1997),
Evaluating the Effect of Teacher Degree Level on Educational
Performance in W. Fowler (ed.), Developments in School Finance,
1996, NCES 97-535, Washington DC: National Center for Education
Statistics; Darling-Hammond, L. (2000), Teacher Quality and Student
Achievement: A Review of State Policy Evidence, Education Policy
Analysis Archives, 8(1). |
11 |
Stigler, J.W., Gonzales, P., Kawanaka,
T., Knoll S., and Serrano, A. (1999), The TIMSS Videotape Classroom
Study: Methods and Findings from an Exploratory Research Project
on Eighth-Grade Mathematics Instruction in Germany, Japan, and the
United States, NCES 1999-074, Washington, DC: National Center for
Education Statistics.12 Mayer, D.P., Mullens, J.E., and Moore, M.T.
(2000), Monitoring School Quality: An Indicators Report, NCES 2001-030,
Washington, DC: National Center for Education Statistics; Kaufman,
P., Chen, X., Choy, S.P., Ruddy, S.A., Miller, A.K., Fleury, J.K.,
Chandler, K.A., Rand, M.R., Klaus, P., and Planty, M.G. (2000),
Indicators of School Crime and Safety, 2000, NCES 2001-017/NCJ-184176,
Washington, DC: U.S. Departments of Education and Justice. |
|