Learning from comparative studies

By Mochtar Buchori

JAKARTA (JP): International comparative studies about educational performance are always interesting, but very hard to do. The reason is that performance of educational systems is so much influenced by cultural factors that to make a meaningful comparison you have to take those factors into account. This usually makes the study very complex and very hard to manage. If you want to reduce the complexity of your study, you have to define variables in a universal format that is culturally comparable. Such variables are very hard to identify and define.

Despite such difficulties, researchers have persisted in their efforts, and eventually successful international studies are conducted, with their results widely disseminated and discussed. According to The Economist (March 29, 1997), the latest of these studies was the Third International Math and Science Studies (TIMSS), in which 41 countries took part. All 13-year-olds in the 41 countries were tested in math and science, using a uniform test, and the national averages were then compared. These national averages were then ranked separately for math and science, and the position of each participating country in these two fields was then be determined from the ranked lists.

Why math and science? I think there are two reasons for this. First, math and science are relatively culture-free. Wherever you live in the world, studying math and science is learning about the same concepts, the same phenomena, the same methods of inquiry, and the same body of theoretical knowledge. Second, math and science are now recognized as the foundations of modern technology and considered indispensable for modern life. No one can in these modern times afford to remain ignorant about math and science. It was for these two reasons, I think, that international comparative studies about math and science have been conducted.

What are the main results of this study? In mathematics, the highest rank achieved was by Singapore (national average score 643). The lowest rank was occupied by South Africa, with a national average score of 354. In science, the highest rank was again Singapore (national average score 607), and the lowest rank was again occupied by South Africa (national average score 326). The international average score for mathematics was 500, and this position was occupied by the United States. The international average score for science was 517, and Spain occupied this position.

Among one of the more surprising results was that countries with a longer history of formal education performed worse in this study than countries with younger systems of formal education. Countries like Sweden, Germany, England, Denmark, the United States, and Scotland ranked lower in both mathematics and science than Japan, South Korea, Singapore, Thailand, and Hong Kong, countries whose systems of formal education are known to be younger.

Another surprising fact is that countries with a higher expenditure per pupil, per annum, also ranked lower than countries which spent less on their students. Denmark, the United States, and Germany spent around US$6,000 per pupil annually, yet they did much worse in this study than the Czech Republic, South Korea, and Hungary, each of which spent less than $2,000 per pupil, per year.

I think it is important for us to ponder carefully the meaning of these results. It is important that we do not draw unwarranted conclusion from these findings. It is clear from these results that countries which ranked high on these lists -- i.e., Singapore, South Korea, Japan, the Czech Republic -- have a better system of teaching math and science at the elementary level than lower-ranked countries, such as South Africa, Colombia, Kuwait, and Iran.

It does not necessarily mean, however, that in any other field outside math and science -- language, history, character building, sports, and the arts, for instance -- countries which were high on the list are also much better than those countries which were on the bottom. Thus, it does not necessarily mean that elementary education in Singapore or South Korea is much better than that of South Africa or Colombia. Maybe they are, but maybe they are not. We just don't know. There is not enough data in the report to make that conclusion.

This study also shows clearly that there is something to be learned from Singapore, South Korea, Japan, and the Czech Republic, in the teaching of math and science at the elementary school level. How does our elementary school system compare with Singapore, South Korea, Japan, and the Czech Republic in math and science? Again the answer is we do not know. We can claim that we are better, but claims which are not supported by data are not of much use. The issue here is not national pride, but national performance. I personally think that if those four countries at the top of the list show better results than countries with older systems of formal education, then they must have something which is worth learning.

Can we conclude from this study that the teaching of math and science is the most important part of school education? I would not say so. Rather, I would say that in these modern times, knowledge of math and science is indispensable, but there are also other equally important areas of teaching as well. I mentioned in a recent article of mine that the teaching of emotionality is an important agenda, which has so far been very neglected in schools throughout the world.

In addition, I would also say that the teaching of languages is another very important area -- equally important to the teaching of math and science. The fact that this international comparative study was conducted in the field of math and science should not be interpreted that other areas of teaching are not important.

I think it would be very ideal for us if we could assess, in a realistic manner, how we compare with other countries in our systems of elementary and secondary education, and decide what we can learn from other countries to improve these two systems. It is important to note, in this regard, that the 41 countries participating in this international study did.

The question is, do we think it is important to have a realistic picture of our education, or would we rather live with a narcissistic image of ourselves?

The writer is an observer of social and cultural issues.

Window: The question is, do we think it is important to have a realistic picture of our education, or would we rather live with a narcissistic image of ourselves?