Pros, cons of opinion poll and integrity of pollsters

Muhammad Qodari, Jakarta

On Aug. 12, The Jakarta Post published an article entitled Analysts question pollsters' influence ahead of runoff, based on an interview with Eka Ginting, an expert on information technology.

Ginting questioned the results of polls conducted by the International Foundation for Electoral Systems (IFES) and the Indonesian Survey Institute (LSI) prior to the April 5 legislative election and the July 5 presidential election, since he said they proved to be vastly different from the election's official results.

He also said, "From the perspective of scientific methodology, the result of the IFES' and LSI's opinion polls are not accountable because the sampling of population was not proportionate nor representative and the margin of error was very high".

Ginting's statements need to be responded to and discussed not only because he questioned the credibility of the LSI and other pollsters but also because opinion polling has become a significant feature of the 2004 elections.

Ginting is not correct when he says the poll conducted by the LSI prior to the April 5 legislative election proved to be vastly different from the election's actual result. Table 1 shows the comparison between the results of LSI's survey conducted between March 18 and 24 and published on April 2, with the official results of from the Election Commission (KPU).

From the comparison, we know the LSI was correct in predicting Golkar would become the winner and the Indonesian Democratic Party of Struggle (PDI-P) would slide down to the No. 2 spot. The LSI was also correct in predicting the rise of the Prosperous Justice Party (PKS) and the Democratic Party. Except for the United Development Party (PPP) and Democratic Party tallies, the difference between the LSI's projected votes and the KPU's actual totals remains within the margin of error of about two percent. In fact, the LSI poll deviated the least of all the pre-election day polls from the final result.

Regarding the July 5 presidential election, the LSI conducted a poll published on July 2 from a survey carried out between June 20 and 24. That poll correctly predicted Susilo Bambang Yudhoyono and Megawati would win the election and go on to contest the runoff in September (see Table 2). It is interesting to note that although almost all polls prior to election day predicted Susilo as the winner, they differed in predicting who would come second. Some even predicted Amien Rais would be the runner-up. Amien, we may remember, finished fourth.

The comparison between the LSI's predicted vote and the election's actual result, however, showed Susilo won fewer votes while Megawati and Wiranto got more support than expected.

How can one explain the difference between LSI's poll and the actual election result? There are at least three possible explanations: Methodological error or weaknesses; the manipulation of data and its processing by the pollster; and the dynamics of public opinion.

LSI polls, including the ones published prior to legislative and presidential election, always sample a representative group and are consistently proportionate and representative to the population in terms of important characteristics such as gender, the rural-urban split, religion, ethnicity, education, income, and age group.

It has always been the LSI's main aim to maintain the objectivity of its data collection and processing. One of the measures taken by the LSI to maintain the objectivity of its polls is to separate the stages of research. Field work is conducted by interviewers in the regions headed by area coordinators. At least three independent analysts take part in the central data collection and analysis.

If the sampling methodology is sound and the objectivity of research can be guaranteed, the only explanation for the discrepancy between the poll and the actual result is the dynamics of public opinion.

We must remember that the data from polling is a portrait or snapshot of public opinion at the time of data collection. The time lag between data collection and the election day can be up to two weeks. The many things that happen in between are likely to change voters' preferences.

The reason for the lower accuracy of the LSI's presidential poll compared to its legislative poll is perhaps due to voters' weaker attachment to the candidates than to the political parties. Susilo's popularity, for example, went up and down quickly. Prior to the LSI poll on March 2004, opinion polls showed Susilo was a less popular candidate than Megawati or even Amien Rais. But when the LSI conducted its next poll in mid- March, his popularity had overtaken Megawati's, with around 20 percent support. His star continued to rise, reaching its peak at the end of May with almost 50 percent support, only to trail off gradually from the third week of June until July 5.

The dynamic of public opinion in the presidential election is influenced by several possible factors. Susilo's popularity decreased possibly due to his weaker campaign machine and the limited political networks of his Democratic Party.

Meanwhile, support for Megawati increased because her political machine began working harder; the PDI-P team was spooked after some polls predicted Megawati would not make the second round, her TV advertisements improved; by focussing on the economic improvement of the past three years, and her improving personal image; Megawati is now more communicative than she used to be.

The LSI accepts criticism about its polls in order to improve their quality. It also welcomed closer monitoring by the KPU on the publication of election polls. KPU deputy chairman Ramlan Surbakti says the commission will in future oblige pollsters to include information about who funds their poll, their sampling methods, the time of polling, and the margin of error.

The LSI welcomes this obligation because this kind of information will allow the public to better understand the strengths and weaknesses of each poll.

Table 1. Legislative Election, April 5 Political PartiesLSI Poll (%)KPU's Actual Result (%) Golkar23.221.6 PDI-P17.518.5 PKB9.510.6 PAN7.46.4 PPP5.78.2 PKS5.57.3 Demokrat4.97.5

Table 2. Presidential Election, July 5 Presidential TeamLSI Poll (%)KPU's Actual Result (%) Susilo B. Yudhoyono-Jusuf Kalla43.533.6 Megawati S.-Hasyim Muzadi20.326.6 Wiranto-Solahuddin Wahid14.922.2 Amien Rais-Siswono Yudohusodo13.614.7 Hamzah Haz-Agum Gumelar 2.3 3.0

The writer is the director of research at the Indonesian Survey Institute (LSI), Jakarta