Counting the Opinions

People who might never have heard of George Santayana would still be familiar with the quote "Those who cannot remember the past are condemned to repeat it".

At the time of writing, Australia is going through a process which repeats mistakes from the past.

The 1936 US presidential election was between the incumbent Democrat Franklin D Roosevelt and Republican Alf Landon, Governor of Kansas. Roosevelt was partway through implementing his New Deal program, something which was seen by many conservatives as ushering in a form of socialism and anathema to those who saw libertarianism, small government and individual personal freedom as the foundations of US polity.

The magazine The Literary Digest mailed out about 10 million survey forms to subscribers and others asking them to predict who would win. 2.3 million replies were received suggesting that Landon would win 370 of the 531 Electoral College votes, and therefore the Presidency. Roosevelt received 500 Electoral College votes, and over 60% of the popular vote.

So how did The Literary Digest get the prediction so wrong?

One suggestion was that the subscriber base of the magazine consisted predominantly of conservative voters so they were the wrong people to ask. The real problem however was a combination of this and also that the respondents were a self-selected sample of people motivated enough to fill in a form, put it in an envelope and go to a post box to return it.

Another player in 1936 was advertising executive George Gallup who thought that there was a scientific way of predicting the results of the election. Using a sample of 50,000 people he said that Roosevelt would win.

I remember having hammered into me in the first courses I did in statistics at university the importance of sample selection, and how samples had to be both random and representative of the population under study. These might seem conflicting ideals but it is possible to randomly select subjects while still ensuring that the overall sample is representative. Large amounts of advertising expenditure and media company profits are based on television and radio ratings. These ratings are based not on counting everybody who watches a particular television program but on monitoring behaviour of a very carefully selected sample of a couple of thousand people across the country. Political polling can be very accurate with a small sample, but that sample is drawn from every electorate in the country. (The 1975 Australian federal election following the dismissal of the Whitlam government was one of the most polarised in history. The newspaper Nation Review had a competition to predict the relative percentages in two-party preferred voting for the Senate. The winner got the percentages correct to the third decimal place! He worked for the Gallup organisation and used the figures from an opinion poll one week before the election.)

Australia is now in the midst of a massive opinion poll to answer a single question – should the law be changed to allow same-sex couples to marry? (The result should be known by the time you're reading this.) Showing that we have learnt nothing from the lessons of 1936, the survey is being conducted as a postal vote. Although everybody on the electoral roll has been sent a survey form, participation is optional so only people really motivated by the question will feel the need to respond.

Motivation to express an opinion might not even be enough. I live in a small country town with approximately half of the residents of the local government area living on farms outside the town. There appears to be only one post box in the entire area for people to return their voting papers, and I've spoken to many people who only come into town about once a month to do shopping. If you don't post a lot of letters it's easy to forget that you need to take one with you the next time you go for groceries.

Three weeks after the first voting forms were posted out, there were three estimates of the response rate. Two surveys conducted by private pollsters (sample sizes about 1,000) suggested that about 75% of the forms had already been returned. The Australian Bureau of Statistics said in the same week that 57.5% had been returned. The ABS figure was itself an estimate based on counting envelopes in a few boxes and extrapolating to a warehouse full of boxes and if votes were coming in about 25% per week just a few days delay in postal delivery could account for the difference.

I've posted my vote, but I'll have to wait till November like everybody else to see the result (and of course the ABS analysis of the validity of the count). I hope we get value for the large amount of money the survey cost.

This article was published as the Naked Skeptic column in the September/October 2017 edition of Australasian Science

See more from Sketchplanations here