Infernal Ramblings
A Malaysian Perspective on Politics, Society and Economics

Anecdotal Evidence is No Evidence At All

Written by johnleemk on 2:18:03 pm May 30, 2007.
Categories:

One common problem when discussing policy is people who try to bring in and rely on anecdotal evidence. As compelling, heartwrenching or simply interesting these stories may be, they do not make for good policy.

The implication made whenever ancedotal evidence is brought up is that these "real stories" are not outlying data points; that they are an accurate and typical representation of the population.

But as any statistician can tell you, a sample of a few dozen will not do for a population of twenty million, let alone a sample of two or three for a similarly sized population!

Moreover, the sample chosen is obviously biased towards data points which support the predetermined conclusion of whoever is trying to push them; there is no real objective attempt to look at an appropriate random sample of the population and see what conclusions can be drawn from them.

In debates between armchair policy wonks, this is not too big a problem. Earlier tonight, I was following an internet discussion group devoted to criticising Singapore's policy of paying its politicians million-dollar salaries.

One person criticised those who advocated a more liberal democratic form of government, bringing up examples of civil rights abuses in liberal democracies such as the United States and United Kingdom.

The person who gave the rebuttal merely focused on the inaccurate presentation of these abuses, and how there was no evidence of actual abuse found. But there really was no need.

How can you possibly generalise from a sample of one or two to a population of over 50 million (for the UK) or 300 million (for the US)? How can you justify this on any reasonable, sensible, logical grounds?

Are you really going to say because in one case, the police of some rural American county tried to frame an innocent man, the logical conclusion is that in liberal democracies, such abuses are just as likely to happen as they would in an authoritarian state like Singapore?

Fortunately, the damage armchair policy wonks can do is limited. But this sort of horrifying reasoning is not limited to just armchair policy wonks alone; the "leader of the free world", as US presidents were once billed, is just as susceptible.

For instance, during the 1990s, Paul Krugman criticised Bill Clinton for appearing to base his policies on a shrinking middle class, simply because there was anecdotal evidence of middle class layoffs. Krugman presented statistical evidence that it was the lower economic strata which had suffered, and not the middle class, which had remained stable.

Likewise, the catastrophe of the Iraq war may have been based on similar illogical thinking. The George W. Bush administration's main exposure to Iraqis was to people like Ahmad Chalabi, who were blatantly and openly pro-American.

Thus, generalising from such a small and biased sample to the whole Iraqi population, Vice-President Dick Cheney proclaimed that the invading forces would be welcomed as liberators. We all know how that turned out, don't we?

Anecdotal evidence is no evidence at all. What we need are cold, hard facts, not wishy-washy stories which sound compelling, but don't really tell us anything significant.


If you'd like to keep informed about updates to the site, consider subscribing to our web feed:

Infernal Ramblings is a Malaysian website focusing on current events and sociopolitical issues. Its articles run the gamut from economics to society to education.

Infernal Ramblings is run by John Lee. For more, see the About section. If you have any questions or comments, do drop him a line.


Latest:
Najib's Orwellian 1Malaysia

Most Recently Read

  1. Absolute vs Comparative Advantage
  2. Malaysia, A Statist Economy
  3. Ivy League, Here I Come
  4. Malaysia's Poor Leaders, Each PM Worse than the Last
  5. Fix Parliament, Then Fix the Rest
  6. Malaysia and Its Singaporean Inferiority Complex
  7. Extracurricular Activities Are Worthless in Malaysia
  8. Separating Head of State from Head of Government
  9. Taking A Stand Against Defamation and For Freedom of Speech
  10. Sepet, A Malaysian Movie
Quoth the webserver...
The misery of being exploited by capitalists is nothing compared to the misery of not being exploited at all.
— Joan Robinson