Infernal Ramblings
A Malaysian Perspective on Politics, Society and Economics

Benchmarking Leaders and Policy

Written by johnleemk on 11:45:37 am Feb 21, 2007.

An important part of public policy is setting benchmarks — and not just any plain old benchmarks. Bureaucrats love to measure their accomplishments in terms of input, rather than the output of what they put in. To accurately measure and judge the success of a policy, there have to be clear output targets. How many schools or teachers we have are statistics that mean nothing — especially when compared to meaningful statistics like our literacy rate or how many scientific breakthroughs can be attributed to our scientists.

There is a problem, however. Reasonable people are often forced to fall back on input statistics because it is impossible to obtain objective figures for output. Returning the example of education, how do we determine what constitutes a scientific breakthrough? How do we measure economic development? Obtaining statistics for these things is difficult, if not impossible.

As a result, people resort to input-based measures of accomplishment. For example, we can't measure how many scientific breakthroughs our country has made, but we can measure how many PhDs we have per thousand people (just to pick one example out of the infinite number possible; other possibilities including universities per capita or the proportion of university faculty with a terminal degree in their field). We can't possibly determine how well off a country is objectively, but we can determine how much its people spend in terms of currency — their gross domestic product (GDP) — and use it as a measure of development.

These input statistics, of course, have a lot of failings. They will never be completely accurate, because it is always possible to cook the books. If a flood strikes the nation and millions have to be spent cleaning up, the GDP will increase even though any reasonable observer would consider the nation less better off than the year before — despite the GDP increase. And what if a billionaire like Bill Gates decides to immigrate to the country and begins flaunting his wealth? The country's GDP would increase, but it would be difficult to say that this increase was an accurate and proportional representation of the increase in the country's overall economic well-being.

It may be possible to refine input figures, of course — and indeed, this is the route normally taken when people seek greater accuracy in benchmarks. Instead of simply measuring GDP, we calculate GDP per capita. To guarantee the relevancy of figures like the number of doctors per thousand people, we ensure there are minimum standards for practicing medicine.

The problem is that refining what are essentially innately flawed figures soon reaches the point of marginal return. There is only so much you can do with inaccurate representations of the truth. When it comes to measuring success, inputs are a necessary evil — with an emphasis on evil.

Unfortunately, people often tend to assume that the only statistics we can ever use are those for inputs. By overlooking the possibility that a relevant output number may exist, we cheat ourselves of the opportunity to assess ourselves where it really matters.

One reason output numbers are not popular probably lies in the fact that first you have to define what you want to achieve — what you want to benchmark. Do you want to see how many doctors we have per thousand people, or do you want to see how accessible medical treatment is? Do you want to measure how many schools we have for each town, or the qualification attained by the typical denizen of each town?

If you think things through, you can't hide behind flimsy figures meant to prop up a particular lie. You can't say that "Oh, we had one trillion dollars worth of trade last year, so our economy is good", because you will know that you are not looking to see how much you are trading, but how much this trade benefits the average citizen. Unless trade alone is your ultimate goal, the value of your trade is not as relevant a figure as how much the purchasing power of the average citizen has increased.

It is of course true that few situations are so clearcut. Usually, it is difficult to ascertain precise figures for output. But still, opportunities abound for a rough measurement.

To take just one example, the ex-Malaysian Prime Minister, Mahathir Mohamad, is usually assessed in terms of input. What did the man bring to the table as premier? A hell of a lot. His is one of the finest minds Malaysia has ever produced. His iron will meant he could get things done when he wanted them done. His guts meant he could stand up for what he believed in (as perhaps evinced by his criticism of the Jews at the height of the Asian financial crisis, despite widespread international condemnation). All inputs clearly combined to produce a brilliant leader, at least on paper.

But what of the outputs? I've written about them before, so I won't delve into them too deeply, but a brief summary: Mahathir was one of the more chauvinist MPs in his day, being expelled from his own racialist party for his extremism. When he became premier, he rendered the judiciary subservient to him by sacking the top judge and several of the Supreme Court judges. He locked up dozens of prominent politicians and activists without charging them with any crime. He amended the Constitution and several laws to prevent judicial interference with his policies. His economic policies themselves focused on centring the economy around the state, and fostered a dependence on petroleum. (At present, about half of the Malaysian budget is derived from petroleum-sourced earnings — something I will write about in the future.)

Mahathir's supporters usually point to one thing — Mahathir's placing Malaysia on the map. But this is more an illusion fostered by the mainstream Malaysian media (almost all of which is literally owned by the ruling regime). When I speak with well-educated Westerners, hardly anyone knows where Malaysia is until I mention that we're just north of Singapore. (Some of my friends still think I'm a Singaporean instead of a Malaysian.) Others occasionally cite Mahathir's "megaprojects", most of which either originated from the time of Tun Abdul Razak, who was Prime Minister in the early 1970s, or are nothing but colossal wastes of money (e.g. the Petronas Twin Towers, which contrary to what the local media would have you think, didn't put the country on the map).

Yet, a surprising number of people, including many intellectuals, have a very positive view of Mahathir. When discussing him, they focus on his positive attributes, almost all of which are inputs. The man's intellectual calibre is unquestionable, yes. His leadership is impeccable, true. But what did Mahathir produce with all these inputs? What was his output? Very few people pause to consider this, but once they do, it's difficult to argue that Mahathir was more than a lacklustre leader (at best). His potential was wasted on pursuing ill-thought out dreams, like establishing Malaysia as a regional automotive powerhouse (instead, Mahathir's brainchild, the Proton company, remains dependent on government subsidies and tariffs over 20 years after its founding).

Taking another example from Malaysia (which I do quite frequently, since it's the country I'm most familiar with), its New Economic Policy set clear targets and benchmarks by which it could be assessed. The policy is quite controversial because of its aggressive affirmative action measures, but few would disagree with its two basic goals: the eradication of the identification of race with economic function, and the reduction of Malaysian poverty, with special emphasis on the Malays (the largest ethnic group, but the poorest one as well). Throughout its lifetime, which extends till the present and beyond by de facto (leading some to label it as the Never Ending Policy), the goal of reducing Malay poverty has been the main emphasis — an unfortunate fact.

Whatever the case, the policy set a clear benchmark by which it could be assessed: the Malays were targeted to hold a 30% share of equity by 1990, 20 years after the NEP's implementation. An obvious defect is the complete ignorance of the "identification of race with economic function" goal, which is really the largest innate failing of the policy. However, that aside, the policy is remarkable for setting a clear goal, because when you have a clear output benchmark, it is difficult to argue that an injustice has been committed. Either the policy has failed, or it has succeeded.

The problem is that when the 30% goal was not reached at 1990, the time aspect was discarded. The government then began focusing on the 30% figure, without setting a new clear date by which this 30% was to be achieved. (Not too long ago the deputy premier suggested a timeframe on the scale of centuries.) Because now nobody can fairly and objectively evaluate the policy's success, discontent with the policy has grown significantly.

Recently, the government has shown signs of considering moving the goalposts around even more, with one prominent leader suggesting that the target percentage be increased to 70% if people questioned the policy further. Without an adequate benchmark by which the NEP can be assessed, it has completely collapsed. The existing perceived injustices of the policy are exacerbated by the fact that there is no longer even an objective definition of what constitutes justice.

A similar thing can be said for the much-touted goal of Malaysia becoming a developed country by 2020 — a pet goal of Mahathir's. Without an objective definition of what constitutes "developed", this output target becomes meaningless. The full ludicrosity of this becomes apparent when one considers that a couple of years back, Selangor, a Malaysian state, proclaimed itself to be "developed" on questionable statistical grounds.

There are three dimensions to any successful policy benchmark. The first is input versus output. Ideally, the target figure should be in terms of output, unless there is no objective way to measure it, in which case an input figure can be accepted. Then the target figure has to be objective — in other words, you can't use a subjective term like "developed", since it can be so easily abused. Finally, there has to be a timeframe so we can know when the target should be achieved.

Benchmarking our leaders, their policies, and their achievements is something people don't often do. We prefer to measure people and policies by how much they invest in something, rather than the returns they are getting. The world would be better off if man could only start to benchmark his leaders properly.

If you'd like to keep informed about updates to the site, consider subscribing to our web feed:

Infernal Ramblings is a Malaysian website focusing on current events and sociopolitical issues. Its articles run the gamut from economics to society to education.

Infernal Ramblings is run by John Lee. For more, see the About section. If you have any questions or comments, do drop him a line.

Najib's Orwellian 1Malaysia

Most Recently Read

  1. An Argument For Vernacular Schools?
  2. Apartheid and Protectionism, Internal Issues?
  3. Feminism and False Evidence of Discrimination
  4. The Importance of Educational Leadership
  5. Mamak Stalls - the Great Unifier?
  6. Saying No to Pigou?
  7. Malaysia, A Statist Economy
  8. Consequences of Schooling Autonomy
  9. Malaysia and Its Singaporean Inferiority Complex
  10. Libertarianism, Communism and Anarchism
Quoth the webserver...
Men who are unhappy, like men who sleep badly, are always proud of the fact.
— Bertrand Russell