Skip to content

The limits of the scientific method and the need to merge science and innovation

After a visit to CIGI to discuss the value of think tanks I blogged about Roger Martin’s presentation and idea of the ‘logical leap of the mind’. He has published an article on the issue of the limits of the scientific method in economics and the world that is a must-read for anyone advocating for ‘evidence based policy’ or the adoption of impact evaluations and randomised control trials as the gold standards of decision making.

Martin beings by describing the confident exposition of an economist’s explanations of the current economic crisis. The same economist who had mistakenly predicted the crisis a few months prior. He argues that at the core of the problem with many of the current initiatives to make policymaking a technocratic affair is the inappropriate application of the Aristotelean scientific method:

The roots of the problem can be traced right back to Aristotle, the father of modern science, who around 400 B.C. laid down the first formal conception of cause and effect.

But … as much as Aristotle was a proponent of his scientific logic, in the best scientific tradition, he established boundary conditions for his theory.  It was for the part of the world in which things could not be other than they are. An oak tree is an oak tree and cannot be something else.  For this world, Aristotle laid out the seminal scientific method and argued that it was the optimal way for understanding that part of the world.

… he also cautioned that there is another part of the world that can be other than it is, and there was another method that needed to be used to understand it. The scientific method would be wholly inappropriate.

That part of the world consists of people – of relationships, of interactions, of exchanges.  In this part of the world, relationships can be good, bad or indifferent; close, distant or sporadic.  They change – they can be other than they currently are. For this part of the world, Aristotle said that the method used to develop our understanding and to shape this world is rhetoric; dialogue between parties that builds understanding that actually shapes and alters this part of the world.

At the CIGI meeting Roger Martin argued that to make a difference, think tanks must make sure that they understand this and pay sufficient attention to creating new ideas (these logical leaps of the mind) -and warned against the current practice of applying science to the vast track of the world where things can be other than they are:

Science advances our knowledge of the world in which things cannot be other than they are.  But the modern practice of applying science to the vast tract of the world where things can be other than they are is unhelpful, as demonstrated by the unreflective economist. Extrapolating the future to be a straight-line projection of the past is neither accurate, nor is it helpful in creating better understanding and newer ideas.

As much as it is helpful to the world to create, test and prove out novel new hypotheses about things that cannot be other than they are, I would argue it is more critical to the world to create novel new hypotheses for things that can be other than they are – like economic growth, environmental sustainability, and peace and security.

To do so, we have to break the iron grip of science on the part of our world that for which mere extrapolation of the past is ineffectual, for which the creation of a better future must be the goal.

The idea of the logical leap comes from Charles Sanders Peirce who:

… concluded that no new idea was ever derived from the analysis of the past using inductive and deductive logic – the two forms of logic our modern scientific method utilize.

However, deductive and inductive logical analyses aren’t so hot for things that can be other than they are – like economy for example.  These things change constantly due to the interactions of the people and organizations. The fall of 2008 wasn’t an extrapolation of the past – it was discontinuous with the past.

The obvious implication of this is that if think tanks are being Peircian they would be asking more fundamental questions about the persisting and new problems faced by the world. They would not be looking at hard data about what has happened to lead them forward, but rather be looking for alternative explanations.

In Chile, the think tanks that appeared after the Pinochet coup in 1973 began their work by looking at the causes of the breakdown of democracy in a attempt to understand what had happened. Their theories and what they knew of Chilean politics and society could not explain it. Had they now focused on this they would have not been able to adapt and find solutions to what was, for Chile, an unimaginable situation.

Rather than forgetting entirely that their theories were demonstrated to be totally lacking, and then going on to analyze some more and predict more based on those theories, they would have created a new hypothesis to explain what just happened.  This would be what Peirce evocatively called ‘a logical leap of the mind’ and ‘an inference to the best explanation.’

That is the merger of science and innovation.  It is what Peirce called ‘abductive logic’.  It is the formation of a new hypothesis.

And what do you do with an idea in the absence of data to prove it? You discuss it: you dialogue.

Martin does not argue that there is no place for deductive or inductive logic in the work of policymaking and think tanks:

Rather, I am arguing that we need to reign in faux science, which is appropriate only for understanding things that cannot be other than they are, using the tools of deductive and inductive logic.  And we need to release the energy of a broader conception of science and innovation that helps us to shape those aspects of our world that can be other than they are, using abductive logic.  It won’t always be clear in advance which is which, but it is important that we not default reflexively to analysis rather than innovation.

14 Comments Post a comment
  1. 1. Re ” the confident exposition of an economist’s explanations of the current economic crisis. The same economist who had mistakenly predicted the crisis a few months prior.” I am not sure I would use this event as evidence of the “limits of the scientific method”. In fact it is almost the opposite The economists explanation was a post hoc explanation, a historical analysis at best and a “just so story” at the worst. The scientific test of his/her “scientific knowledge” claims would be to get him/her to make predictions then wait an appropriate time to observe whether they occurred or not (Apparently the economist’s past predictions had not been very successful). Then we have the not inconsiderable task of working out how representative this economits predictions were of all economists predictions in a given setting of interest. A wider sample would be n order. And then we need to remember also that science is not a monolithic entity, its a creative social process involving debate with the majority of opinion on any matter often changing over time. So even within a wider sample the analytic methods used by a minority who made successfull predictions re 2009 might become more widely used in future. We dont look back on scientists of past centuries and say they were not really scientists because they had some errroneous views on how the world worked. Rather we tend to focus on their approach to analysis problems.

    2. Re the second part of the world -“That part of the world consists of people – of relationships, of interactions, of exchanges” – this made me think that perhaps Aristotle was the first complexity theorist. The problem of unpredictability arisses when there are dense networks of connections between entities, creating multiple feedback loops i.e. modern economies. In these settings making “straight line projections” is extremely risky. There have been plenty of analyses verifying the chaotic nature of stock market movements. Some of the more interesting approaches to analysing and dealing with economic crises involve network analysis based approaches, looking at how to decrease inter-dependencies between some of the major categories of actors and increasing the resilience of banks (via bigger reserve requirements) -forms of autonomy possible within networks

    And re “The fall of 2008 wasn’t an extrapolation of the past – it was discontinuous with the past” this seems illi-nformed. There have been dozens of major market crashes over the centuries, including some very big ones. The more recent ones are characterised by differences in degree rather than kind, arising from the more globally nature of the interconnections, and even more so, the high speed of those connections.

    Long live scientific inquiry, including in the field of economics!


    November 14, 2011
  2. As a researcher and now an educator with a background in science, I found this post really interesting. Thank you for sharing!


    November 14, 2011
  3. I think the first point is that there are limits. And those limits must be understood better. It may be able to explain what happens and predict what may happen but cannot tell us what to do. That choice is not one that is based on fact alone.

    Sure. Nowadays that complexity has become a new buzzword, I wonder how many actually have given it enough thought. There is also a sense of history that i lacking. The idea that we’ve never been here before. This obsession with the gold standard of science in social and economic policy is not new. We’ve been here before. Woodrow Wilson warned against the dictatorship of the ‘expert’, the technocrat and of science in politics. The history of think tanks is full of metaphors relating to application of science and medicine to the world of things that can be other than they are.

    And on the crisis. It would be terrible if we agreed as it would contradict what I was trying to say, but, yes, there have been economic crises before. But they do not all happen because of exactly the same reasons. (But I think you are actually agreeing with him). A crisis is usually a good time to rethink your theories of the world.


    November 14, 2011
  4. After reading between the lines “The Limits of the scientific method …” , I consider myself a more refined and knowledgeable researcher than before. This is because, Roger Martin’s arguments have shed some light on some of the hurdles I have often encountered during my research work, which include but of course by no means limited to:

    1.The relentless search or should I call it pursuit for some specific solution to an economic and social problem in such a manner that by all accounts attests to the fact that the’ hunter knows where the hunted is and which form it takes’, which reduces the essence of it all. Why on earth should one spend immense efforts on trying to find or discover something that was already known to him in the first place (crunching numbers in endless analyses to get the result that is predicted ex ante)
    2. Panacea models that work in all situations and conditions. Economists, have come to believe that a certain model is the best to resolve a certain problem, and where the solution does not come to light using such a model, then the problem lies not in the inappropriateness of the model but elsewhere (data inaccuracy, and other problems that relate to how the model is used to make the estimate but not the superfluousness of the model.)
    3. The belief that economic phenomena can be caught in their entirety by quantifiable variables, for which data can be collected; which is contrary to the economic discipline itself which is a social science that if anything, contains more social, unquantifiable variables than quantifiable ones. This has also meant that whenever an economic crisis occurs public attention goes to well established economists who have developed models that purportedly have the proved and tested capability to capture the phenomena, determinants thereof, making possible to suggest ways forward, Sociologists, anthropologists , even political economists are rarely engaged. This leaves with simple solutions to complex problems, which makes them short term and not sustainable, to the detriment of society as often panacea solutions in the end turn out to be turnkey solutions that work only if and only if premises and assumptions employed play out, rather than in all circumstances. ACADEMIC ARROGANCE is at its best here, with an astronomical cost to society.

    Using Roger Martin’s analysis, I can not but contend that, if it can apply to the 2008-2009 financial crisis, it should help us come up with innovative models that can explain what has led to the enormous debts Portugal, Greece, and Italy have accumulated at a time when EU provisions are in place to prevent that from happening. In any case, we do not have past examples on some members of a monetary union facing doldrums, while others are performing so well (Germany the largest economy is export oriented; The Netherlands, Sweden, Finland, are all doing well), while Ireland, Greece, Portugal (both relatively small , and Spain and now Italy (relatively big , hence qualifying to be called too big to fail ). Good performers do not have specific features that distinguish them from poor performers, unless one uses the primordial description of southern Europe which is not very appropriate because eastern EU economies despite being recent entrants into free market system in general do not as poor states of public finances as currently battered southern economies. Germany is one of the world’s leading exporters (largest EU economy) but has to some degree been able to steer its economy in the right direction; UK is not a member of the monetary union and was heavily clobbered by the 2008-2009 financial crisis, and spent staggering amounts in bailouts and recapitalization efforts of its financial services mainly banking sector. Despite that, her finances have not buckled under the weight . Netherlands, Sweden are also good performers but do not have exactly similar social policies, complicating the jigsaw puzzle. So are the poor performers. In light of that, scientists should have to come up with logical leaps of the mind to identify what has actually happened, instead of using models that were perhaps developed to handle regional economic integration rather than monetary union, hence will not help in discovering the core problems, let alone coming up with lasting solutions. Crunching numbers in this case, does not seem to generate the solution as extrapolation is not feasible , and even if forced, will create temporary ‘Eurekas’ for those involved in making the analysis but protracted problems for those , and this is the majority, who take such results as panaceas , now and in future. Innovation and science are likely to generate solutions, rather than blindly sticking to science amidst all abundance evidence that in fact as many non scientific variables are playing a role, and increasingly so, as scientific ones. Public policy has definitely played a part, and by implication, politics (political systems are therefore to blame as well); EU policies which are increasingly centered in Frankfurt (ECB), Strasbourg (EU parliament), and the EU commission (Brussels) are also to blame for imposing similar monetary policies on economies that have varying macroeconomic, microeconomic, even mesoeconomic fundamentals. The list goes on… What all this underscores is the fact that economies or economic phenomena are not pure science phenomena that must have pure , easily captured variables that can be analyzed to generate findings that are not inherent in the specification of those models. In other words, science models applied here do not generate anything new save what is already known, yet what we want is exactly insights and discovery of what happened, not flashbacks, that can be replicated in future because in many cases they are nonexistent.


    November 15, 2011

Trackbacks & Pingbacks

  1. Happy Holidays from onthinktanks! « on think tanks
  2. Not evidence but arguments: translating evidence into policy in Ecuador « on think tanks
  3. Have the social sciences failed us? Andrew Gamble says no « on think tanks
  4. A new political economy of research uptake in Africa: overview « on think tanks
  5. Paradox of Hoaxes: How Errors Persist, Even When Corrected « on think tanks
  6. Communication as Organisation: Implications for Policy Research « on think tanks
  7. Research uptake: what is it and can it be measured? | on think tanks
  8. Setting up a think tank: step by step | on think tanks
  9. Lies Your Optimization Guru Told You
  10. Do not dumb down you audience –or rather, how should think tanks affect policy chance? | on think tanks

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Get every new post delivered to your Inbox.

Join 7,076 other followers

%d bloggers like this: