Target outcomes of evidence-based decisions
We help founders, SMEs and non profits to make evidence-based decisions that, in turn, create strategies and plans that are both more robust and more ambitious.
EVIDENCE BASED MANAGEMENT
5/8/20249 min read
The discipline of evidence-based management — gathering, appraising, synthesising and acting on the best available evidence — is not free. It demands investment in data infrastructure, analytical capability, time, and the organisational development needed to embed new habits into how decisions are made. Like any other investment, it is reasonable to ask what the returns are. This article sets out a framework for thinking about what those returns look like, when they tend to appear, and how to measure them.
Evidence as an Investment
Most organisations recognise intuitively that decisions made on better information tend to produce better outcomes. What they find harder to articulate is the mechanism by which that improvement translates into financial and strategic returns, and over what timeframe those returns should be expected to materialise.
The starting point for any response to that dilemma is an acknowledgement of two things.
Costs are real: The costs of an evidence-based approach are real. There are direct costs of data, platforms, analytical tools, external research or expertise; the personnel costs of the time spent gathering, appraising, and synthesising evidence; and the opportunity costs of decisions deferred while better information is assembled. None of this should be neglected.
Benefits are genuinely difficult to isolate. When a decision goes well partly because it was informed by good evidence, the evidence rarely gets the credit. The decision gets the credit, or the manager who made it does. The counterfactual - what would have happened in the absence of good evidence - is generally ignored. We shouldn't think for a moment though that this only applies to evidence-based management: It also applies to investment in legal counsel, engineering or IT consultancy, or financial modelling from an internal finance team, or memberships to professional body, amongst countless examples. The difficulty of attribution does not mean the value is absent. It means it requires a deliberate framework.
The Short and the Long of It
Perhaps the most important framing principle for understanding the returns from evidence-based management is the distinction between short-term and long-term outcomes.
Short term: Broadly, within the first 6-12 months of embedding evidence-based practices, the returns that appear are often skewed towards being operational or cost-related. (Even if evidence seeds new revenue streams, they will likely not have matured to anywhere near their potential, and they might not even have been implemented yet.). By contrast, risks can be avoided quickly. Process efficiencies can be realised by addressing redundant or ineffective practices. Specific decisions that have previously been made on habit or assumption are revisited in the light of evidence, and some of them change. These gains tend to be modest in scale compared against the full potential. Moreover, they are generally attributable to specific initiatives or people, rather than to a broader cultural environment.
Longer-term: More substantial returns emerge over the longer term, as the evidence-based approach compounds. Each decision that is made with better evidence not only produces a better outcome in itself, but also generates organisational learning. If this learning is captured and tracked, it generates a richer, shared understanding of what works, what does not, and why - which improves the quality of the thousands of everyday intuitive decisions, as well as the ones that are formalised and given special attention in research or analysis projects. Eventually, the learning becomes "how things are done round here" - it learning accumulates into something that functions like an institutional asset. or "knowledge capital". The value is in a body of knowledge, and associated set of decision-making habits, that reduces the cost of uncertainty and increases the probability of good outcomes across the whole range of decisions the organisation faces. The origin of better decisions can be forgotten over time too, however.
This compounding dynamic of forms of knowledge capital is significant, and we can see it in a range of ways. For instance, Nucleus Research, analysing returns on analytics investments across a broad population of organisations, found an average return of $5.44 for every dollar invested, with top-performing organisations exceeding $13 per dollar. These returns are not from single projects though. They reflect the accumulated return from a capability that has been built up, used widely and consistently, and optimised over time.
Let's address the target outcomes:
Outcomes
Cost Outcomes
Cost-related benefuts tends to be earlier in the rollout of Evidence-based approaches, with measurable and tangible returns: cost savings are easier to count than revenue gains and tend to emerge quickly once inefficiencies are identified.
The most direct cost savings come from the elimination of activities that evidence demonstrates to be ineffective or redundant. Manual reporting processes that absorb significant analytical time without producing decisions of corresponding quality are a common example. Organisations that systematically evaluate how their own internal resources are being used regularly find that a significant proportion of reporting and analysis effort is directed at producing outputs that no one acts on. Another example is by finding marketing activities that are unprofitable and are also not contributing to other profitable marketing activities. Redirecting time and money is a cost saving that does not require any additional investment.
Another (potentially bigger) cost reduction is the avoidance of imminent projects, initiatives, and strategies that will not work. The Project Management Institute has estimated that, across a broad sample of US organisations, approximately 12% of total project expenditure — roughly $122 million for every $1 billion spent — is wasted due to poor decision-making. This figure does not capture the opportunity costs of initiatives that consumed resource and management attention without delivering results. Evidence-based management does not eliminate failure. It reduces its frequency and limits its scale.
Supply chain and operational decisions are a third area where evidence-based approaches consistently generate measurable cost returns. Organisations that use systematic analysis of operational data to identify inefficiencies in logistics, inventory, procurement, and production regularly achieve cost reductions that would not have been visible through intuitive management alone. However, stakeholder and expert input is required to interpret the raw data.
Revenue Outcomes
This is arguably the area where evidence-based management is often intuitively most exciting to a company, because the mechanism connecting better evidence to better revenue is relatively direct, and there is a particular form of satisfaction gained from achieving growth.
Organisations that understand their customers better through systematic analysis of purchasing behaviour, satisfaction data, and retention patterns consistently outperform those that rely on anecdote and assumption.
For instance, McKinsey's research across hundreds of organisations finds that intensive users of customer analytics are 23 times more likely to outperform competitors in new customer acquisition. The mechanism is not mysterious. When a decision about pricing, product development, channel mix, or customer segmentation is informed by what the data actually shows rather than what the leadership team believes to be true, the decision is more likely to be right.
More specifically, evidence-based approaches to commercial decision-making tend to generate revenue improvement through three channels. The first is identifying and capturing growth opportunities that are invisible to organisations relying on intuition — cross-sell and upsell opportunities, underserved customer segments, or markets where the organisation has an unrecognised advantage.
The second is reducing customer churn. Systematic analysis of the patterns that precede customers leaving consistently allows organisations to intervene earlier and more effectively than organisations that rely on relationship management alone. Churn analytics typically deliver disproportionately high returns because even a small reduction in churn compounds significantly in businesses with recurring revenue models.
A third lever for revenue growth is from marketing and commercial efficiency i.e. understanding which interventions actually drive conversion and which only appear to, and reallocating resource accordingly. Organisations implementing proper attribution of marketing performance typically see 20 to 40% improvements in marketing efficiency on existing budgets.
All of these require the same underlying disciplines: asking precise questions, gathering relevant evidence, appraising it critically, and updating assumptions when the evidence warrants it.
Risk Outcomes
This is the area where the returns from evidence-based management are most significant in scale but most difficult to measure, because the value of risk avoidance is inherently counterfactual i.e. it is the cost saving of avoiding things that did not happen. For this reason, it is arguably the area where evidence-based management is under-served.
Examples of direct risk returns come from regulatory compliance and fraud reduction. Organisations that systematically monitor relevant data for signals of non-compliance or fraudulent activity are able to identify and address problems before they escalate. Many well documented cases show that the cost of a significant compliance failure (e.g. fines, remediation, reputational damage, and management distraction) vastly exceeds the cost of the monitoring and analytical capability that would have prevented it.
Strategic risk is a more abstract and perhaps even subjective to define. However, it is also potentially the most consequential source of value as the benefits span time and typology. Many major strategic failures share a common feature: the decision-makers had access to evidence that should have led them to a different conclusion, but did not use it - either they ignored it completely or took a binary approach to a decision rather than one that reduced or diversified risk. They over-weighted their own experience and prior beliefs, engaged in group-think to reinforce flawed assumptions, and dismissed evidence that contradicted their existing view. The literature on cognitive bias in strategic decision-making is extensive. Evidence-based management does not make leaders immune to these errors, but it creates a process that makes them far less likely.
The risk of strategic inertia deserves particular to be mentioned as an important and perhaps separate sub-set to strategic risk. Many of the most costly organisational failures involve not a single bad decision but a sustained failure to update a course of action in the face of accumulating evidence that the current strategy is not working. A culture of evidence-based management — in which outcome assessment is a normal part of the decision cycle rather than an occasional exercise — makes this kind of failure substantially less likely if people commit to the proecss.
Valuing this process is nearly impossible without making contestable assumptions.
Organisational Learning Outcomes
Beyond the three more tangible categories of outcome above, there is a fourth and ultimately, the most important of all: the cumulative development of organisational knowledge and the quality of decision-making as a capability in itself.
Each time an organisation applies an evidence-based approach to a significant decision, it learns something. It generates data about its own performance that has the potential to enrich future decisions. However, this learning is not guaranteed with steps put in place to capture and interpret it for the future and make it sharable when needed again.
Over time, the result is what we call "informed intuition": a form of professional judgment that is faster and more confident than a formal evidence-based process, but, if interogated, would be grounded in a cumulative body of well-tested knowledge, having been critically evaluated at some point in its history.
Experienced practitioners who have consistently reflected on the outcomes of their decisions, updated their mental models when the evidence warranted it, and sought to distinguish what they genuinely know from what they merely believe, develop higher quality judgment
In some ways, this is the ultimate goal of embedding evidence-based management in an organisation. Not to replace intuition with process and bureaucracy - which would be neither possible nor desirable — but to ensure that the intuition that shapes decisions is, over time, increasingly well-calibrated on ever firmer foundations.
Characteristics of reaching this point would be (as well as a number of case studies) leaders who are conscious of the difference between decisions they are making on good evidence and decisions they are making under uncertainty, and who adjust the investment accordingly. There is an organisational culture in which questions like "what does the evidence actually show, and why don't we feel comfortable acting on it?" or "does this evidence not contradict this other evidence?" or "have we got enough evidence to make this investment at this scale?" are nornalised, and not a challenge to authority.
Measuring the Return
Measuring the return on evidence-based management is not straightforward, but not impossible. The most practical approach involves tracking a small number of leading and lagging indicators that together paint an adequate picture of whether the investment is producing value.
Leading indicators include the quality and breadth of evidence actually used in significant decisions, the proportion of major decisions that are followed by systematic outcome assessment, the frequency with which assumptions are challenged and updated when evidence warrants it, and the degree to which evidence from all four sources — scientific, organisational, experiential, and stakeholder — is routinely incorporated into the decision process. These are process measures rather than outcome measures, but they predict the quality of outcomes that will eventually follow. A semi-regular (e.g. annual) single organisational or management survey could capture proxies to many of these quite efficiently.
Lagging indicators include the financial metrics directly traceable to specific evidence-based decisions e.g. revenue or profit gains from initiatives informed by systematic analysis, costs avoided from projects not pursued on the basis of negative evidence, compliance incidents avoided or detected early. They also include the broader organisational performance indicators that reflect the cumulative quality of decision-making over time: growth relative to the sector, customer satisfaction trends, staff retention, and operational efficiency. These measures are not exclusively attributable to evidence-based management, but they are the measures that evidence-based management is, in the long run, trying to move.
These are sometimes possible to "game" - they are not hard and fast like a revenue or cost number. In particular, the factor of time makes attribution quite difficult. Several years of consistent practice are generally needed before the compounding effect of any cultural shift produces a clearly distinguishable performance advantage.
But its worth putting these issues into context with the costs of bad decisions: McKinsey and the Institute of Directors estimate that poor strategic decision-making costs a typical Fortune 500 company the equivalent of $250 million annually in wasted wages alone, before the opportunity costs of missed strategic moves and failed initiatives are even considered.
Its ultimately therefore a company-specific task to identify the business case for investing in evidence and evidence-based management approaches. This article lays out the lego-pieces for the case to be made.
This article is part of a series on evidence-based management drawing on Evidence-Based Management: The Basic Principles by Eric Barends, Denise M. Rousseau and Rob B. Briner (Center for Evidence-Based Management, 2014); research by McKinsey Global Institute, Nucleus Research, Forrester Consulting, and the Project Management Institute; and the academic literature on decision quality, cognitive bias, and organisational learning.