In the 16th century, bloodletting was a trusted treatment for many ailments. It was widely used by experienced professionals, despite the fact that it didn’t work and actually killed many patients, including George Washington.
Why wasn’t it obvious that this treatment was a failure, and why was it a go-to treatment for hundreds of years leading up to the 19th Century? Before we shake our heads in condescending hindsight, we should ask ourselves a more current question: Why do innovators working in the complex varied contexts of humanitarian aid and development continue to invest in Silicon Valley-inspired pilots?
The scenarios are far more alike than we’d like. One humanitarian innovation lab leader described the portfolio of pilots they had supported as a ‘cabinet of broken dreams’, while another international aid innovation program executive quipped that they ‘would have to get a bigger budget to build more shelves’ to hold all their failed pilots.
It’s not simply that innovation pilots have a horrendous record of success at scale. My work as an innovation advisor, coaching ‘promising’ pilot programs, has often revealed that pilots lead project teams in the wrong direction and that, frequently, the only path forward is to tear most of the work down and start over with a different mindset.
Can we save the practice of innovation pilots in aid? Or, is it time to admit that pilots, which a decade ago seemed like such a good idea, just aren’t working?
ANSWERING THE WRONG HARD QUESTIONS
The basic idea behind pilots as we know them seems sound. We should as quickly and cheaply as possible answer the hard question(s) that determine the success of our proposed innovation. In Silicon Valley, where the current pilot practice emerged around 2010, that big question was usually “Is there a market for this idea?”
In the aid sector, the pressing question is more often framed as “does this idea deliver value?” Pilots build out a lightweight version of the idea and then develop measurable evidence of its impact.
However, this is seldom the hardest or even most consequential question in a complex and messy aid context.
What the sector has found over a decade of investment is that even great pilots fail to work in practice, get adopted, or be sustainably operated. Complex challenges and unanticipated barriers mean that, despite many pilots’ early promise, there is often exactly zero impact.
And yet, most pilots are still intentionally designed to be simple and lightweight, avoiding hard questions like:
What is the sustainable business model?
How will the innovation integrate with other technologies and operations?
Who will actually be motivated and able to adopt the solution?
Who will remove key political, technical, or collaborative barriers?
How will key gaps in capabilities and resources be filled?
In a study that Hannah Reichardt and I did for DG ECHO, we identified over 50 complex challenges that technology innovators in the aid sector face, and that need to be addressed over the full lifecycle from ideation to eventual scaled operation.
Pilots are not in the business of asking and answering so many hard questions. So they often finish their time-boxed project without a sense of what their most difficult challenges are, whether they can be addressed, and how they can find a way forward.
To indulge in metaphors again, this is like setting out on an arduous journey through an unmapped mountainous jungle and proudly announcing that you’ve prepared by picking out a good pair of socks… Potentially useful, but woefully insufficient for the (scaling) task ahead.
POINTING CONFIDENTLY IN THE WRONG DIRECTION
Aid sector innovation programs have begun to recognize the size of this gap and stepped in to fill it. Labs increasingly offer extended funding to support innovators after their initial pilot project. They are also providing skilled coaches to help innovators fill in the missing parts of their solutions. This is a substantial investment, and if it addressed the one shortfall of pilots, it might well be seen as the natural development of the sector’s innovation practice.
Unfortunately, pilots have an even deeper defect. They encourage an immature rush to judgement, thinking that leads to naive views of complex challenges and small solutions that fail to make the best use of precious innovation investments.
As innovators working in a field where people’s lives and wellbeing hang in the balance, it is not enough to simply develop ‘a solution’ to ‘some problem’. As a matter of strategic policy, we should be looking to address the most important challenge with the best solution possible.
The typical pilot ideation and funding process does little to drive toward this higher standard.
Pilots are typically rooted in someone’s creative idea. The idea might emerge from an innovator’s personal experience, a hackathon event, or a call for proposals. There can be a lot of energy around this part of the work (so innovators like to do it), and the resulting neatly bound ideas are easy to fund (so donors like to support them).
This natural rush to ideation and the enabling pilot funding often sidestep a big-picture view of the challenge. As a result, innovators fail to see where the most impactful opportunities are or recognize the need for more sophisticated solutions. This harder messier version of a challenge might well not have a quick and neatly boxed-up pilot project as its solution. Hans Rosling in his fabulous book Factfullness relates the story:
““[During the Eboloa outbreak] we had hundreds of health-care workers from across the world flying in to take action, and software developers constantly coming up with new, pointless Ebola apps (apps were their hammers and they were desperate for Ebola to be their nail).””
Unfortunately, pilots, much like bloodletting, are not simply innocent practices. They actively encourage innovators to indulge in quick ideation and funders to support small ideas without a clear vision of where the biggest opportunities actually lie. We waste the precious opportunity to do powerful important things when our efforts and our funding are diverted into small poorly targeted work.
JUST STOP AND MOVE ON
For nearly a decade, aid sector innovators and the programs that support them have been asking the question “How do we make pilots deliver scale at impact?” And, there have been increasingly sophisticated approaches to the work.
But maybe it is time to just stop.
Innovation in the aid sector is too important to continue pursuing a structurally unsound practice that doesn’t fit the challenges the sector faces. It may seem radical, particularly given the amount of effort and culture change that has been invested in making pilots part of aid sector practice. Nonetheless, if pilots don’t work for complex, varied, and messy aid sector challenges, it will be better to look for better practices.
Increasingly, there are changes on the ground that move in this direction. Both innovators and innovation funders are shifting their approach, setting aside key elements of the Silicon Valley-inspired pilot model for a more systems-based approach:
A climate action organization is systematically mapping out the entire complex ecosystem of organizations, technology, policy and resources in the challenge areas where they plan to invest in innovative action.
The Response Innovation Lab (RIL) routinely steps back and develops a big-picture view of humanitarian challenges in the countries where they work.
Innovators working on aid sector goals like localization of purchasing or circular economy in humanitarian aid are starting with solutions that have multiple interconnected parts instead of an overly simplified piece of the solution.
In each case, complexity is recognized and embraced from the beginning. These innovators, and their funding sponsors, see this complexity as an opportunity to claim a bigger more impactful challenge and then develop a sufficiently ambitious solution.
However, these more thoughtful programs don’t throw out all the good lessons about nimble learning that were part of the pilot practice. They move with urgency and learn from action. They iteratively evolve their sophisticated solutions, actively engage stakeholders, and are willing to pivot and adapt along their journey.
The difference is that they have identified the hard parts up front, and therefore look to answer the hardest and most important questions early on. That might be, “What’s my long term funding strategy?” Or “How will we get this key organization to participate?”
Taking this bigger picture approach to the aid sector’s hard problems and leaning into sufficiently complex solutions will almost certainly result in fewer projects than were possible in the heyday of pilots. But, it also has the potential to set us on a path to far more initiatives that actually deliver impact at scale on problems that really matter.
about the Author
Dan McClure
Dan McClure has spent over three decades working on the challenge of disruptive and complex systems innovation. He has advised global commercial firms, public sector agencies, and international non-profits in support of their ambitious efforts to imagine and execute agile systems level innovation. He is an author of the book "Do Bigger Things - a practical guide to powerful innovation in a changing world.”