Innovation Metrics - Why do they matter and why are they so hard?
Last week I co-hosted a breakfast with Ian Forshew, founder of The OMTM. We brought together senior innovation leaders from a broad range of sectors (including FMCG, pharma, media, oil & gas, law, engineering, tech) to discuss innovation metrics. Why are metrics important? What have people tried that's worked or not? What are the biggest challenges surrounding metrics?
Take-outs from the breakfast
At last week's breakfast, it was great to have such a diverse group of people in the room so we could see where our thinking and experience aligned and where it didn't. And we could share learnings. These are the key points that I took away from the session…
1. Metrics are a mechanism for learning
Metrics provide feedback on what's really going on and enable a team or organisation to measure the current effectiveness of its innovation efforts. The important thing is what's then done with this feedback - we need to use this data and these learnings to optimise our innovation activity and ensure that innovation investment delivers maximum value to the business.
By gathering metrics over time, teams may also get better at doing predictive modelling and be able to use this analysis to inform decision making and optimise investment further. This is in the future though - most businesses are a long way from this level of sophistication at the moment.
2. Metrics are needed to access funding
Metrics are important when it comes to decision making as they enable people to make data-driven decisions. This is especially the case when it comes to accessing additional investment from the business - for example for scaling up a prototype. One of the challenges comes when innovation projects are competing against core business projects for the same pot of funds / resource. There are bigger strategic questions around whether this should be the case but I’m going to park those for now as the reality is that this is often the case. Core business projects are likely to be backed up by a business case underpinned by years of operational data. How do we enable decision makers to benchmark a riskier assumption-based innovation project against the safer bet of a core business project? What benchmarks can we put in place to enable a better comparison of apples and pears?
3. Metrics enable the team(s) working on innovation to know if they're making progress
It's important for those working on innovation (as it is with anyone in any team) to be clear about what success / progress looks like.
This is where things often get sticky as the most valuable thing that an innovation team might be able to do is to stop everything they're working on in a particular period, learning lots but not progressing those specific ideas through the funnel.
We had a good discussion around introducing "project / idea failure rate" metrics as people had had mixed experiences of doing this. Failure rate metrics only work once you have strong psychological safety within the innovation team and a genuine belief embedded into the culture of the team that failure is ok and important - broader organisational and societal norms mean that it can still be hard for people to feel comfortable with failure, especially failures which are made public by published metrics.
Several people had also found that these failure metrics didn't land well with senior management. Even where the senior management intellectually understood that a healthy pipeline with a decent level of ambition needed a high failure rate, they still struggled to embrace metrics around this in a positive way. The companies which do authentically celebrate failure are those where innovation mindset and ways of working is an integral part of the organisational culture - from the very top of the business and right the way through it, championed and role modelled at every opportunity - "it's how we do it, not how we talk about doing it". We talked quite a bit about the systems and processes in place at Google - probably the topic of a separate post but there's a lot of useful stuff on this website and also in Work Rules by Laszlo Bock, which might be a bit out of date now but still has a lot of great stuff in it.
A failure metric which people had had more success with was "time to fail" (i.e. of the failed ideas, how long did they take to fail). In most companies this is far too long because of bureaucracy, sunk costs and loss aversion. Someone also said that failed ideas were kept in the pipeline for too long because there weren't enough other ideas in the pipeline to take its place - and I would assume senior management were interested in the size of pipeline. When it comes to failure, we also said that it can be useful for the CEO (or other leader) to ask themselves in all honesty "what would a new CEO do?".
Ultimately, however, innovation is about delivery to market and so failure can only go so far - if, at a portfolio level, your innovation delivers no growth / value to the business over a number of years then your innovation isn't delivering. But you need to be clear about the period of time you're making this assessment over - and not take metrics that are used at that portfolio level and over that time period and apply them at a more granular level.
We also need to get better at tracking benefits realisation for projects which have come from the innovation area and been delivered / scaled up within the business - to what extent are they delivering the value that was anticipated? Why / why not? And what can we learn from that? We need to be measuring value right the way through to the end of delivery, not just within a silo of a specific innovation team or project.
4. The right metrics can help change behaviours & culture
Where an organisation is looking to change culture, behaviours and ways of working (e.g. encouraging experimentation), metrics can be a really effective way of doing that. Again, there are some great examples of this from Google.
But again, the value of these metrics comes back to whether the senior team is really bought into the importance of culture and behaviour change in enabling innovation - if they aren't then these metrics, whilst agreed at the outset, might be discounted when it comes to making an assessment of innovation progress and hard decisions about future innovation investment.
On the flip side, we discussed the risk that implementing a robust metrics framework might kill innovation culture and drive people away from the team. This is definitely something to be mindful of - how can you set up a system which is fairly light touch, pragmatic and useful? I've seen this where start ups have grown and need to implement more robust systems and processes (e.g. HR, Finance - fancy raising a PO every time you buy something?!) and people who were there at the beginning need to have some very honest conversations with themselves about whether this is still the right organisation for them. Some (and the right) metrics are necessary - but can you take it too far and end up driving people away or stifling culture? What is the culture you want and how can metrics be a tool to enabling that?
5. Metrics are a communication tool
Storytelling is a vital tool within innovation teams and metrics enable the team to ground those stories in fact, which is so often important for influencing the rest of the business, especially senior management. Having data and presenting this in a way that makes it easy to understand (e.g. someone talked about using Power BI, which I am a big fan of as I love visualisation) can really help.
One of the attendees had also found that metrics had helped longer term adoption of an innovation project within business teams - by clearly being able to communicate the long term positive impact that adoption would have on the business team (e.g. time saved, duplicated processes removed) they were more willing to go through the short-term workload and change. Innovation projects so often fail to deliver the anticipated value once they are transferred to the business and so using metrics in this way to enable success is a great idea.
Something that came out loud and clear is that one of the biggest challenges people face around metrics is knowing what to measure and when. For example:
Which metrics at different stages of the innovation process? E.g. you need something very different at ideas / test-and-learn stage vs at scale up & launch.
How do you get the right mix of qualitative and quantitative metrics? And how do you make sure everyone recognises the value that each metrics brings?
Short term metrics versus long term impact. One of the challenges seen across many businesses is the inability to stay the course of an innovation strategy or project - how do you put in place metrics that prevent knee-jerk reactions or leaders getting cold feet in the earlier stages of a project or strategy?
Metrics on projects and portfolio vs metrics on the team and the process.
We shared some examples of what has and hasn’t worked within our different organisations but there is much more potential in this area for sharing learnings. And this is likely to be a next step.
And one of the biggest challenges which pretty much everyone agreed on is around influencing senior leadership outside the innovation team so that they are comfortable with making decisions in a different way to core business decisions and using different metrics. There's a lot of evidence that the innovation metrics that really matter are the softer, messier metrics (e.g. around team and leadership behaviours or psychological safety) - but these are not as easy to measure as other less-impactful metrics and they're not as familiar. You need a level of psychological safety at senior leadership level to achieve this.
P.s. we're going to be hosting another breakfast in a couple of weeks time and it'll be interesting to see what else comes up as I am sure that the conversation will go in a different direction with different people in the room. Watch this space…
For more information around future breakfasts and events, please get in touch with email@example.com