The State of Innovation with John Bessant

John Bessant has been at the forefront of helping educate us all on innovation for decades. Needless to say, it is a great honor to have John join Colin Nelson, the new co-host of The Innovation Room podcast, for this conversation.

John has dedicated over 40 years to research and consultancy in technology and innovation management. John has advised various national governments and international organizations, including the United Nations, The World Bank, and the OECD. He is also known for his extensive contributions to the literature on innovation, having authored numerous books on the subject. 

Currently serving as Emeritus Professor of Innovation and Entrepreneurship at the University of Exeter, he brings a wealth of expertise to the discussion. 

Colin is joining the Innovation Room to tap into the knowledge of some of the brightest minds in the innovation field. Colin has worked in corporate innovation for over 20 years, initially as an intrapreneur within the technology sector, and for the last 15 years as a consultant to the industry.   

A social scientist at heart, Colin has been at the forefront of enabling organizations and communities to harness the collective intelligence of large or diverse groups to create real impact.

Currently, Colin serves as Chief Innovation Consultant at HYPE Innovation.

Colin: Let us start with your background and your career. We've known each other for a while and have discussed innovation from time to time. I'm curious, how did you first get into innovation? 

John: It's a combination of accident and frustration. Once upon a time, I was a chemical engineer working in the chemical industry. But one day, my old boss from university suggested I pursue a PhD. Back then, I thought PhDs were all about sitting in a lab, conducting experiments. But this one was different; it focused on understanding how innovation works by actively participating in it within a company. 

That led me to land a job as an engineer in a chemical company, tasked with studying how innovation occurred and could be improved. Now, this was back in the mid-'70s, quite a while ago, and the field of innovation wasn't as advanced as it is today. Many people really had no understanding of it. It was essentially a kind of a barefoot experiment. 

So, I got thrown in the deep end, and then frustration stemmed from my feeling that even though I had been working in big companies before that, we didn't quite get the hang of this innovation thing. We didn't fail, but things could have been faster, products could have been better for the market, process changes could have been smoother. All these aspects of innovation, which we now understand, frustrated me. So, this opportunity allowed me to study it, and I loved it. 

I had three years for this project. There were around 1000 people on-site, and gradually, they became enthusiastic about it too. We had great fun, and I believe we made some positive changes. But this experience ignited my passion for innovation. 

Another thing was that I was fortunate to ride a technical wave. Nowadays, we experience these periodic revolutions. However, back then, it was the era of the microelectronics revolution. They had just begun to extensively use this technology, and people were talking about it just like today people talk about AI – microelectronics was going to change the world. Not everyone understood how it functioned as an innovation. Having direct experience in the field helped me in building a research career at the university. 

I've never really felt like I was a traditional university person. While I enjoy researching and understanding things better, my laboratory is the real world. I like to try ideas out, learn from and with the real world. 

Colin: Given that standardized approaches, now considered commonplace in innovation, were not established back then, and innovation was at a very different level of maturity, how did you judge what good looks like? How did you reach a point where you could discern good practices from ineffective ones? 

John: When I was doing research, I noticed that the frustrations I felt were shared by others too. People felt, 'Hey, we could do better.' You can put metrics around it. For example, how long did it take us to get a new product to the market? And when it got there, was it the one that the market actually wanted? That's a fairly straightforward measure. 

The aspect that I was particularly engaged in was process change, introducing microelectronics, new technologies, and getting them accepted. And of course, you can imagine the shop floor wasn't too keen on technology that could potentially change their skills or could potentially put them out of work. Success is very much about bringing the organization with you. It's very much about the feeling of confidence – yes, we are doing this right, and we've learned to do it, rather than that feeling of frustration.  

So yes, you can stick some nice, neat metrics on it, but I've always felt the innovation story was a little more complex than that and much more human. 

Colin: And much more interesting, isn't it? More nuanced depending on the nature of an industry, a culture, and a set of objectives that an organization has. It isn't as simple as one standardized approach that everybody can just copy, which makes it more interesting and creative in terms of figuring out the right fit for a particular organization.

Colin: At what point did you transition to formal teaching and writing on the subject? 

John: Well, I finished my PhD and then was looking for work, and a fairly traditional route was to pursue a postdoctoral fellowship. And as during this period microelectronics revolution was taking off, I got funded by the Department of Trade and Industry [UK, dissolved in 2007]. This government department was examining the impact of microelectronics across various sectors. 

It was a great job because it gave me the opportunity to visit numerous firms of all sizes and shapes across diverse sectors to understand the potential impact of microelectronics. I had a chance to talk with many managers about their concerns regarding its implementation, touching upon more human-centric questions such as how to derive benefits from it.

I branched into consulting through a government scheme aimed at promoting the adoption of this technology. A classic example was that the industry recognized the need for action but hesitated due to the associated risks. In response, the government offered cash incentives to fund feasibility studies. I collaborated with a startup within the university comprising of electronics engineers, taking on the role of technical sales. 

Together, we approached companies, informing them that the government would finance their exploration of this technology, emphasizing that they had nothing to lose. It served to bridge the innovation gap, which is often a challenge for universities, as they possess knowledge that is difficult to translate into practical applications. 

Colin: I think this is an interesting topic, especially the introduction of new processes and technologies. I perceive two distinct worlds, and I'm curious about your experiences when you first entered the field. 

On one end of the scale, there are organizations introducing new methodologies and technologies primarily to drive growth. Their goal is to excel in the commercial world, striving for increased profitability. 

Conversely, on the opposite end, there are organizations prioritizing automation and the adoption of new technologies to enhance efficiency. This conversation with companies about innovation is an interesting one, do they view it as a means to innovate products, capabilities, and business models, or do they focus more on efficiency? 

Back then, when it was in the era of microelectronics, what was the angle of approach? Did organizations emphasize automation and cost reduction, or did they have a bigger vision? 

John: It varied. Innovation has always been about doing better for me. It's about doing better what you already do, focusing on efficiency. This emphasizes the importance of refining our processes rather than inventing entirely new products. So, it's about achieving efficiency through improvement and differentiation. 

And of course, what electronics brought about was massive. The scope for productivity improvements, efficiency gains, particularly getting tighter control, that's what it was very good at. It widened the scope for productivity enhancements and efficiency gains, particularly in achieving tighter control. However, it also enabled possibilities that were previously unimaginable, presenting new challenges. 

And there were visionary companies who looked at the situation and thought, "We've been in this game for a long time, doing things the analog way. What if we explored the possibilities with digital?" However, they recognized the need to educate their market about these new advancements as well. 

This is where we do a lot of things better that's going to keep us afloat, but let's also try the different stuff and get the balance right. 

Looking back, it was clear even then that there existed tension, particularly in competing for resources. But the smarter companies adopted a portfolio approach. They focused on improving existing processes to keep them afloat, but they also were trying new things to get the right balance. 

Colin: I think this is a really interesting observation, and you're absolutely right when you mention that when resources are in question, conflicts and competition are inevitable. We often see this in traditional industries where the world of continuous improvement, innovation, and research and development (R&D) are kept separate. 

You’ve done a lot throughout your career, including teaching, writing, consulting, and advising governments and other institutions. Of all these experiences, which one do you find most enjoyable? 

John: I think I could crystallize it: it's enabling learning. Now, that's much more than teaching because, as I've implied, I see my laboratory, my space, as much more than just the classroom. It is more about helping people to learn about innovation, and that's a selfish motivation because it is helping me learn. I still enjoy the fact that when I try and explain it to someone else, I understand it better.  

The aspect I've enjoyed the most is non-traditional teaching. While I do like teaching students or an MBA class, what really excites me is taking a group of people who've never even considered innovation and giving them an appropriate language, relevant concepts. That involves translating what I've learned from other areas. It's been enormously rewarding and exciting.  

The other part I enjoy is bringing practitioners together. I'm a great believer in peer-to-peer learning, learning networks where people can challenge each other. You can't fudge it - if you're not doing something well, someone else will call you out in that environment. So, facilitating that shared peer-to-peer learning, I love it. We get a lot from it. 

Colin: I often find that organizations can be categorized based on their level of innovation maturity, which is usually closely connected to their historical emphasis on change and reinvention. Certain industries are compelled to continuously evolve to survive competitive pressures. Every organization innovates to a greater or lesser extent.  

What's particularly interesting is when organizations at a lower level of experience, such as service-oriented organizations that have remained relatively unchanged for decades, tries to learn from an organization where innovation is ingrained in their DNA and often many of the challenges they encounter have been faced before, albeit from a different perspective. I think this is where the magic happens. This sounds like something that you enjoy. 

John: In many of Shakespeare's plays, there's a recurring character known as the fool. The fool is the one who can openly criticize the king, saying, "That was a really silly thing to do, your Majesty." Anyone else would risk losing their head. But the fool is licensed to play that role, and I like that idea. I believe I can help people in learning while occasionally challenging them, prodding them with criticism.

Regarding what you just mentioned, I see it as a two-step process. Firstly, there are organizations that don’t really know much about and haven't seriously considered innovation. Step one is thinking about and making it a regular practice, asking, "What have we discovered so far?" This alone is a big step in developing the capability.

However, step two (and what the really smart organizations are doing) is consistently challenging oneself. These organizations question whether what was once correct still holds true, or if changes are necessary. This, to me, relates to what you mentioned about innovation maturity and where I observe the most mature players, those who take time and give themselves a space to challenge themselves to change. 

Colin: What have been the biggest challenges that you faced over your career in modernizing this industry, and helping and supporting innovators to learn better? 

John: So much of it is embedded in the mentality of "we need to keep performing," sticking to what's working without much thought. So, the first challenge lies in setting aside time to think about things differently and finding the motivation to do so. This can be particularly difficult unless there's a pressing general challenge prompting it. 

Take, for instance, the start of lean. Lean was a revolution, suggesting that success isn't all about technology but also about how you organize and approach the entire process. This was a significant shift. However, it took widespread adoption of lean principles for momentum to build. 

Similarly, there's the challenge of discontinuous innovation. Occasionally, a major breakthrough occurs, upstaging current trajectories. It's only when there's a critical mass of people discussing it that organizations depart from their traditional approaches, opting for a new direction instead of staying on the same trajectory. 

Colin: But that implies a different question: What is the role of the visionary leader? It's someone who possesses the imagination to support the organization, to facilitate shifts, rather than just waiting for the masses to adopt it and trying to be like someone else. Did you see this as the driving force behind the shift? Those with a vision could get ahead of others and then pull everyone else forward with them. 

John: This takes us back to what I mentioned about the position to challenge the organization, while also possessing internal credibility to do so effectively. Numerous consultants and academics may hammer on the door and tell you what you should be thinking about, but it's easy to brush them off. However, having someone inside the organization is much more effective. 

I had a great privilege to work with a big German company and write their innovation history and that was a wonderful experience. But without question one of the things that had helped me enormously was their CEO, who also happened to be a partial owner of the company. That definitely didn’t hurt. He gradually moved away from day-to-day operations, as there was a great COO overseeing things. Instead, he focused more on being the challenger, someone with the time, space, and breadth to be visionary and to bring those challenges in. Not to railroad them, his role wasn't to dictate, but rather to facilitate conversations and debates. I found that very, very interesting. And for me, that's an example of a real innovation leader.

Colin: It's this sort of conflict, isn't it? Some organizations feel like they have to maintain the status quo, unable to quantify the cost of standing still. And so, they stagnate for lack of another idea. On the other hand, there are leaders who recognize their responsibility to protect the organization's future, preparing it for whatever lies ahead. 

This leads me to my next question. You're the first person I've heard discuss the concept of dynamic capability. I have always been fascinated by an organization’s ability to cultivate agility, by design to be flexible and amorphous, no matter the scenario. Could you elaborate on this concept? I believe it's one of the biggest changes in the business landscape over the past decade or two – this desire to become more agile. For those unfamiliar with the concept, could you explain what it entails and how it is manifested? 

John: Dynamic capability originates from the work of David J. Teece. For me, to put it simply, it is about innovation being important and we need to learn how to do it. There is a set of things we can learn to do. We've got to contextualize it. So that's the capability.

Dynamic capability is “the firm’s ability to integrate, build, and reconfigure internal and external competences to address rapidly changing environments”

 

David J. Teece, Gary Pisano, and Amy Shuen

For me, the essence of that dynamic element lies in the ability to step back and reset. It is a bit like adjusting a central heating controller: when It’s hot outside you turn it down, and when it's cold, you turn it up. Similarly, organizations need the ability to step back and reset. And it might be fine-tuning, and occasionally it might be a major shift. 

A textbook example, quite literally, would be Procter and Gamble's (P&G) big shift. After 150 years of doing good R&D, and market research they flipped to what they called Connect and Develop - their open innovation approach long before open innovation became a widespread trend. This shift required a fundamental rethink, and it took them 20 years to work it through. But that is dynamic capability – the ability to step back, reassess, and reset in response to a changing world. 

Colin: I remember when Connect and Develop first emerged and I've seen various iterations of this approach in different organizations as they tried to learn from this thinking.  And one of the observations I have is that there's been a shift from thinking that everything must be owned internally. 

In the past, the focus was on generating and retaining as much intellectual property as possible. Now, there seems to be a shift in attitude, companies need to be fast, dynamic, and smarter in how they deal with the outside world, maybe co-creating with emerging companies rather than building it internally. 

But of course, not every organization is structured in such a way like P&G to take advantage of open innovation. Are you seeing this same trend, this move towards the notion that we need to be smarter with our value chain regarding creating new capabilities, while also considering the importance of owning everything inside? 

John: Absolutely. I've been in this game for quite a long time, and I've seen some big changes. Without question, in the early days, companies saw themselves at the center of things and then they realized that actually, to succeed they had to play with other players, key partners, key suppliers, and key customers.

To suceed, you have to play with other players, key partners, key suppliers, and key customer.

What’s transformed in the 21st century and is accelerating is this networked model, the need to build an ecosystem. That isn’t about one player, it's about mutual dependencies. And it's hard to manage because you can’t control it, you need to learn to work with it. 

So, we've got a new book coming out about scaling innovation, and the big challenge there is you don't scale on your own, just like you don't climb Mount Everest on your own, you've got to partner up. Some of those partners want to get in there with you because they are going in the same direction, others you need to get on board who have very different views. You've got to work at those relationships. But I agree entirely, I think it's about building shared value across networks, creating an ecosystem. 

Colin: When I talk to different ends of the value chain, different members of an ecosystem, what I see is consumers putting pressure on the consumer products companies to be more sustainable, to have a product that could be recycled and is part of a circular economy. And that ripples down the value chain as organizations change what they sell. But my observation is that this change is very slow. So, if we continue with this waterfall approach to adjusting our practices to enhance our sustainability credentials, it might take 5 - 10 years to see changes throughout the entire chain. 

Recently, discussions have emerged within the network ecosystem questioning whether there might be a more efficient approach. Could we bring both ends of the value chain together to accelerate our shift toward a more sustainable future? To what extent do you believe sustainability is pushing for the development of ecosystems, and where do you think this stands in terms of its progress? 

John: It's a great and really important question. We did some work a few years back for the Network for Business Sustainability. They were asking us, 'So what do we know about sustainability and innovation?' And we came up with a framework that resonated quite well. Level one is to do what you do better; it's optimization. You're crazy if you don't do that because it’s going to save you money. So, it's kind of a no-brainer. 

Step two is what a lot of smarter organizations get: yes, we could change our products and services that we offer and build businesses out of that. And there's the business-level advantage to doing that. Where the big gains come, but where the challenges come too, as you said, is at that system level, bringing the different players together, and it's really hard to do. I think that's the challenge. That's why it takes a long time. I think you're right, one of the problems is we don't really have so many tools to help us even think in system terms. 

I remember way back when I studied engineering, systems thinking was a particular mindset, which is seeing all the bits and how they interact. In a chemical plant, you don't want to waste heat. If you spent a lot of money and energy heating something up, think where can you use that heat? Where can you plug it in elsewhere? This kind of system thinking is what we need. If you then can configure ways in which you get some of those key players together and have them co-create a better system-level solution, that's exciting. But then you’ve got to get back to work on shared value in networks. You've got to demonstrate a piece of benefit, a piece of value for everyone who is going to play in that. 

And whilst the optimist in me would like to say it's for the good of the future and our children, the fact is businesses are, to some extent, quite selfish. Getting that bigger system-level thing, seeing the emergent properties we need, it has been and can be done but we still need quite a lot more tools to help us do it. 

Colin:  I agree. I think it's not there yet in terms of the process knowledge and the system thinking. You're absolutely right, there are little slices of emerging good practice here and there, but it's certainly a long way from the consciousness. I really like your analogy with the heating point. I've seen organizations that are conglomerates. In one part of their business they are generating heat and in the other - they need heat, but the factory that needs it is 50 miles away, and they never thought to bring these things together, to optimize that aspect. But it's the sustainability agenda that is making them think differently now. I'm just fascinated to see where that goes. 

So, sustainability is one of the bigger challenges that the organizations are grappling with, but what are the other big themes that you think, if you project out the next 5-10 years, this industry has to deal with in order to progress to the next level? 

John: I think a huge challenge, which is still in its infancy in terms of our understanding, is AI. 

I can remember a pre-Internet world of innovation, and that shift didn't happen overnight. The fact that we could join things together and call it the Internet didn't mean that we were using those tools effectively for probably 10 years. It took time to utilize them well, to the point where now we have collaboration platforms and networks that are sophisticated. 

I believe we're in a similar space with AI. It holds enormous possibilities, but we really underestimate how fast it's changing. We probably also underestimate how much we could configure it ourselves; we're somewhat letting it happen to us. I think we could be a little more proactive. So, I believe AI will be a significant factor that will change the way we approach innovation. 

The optimist in me envisions it as a copilot, sitting in the cabin with you. You can get assistance from it, but you're still in control, setting the destination. 

Colin: I see many organizations struggling with a lack of data scientists. They have difficulty finding individuals with the necessary skills in-house, whether due to quantity or because those with the required expertise are often expensive, I believe addressing this shift is essential before we can fully leverage its potential. 

Another topic that keeps coming up, and I would appreciate your feedback on it, is the ongoing disconnect between executive leaders and innovators regarding a common language for innovation. CEOs, COOs will say that all they care about is money: money in, money out and how quickly can they get to that money. 

Meanwhile, corporate innovators feel frustrated by what they perceive as a lack of understanding from upper management. They express concerns that their innovation efforts are not aligned with the targets set by executives. I view this as a significant area for improvement as the industry matures. I'm interested in hearing about your experiences in this area and any lessons we can learn to do better. 

John: Take 10 people – you will get 11 definitions of innovation! That is part of the problem. If there's something I've achieved during my career, it's simplifying a paragraph-long definition of innovation from back then to a one-liner today: innovation is creating value from ideas. 

But the way in which innovation happens, the different perspectives, I think you're absolutely right, that's a big part of the problem. In our textbook, we have a table: if you only see innovation as this, then you're going to get these problems. And there's a whole table full of these problems.

Source: Joe Tidd, John R. Bessant, Managing Innovation: Integrating Technological, Market and Organizational Change, 8th Edition

What we need is this kind of integrated view into what innovation is. And it is about language, it's an educational thing. I do think it’s a big challenge. 

I think sometimes we do better without labels, talking rather about what innovation does than what you should call it. 

Learn more about John’s work and follow his blog at https://johnbessant.org. 

Connect with Colin and learn more about Hype Innovation consultants here. 

You can listen to The Innovation Room Podcast on Apple Podcasts, Spotify, and YouTube.