Getting Digital Transformation Right: New Opportunities for Market Researchers?

For years now, there have been endless articles, speeches and conference panels discussing the future of market research.  This year, for instance, it is hard to avoid tripping over a POV on whether AI is good, bad or indifferent for the industry and for those who’ve made it their career.  These articles are all well-meaning, and many are thought provoking and smart. 

However, their laser-focus on market research means they have tunnel vision.  Without taking into account the context of broader changes involving digital transformation of entire businesses, these discussions are missing important opportunities for researchers who work within those corporations, as well as those research companies that work with client-side researchers and marketers.

One of the more astute observers of the digital transformation of business over the past decade has been David Rogers, who is on the faculty of the Columbia Business School, and regularly consults with C-suite executives on their digital transformation efforts (and, by way of full disclosure, he and I have had many conversations together over that time). 

He defines digital transformation as “transforming an established business to thrive in a world of constant digital change.”  Crucially, it is not about technology - asking “what is our AI strategy?” is the wrong question, as was “what is our metaverse strategy?” last year or “what is our big data strategy?” the year before.  His argument is that digital transformation is about strategy transformation and organizational transformation. 

Any of those above-stated technologies may help facilitate changes in strategy and organization, but they are not ends in themselves.  His prior book from 2016 (The Digital Transformation Playbook) focused on strategy.  His just published book (The Digital Transformation Roadmap) focuses on the organizational issues that account for why some efforts at transformation succeed, but most fail.

He starts with a simple but startling fact – according to several major consultants’ reports (McKinsey and the usual suspects), around 70% of all digital transformation efforts fail.  Too often, it is viewed as a project, not as an ongoing process, with the Chief Technology Officer placed in charge of implementation; otherwise, business remains as usual.  When you look at the 30% who do succeed, they transform their culture and organization, and embrace the Digital Culture of companies that began in the digital era.

He summarizes Digital Culture with these points:

 

A company imbued with Digital Culture has a very different culture and mindset than the very traditional companies that many of us have worked in historically.  Rather than decisions being made on the basis of what Rogers refers to as “HIP-PO” (Highest Paid Person’s Opinion), decisions are made on the basis of data. 

It is an organization with a relentless focus on addressing customers’ needs – and solutions can come from anyone with an insight into those needs, irrespective of rank or department. It is a modest organization, with senior management freely admitting that it doesn’t know all the answers, but is willing to support experiments until it finds those answers, and will bring people and resources together from different functions to do so.

While Rogers doesn’t specifically discuss what this kind of transformation means for the market research industry, if one thinks about areas of explosive growth the last few years in market research – particularly in Research Technology and Data Democratization tools – they are natural choices of companies who’ve made the kinds of transformation Rogers advocates.

Being able to explore existing first-party data sets in new ways by individuals across the organization, being able to quickly and easily test new ideas for meeting consumer needs, and then iterate quickly based on what you learn, are all critical components of what will drive success in a transformed organization.  ResTech tools have been successful not because they are “merely” cheap and fast, but because they meet an emerging corporate strategic need. 

As the corporate transformation that Rogers advocates becomes more widespread, ResTech will only become more firmly entrenched.  Opportunities will grow for those who do a better job of demonstrating to clients how their systems are flexible, can grow with their needs, and can facilitate their own digital transformation efforts. 

ResTech companies can ensure their future success by better optimizing this process for client companies, which could mean better technology (and yes, AI can be part of that) and better consulting on how their tools can help clients on their own journey.  (Implications for client-side Insights departments will be discussed later.)

The change in culture these changes support provide the basis for what he calls the Digital Transformation Roadmap, and is the heart of the book. The Roadmap has five steps (think of it as a wheel; there is no real beginning and no real ending, the process is ongoing), and drives transformation across the entire company (and I crib here liberally from many places in Rogers’ book):

1. Define a shared vision

• Where is your world headed and how is the context of business changing?
• What is your right to win in that future world?
• What impact do you seek to have on the world?  What problems are you uniquely able to solve?
• What is your hypothesis about how you expect to recover the investments you make for the future?

2. Pick the problems that matter most

• Clear priorities provide direction to digital transformation across the organization, as opposed to just having a series of scattered projects
• Digital efforts are defined by the customer problems they solve, and business opportunities they pursue
• Each statement of problems/opportunities should generate many different possible answers to be tested

3. Validate new ventures

• Fall in love with problems, not solutions
• Think like a scientist, ground every strategic decision in data from your own experiments, not opinions or third-party data.  Every experiment should begin with a theory and aim to validate or refute something specific.
• There are four stages of validation, and they each test different things:  the problem you think you are solving, the solution you think will address it, the product you think the customer will use, and the business you think you can build from this innovation.
  
  • These four stages are done in sequence, but also overlap.
  • You can’t build a business case until you’ve done the first three stages.  Otherwise, you’re just making it up.    

4. Manage Growth at Scale

• Structures are established in the business to provide resources and governance for innovation. These “boards” greenlight new ventures, allocate funding as the new venture go through the four stages of validation, provide guidance to the teams, and act as the liason between the venture team and the rest of the organization.
• New ventures move fast, led by highly independent, dedicated, multifunctional teams.
• Resource allocation happens quickly.
• A steady stream of innovations is managed with “smart shutdowns” to free up resources.

5. Grow Tech, Talent and Culture

• What is the right mix of technology, talent and culture to support the digital growth trajectory of your business?
• Utilize modular architecture of technology to ensure your technology is scalable, secure and flexible.
• Do you have a strategy for how you will invest in and grow your company’s data assets over time?
• What changes have to happen to your company culture to support all of the above?

Rogers has separate chapters on each of these five points, and besides detailed descriptions and examples in each one, he provides a number of tools and self-assessments readers can utilize to help their own companies’ digital transformation efforts. One of the strengths of the book is that it really is a roadmap, and operates at ground-level, not just the conceptual.

So, what does all this mean for the future of market research and market researchers?  

I discuss some implications for research companies above, but the opportunities for market researchers on the client side are even more interesting. Rogers is clear that insights about client needs to solve can come from anywhere in the organization (as an example, Rogers recounts how the idea for Amazon Web Services came from a mid-level engineer). 

If so, what do you need a “Department of Insights” for?  Perhaps it is one of those silos that need to be broken down, and should exist as a more virtual organization?  I think the answer for individuals (as opposed to a department) lies in specific skill sets that market researchers have in “thinking like a scientist” and understanding how tests can be run on ideas to help improve and develop them.  If the future lies in multifunctional teams rapidly doing experiments on ideas/products, then expertise in experimental/quasi-experimental design will be critical skills in these teams. 

A big bonus for researchers is that there is accountability that goes with being a member of these teams; instead of being a “cost center’, and always at risk for cuts, you’re now driving revenue, a much better place to be for career advancement. There is also a role for very senior level researchers to be on the Boards, and using your expertise to guide teams.  Personally, this seems a lot more interesting than the traditional companies I spent much of my career working in.

 Transforming company structure and culture is hard work, no doubt about it, but it is increasingly an imperative for companies’ (and individual’s) future success.  The roadmap outlined in Rogers’ book is a great way help the process of transformation.  It can only happen though if everyone embraces it.  So let me end with a quote from the 1960s – “If you’re not part of the solution, then you’re part of the problem.”  Be the solution.