International

Panels / Communities

Software & Technology

Cint is a global software leader in digital insights gathering. The Cint platform automates the insights gathering process so that companies can gain access to insights faster with unparalleled scale. Cint has the world’s largest consumer network for digital survey-based research, made up of over 155 million engaged respondents across more than 130 countries. 
 
 
Over 2,900 insights-driven companies - including SurveyMonkey, Zappi, Kantar and GfK - use Cint to accelerate how they gather consumer insights and supercharge business growth. Cint Group AB (publ), listed on Nasdaq Stockholm, has a rapidly growing team across 14 global offices, including Stockholm, London, New York, Singapore, Tokyo and Sydney. www.cint.com.

Cint

For businesses looking to stand out and attract more prospective customers

Standard

United States of America

Cint is a global software leader in digital insights gathering.

Advertising Research Advertising Tracking

B2B Research with Professionals & Executives

VISIT OUR WEBSITE

If you’re like me, you have read your fair share of “prediction” articles that are typical of a New Year and a new decade. These prophetic announcements from market research industry pundits vary in quality. Many are self-serving and consistent with the prognosticator’s business objective; few are very thought-provoking. At the very least, they tend to give us perspective about the flood of new products, platforms, and services which also tend to launch in the first quarter of each year. 
Why is this kind of content so prevalent? Most of us are eager to turn the page on the previous year and start fresh. To paraphrase that famous Las Vegas tourism ad slogan, we would all like to believe that what happened in 2019 stays in 2019. Unfortunately, no matter how optimistic we are for change as we say goodbye to auld lang syne, the truth is that—for the most part—we’re in our same jobs, living our same lives, with all their pros and cons. Life is not like our annual budgets: there is no reset button despite the fact that the Earth has made it around the sun again.  
With this in mind, I’d like to offer some New Year’s resolutions for the industry to consider as unfinished business from 2019. 
<h3>1. Let’s Stop Making Excuses for Bad Surveys</h3>
How bad is bad? If one in five people quit a survey⁠—not over quotas, disqualifications, or quality terms⁠—is that too much? (It is for me.)  What about one in four? What about one in three? As an industry, we acted like this problem would evaporate when we moved to mobile. Except it didn’t. In my time at a previous employer, we found that people who dropped out were twice as likely to never come back again. <a href="https://greenbookblog.org/2015/05/14/the-future-of-panels-five-investments-needed-to-improve-participation-and-reduce-bias/">I’ve said this before</a> and will say it again: buyers need to stop acting like dropouts are free, especially since suppliers are capable of—and are now actively —shutting down abusive studies through automation. Why? Because the revenue from that study is less than the cost of replacing all the people who were driven away. 
<h3> 2. Let’s Make Conferences Worth Attending Again</h3>
As a regular market research conference attendee for nearly a decade, and also in a previous century (does that date me?), I’ve been around long enough to have seen pre-digital versus post-digital. It is indisputably cool what we can do as an industry, and it has been fascinating to see this develop in conference settings. Yet in doing so, we have also started letting the tail wag the dog. I get that conferences need to be monetized by their promoters, but in both paid and free conferences, we are now regularly seeing thinly veiled sales pitches and, worse, naked product demos. We&#8217;re the industry that’s being disrupted now, folks. We need to stop pretending we’re the predators and not the prey. 
 
While we’re at it, how about differential pricing schemes for junior staff? The great leaps in my knowledge and my career have come from the learning and networking experiences that conferences represent. We shouldn’t be pricing people out of them.
<h3>3. Let&#8217;s Pay our Bills on Time</h3>
I can’t even believe I have to write this. No matter your company size, or its role in the industry, it is time to sort out your cash flow issues and pay on time. As an ordinary consumer, this kind of behavior would result in lousy credit and all the misery that entails. In short, it is the right thing to do. Pay your bills and keep positive relationships with your partners, vendors, clients and everyone else in your circle. 
<h3>4. Create a Better Workplace for Others</h3>
I believe that there are two unusual things about our industry that affect us in the workplace. One is that practically none of us started out on our career path with a Eureka! moment of: “My God, that’s it! I need to be a market researcher!” The other is that—despite the late hours, the consolidation, the massive change that is bulldozing the industry—most of us like what we do. But people are also deeply frustrated by things like lack of opportunity, tyrannical colleagues, and company cultures that tolerate and perpetuate negativity. Truth be told, this is probably also the case with other industries, but that doesn’t mean we have to accept it in ours.   
 
There are steps we can take to change this reality:
<ol>
<li style="list-style-type: none;">
<ul>
<li style="font-weight: 400;">Be a mentor to someone. While there are formal mentoring programs, like one I’ve participated in through <a href="https://www.womeninresearch.org/mentoring/">Women in Research (WIRe)</a>, you can be a mentor in your own office. Build others up, share your knowledge, give junior staff in your company opportunities to grow their careers. You will uncover your own personal benefits in providing guidance, motivation, emotional support, and role modeling for others. </li>
<li style="font-weight: 400;">Reward and recognize. If you are in a leadership role, use your influence to motivate your employees. This can mean helping people understand how their work is contributing to overall company goals, making what they do meaningful and valuable. It also clearly means investing in external motivators such as salaries, bonuses, and more tangible rewards. It’s simple: recognize a job well done and give people a sense of ownership within the greater structure of the business.</li>
<li style="font-weight: 400;">Listen. We have two ears and one mouth for a reason. How many of us heard that growing up? The truth is that many people don’t feel “heard” by their employers. With the massive change that is taking place, and the unique perspective that the younger generations are bringing to the table, it’s time we stopped adhering to the status quo just because that’s the way it’s always been done. Listen to new ideas, give everyone an equal voice in the workplace. You might be surprised where you end up.</li>
</ul>
</li>
</ol>
<h3>5. Let’s Insist on the Same Representation Within our Companies as We do in our Studies</h3>
We need to put our money where our mouths are when it comes to diversity in our own teams. As we showed in the recent paper I authored with Women in Research (WIRe) called “<a href="https://cdn2.hubspot.net/hubfs/2759897/eBooks/Women%20Executives%20in%20MRX.pdf">Women Executives in Market Research</a>”, we have a long way to go toward equal representation. We found that only 36% of executive roles across 133 companies were occupied by women. Until we start taking targeted and immediate action in the form of explicit recruitment objectives, visible measures of accountability, and internal and external engagement with stakeholders, we can’t hope to make progress in this arena. It’s time to stop talking and start acting when it comes to diversity.  
Can you keep these resolutions in the coming year? Apparently, 80% of New Year’s resolutions fail before we even hit February. The trick is making it stick. The combination of focus, setting daily, weekly and monthly goals, and simply getting started is what it takes. Let’s find ways to make sure we have made progress in these issues when that next flood of predictions hit as we welcome 2021.

5 Resolutions You’ll Probably Ignore in 2020, Just Like You Did in 2019

Editor&#8217;s Note:  We’ve had a series of posts recently about issues with sample, and here we have JD Deitch’s contribution.  It is hard to argue with his main thesis, that sample companies must provide greater transparency into their processes.  More than nice to have, it is critical right now if trust is to be maintained in the foundation of much of market research.
<hr />
Study after study shows that the demand for transparency in every aspect of life and business continues to increase. From product supply chains to wage gaps and from brand communications to market research, the digital age has turned into a quest for accurate information that is openly shared. The sample industry has not escaped this trend. 
As buyers start to demand more from their sample suppliers, they want to understand what’s going on behind the scenes. As sample companies make the move toward automated processes &#8211; managing everything from fraud to respondent experience &#8211; they have more of an opportunity and a responsibility than ever before in positively impacting outcomes and data quality. Buyers are taking note, and naturally, want to know more.
What do they want to know? Unveiling the mystery surrounding recruitment, process, and use of technology is a good place to start.  
<h3>How many respondents are there?</h3>
A traditional panel book is used as a marketing tool designed to show clients the number and types of people the supplier has admitted to the panel and who have subsequently signed up to take surveys. While reach is an important metric, panel books generally don’t provide the whole picture, as the counts typically don’t reflect how many of these respondents are active. Sample suppliers need to clearly illustrate important metrics like <a href="https://www.p2sample.com/interactive-panel-counts/?utm_campaign=Newsletter&amp;utm_source=hs_email&amp;utm_medium=email&amp;_hsenc=p2ANqtz-9l5y9o6RKL1_arTrgpxDFVLMqM2UzcKYZD4tysiJHc44AexAyG00F1Um8Qe51iLK78SbgI">recent activity,</a> completes and spot feasibility. This can start to provide a more tangible understanding of the supplier’s ability to provide the right respondents for a study. 
<h3>How are respondents being obtained?</h3>
Making sure that the respondent base for a study is diverse, providing a representative cross-section of the global population, requires recruiting participants from multiple sources. Sample suppliers that have a multiplicity of partnerships, who can recruit from hundreds of affiliate networks, publisher networks, blog networks, loyalty sites and big online communities, websites, and social media platforms, can help ensure that respondent base will provide quality data. Understanding who these respondents actually are is another important part of this. With the massive amounts of data now at our fingertips, each individual can &#8211; and should be &#8211; deeply profiled with a wide number of demographic points to allow precise targeting. 
<h3>What technologies are being employed to ensure quality at every step along the way?</h3>
Automation and AI provide the sample industry with a way to positively impact essential functions like feasibility, fraud mitigation, respondent satisfaction, and subsequent data quality. By using automation technology at every stage, operational efficiencies can deliver everything from fraud mitigation to respondent satisfaction. APIs can improve supplier earnings and reduce costs to remain competitive. 
When addressing fraud, using artificial intelligence &#8211; and its machine learning subset specifically &#8211;  allows us to study patterns in real-time, and compare pattern behavior with a survey’s unique set of criteria. AI can analyze billions of data components very quickly and detect anomalies, such as large surges of users with specific demographics. Moreover, it can detect shifting patterns to maintain a constant yet dynamic vigil.
For example, 1200 males participating in a survey wouldn’t throw up any red flags. But 1200 men from New Orleans, who have diabetes and identify with the Independent party might be a problem.  Next time, it could be 500 females who own two or more cats. The machine can detect these evolving fraud trends as it learns to adapt based on the data that “teaches” it. Traditional techniques are more static. The massive amount of data points we have available to us as sample suppliers make this a good solution: as we feed more data into the machine, the smarter it becomes at classifying a survey complete as good or fraudulent. 
It’s time we all realized that sample is about much more than price. Sampling is fundamental to market research and provides the foundation for quality insights, so determining a supplier’s practices is vital to success. Suppliers should be able to answer hard questions, including those above, as a matter of course. If they cannot, it may be time to move on. The sample being used is ultimately is providing important data for business decisions, and demands for transparency will continue to rise. Meeting these demands will help change perceptions of sample companies and make meaningful improvements in data quality.

Transparency: Shedding light on sample

Editor&#8217;s Intro: Survey research is increasingly challenged to be better, faster and cheaper. Automation of sample is one important tool to help the industry get there. JD provides a great overview of where we are with sample automation, argues that it is underutilized, and gives a vision of how the industry can do much better than it is right now. This is a timely, important article for survey researchers to read and take action on.
The insights industry loves to talk about automation, which has the potential to speed up processes across the board. In the sample space, current applications are often limited and difficult for buyers to understand what suppliers are actually doing and what benefits are on the table. In fact, it is precisely because most applications of automation are limited that people feel it is more buzz than brilliance. But the truth of the matter is that, when used to its fullest potential, automation can have massive benefits for process and outcomes.
Sample companies can be using automation throughout both their internal and external processes in order to create not only efficiency and lower costs, but also fundamentally better data and agility for clients. Here are 10 ways that sample companies should be using automation:
1. Respondent Engagement
This application of automation is barely used in the industry, yet it represents a concrete and necessary way in which suppliers must take responsibility for sample quality. Sample companies have the ability to track respondent touchpoints ranging from recruitment source, to profile data, to project experiences. These data points can help to indicate the level of attentiveness and engagement of the respondent, helping to determine if the respondent is “real.” Even surveys can be evaluated in this manner, using data such as field statistics and respondent ratings. Automating the evaluation of respondent engagement and survey quality up front can help to not only match the right person with the right survey and maximize the likelihood of survey completes, but it can also have a positive impact on things like over-quota, recontact and retention rates. Taking a thoughtful automated approach to respondent engagement can lead to significantly higher satisfaction and engagement. That means better data.
2. APIs
It is important to remember that automation is not the same thing as an API, or automated programming interface. An API is the exchange of structured information between two systems or platforms. At a minimum, sample APIs facilitate the exchange of information on project specifications. Sample companies should be well-versed in custom APIs to programmatically exchange project specifications to quotas to demographics and hundreds of profile points between supply and demand partners.
3. Buying &amp; Selling
Buying and selling of sample is the most widely-used aspect of automation in sampling. Here, buyers and sellers share project specs and sample counts via an API. At the point of project commissioning, sellers launch sample at the project and transmit demographic information about the respondent. What gets transmitted is often limited though. The process of mapping data fields—that is, sharing demographic and behavioral data about respondents using pre-arranged codes—is tedious and requires regular maintenance. When it is not done, respondents “leak” out of the system or have to repeatedly answer questions to which suppliers already have answers.
4. Feasibility
Over the past few years, the feasibility process has changed dramatically. Yet, many sample providers still manually produce feasibility calculations in spreadsheets using simplistic formulas. Automated feasibility processes can not only take into account a huge number of factors&#8211;from field parameters to individual behaviors&#8211;to provide accurate estimates of how a project will fill, but also deliver greater speed, accuracy and dependability.
5. Quota Management
Automation allows us to keep a silent and constant vigil on each and every project, detecting anomalies instantly, closing quotas automatically as they fill, and otherwise quickly and efficiently ensuring projects remain on track and raising alarms when they are not. Automation thus brings peace of mind to the research buyer and better experiences to the respondent. In a manual environment, problems usually aren’t detected until it’s too late, and respondents suffer.
6. Recruitment
Recruitment is one of the areas that suppliers have at least partially automated, but rarely to its fullest effect. When fully automated, sample suppliers can trace panelists all the way back to the source to optimize sourcing based on quality scores, which intrinsically ensures that they are recruiting real, attentive, engaged, deeply profiled people who genuinely complete studies. By using automation effectively, sample companies can allow for on-the-fly recruitment based on need. Automating the entire process allows the management of more recruitment partners, which means greater reach and diversity, and less “groupthink”. When automation is properly implemented in the recruitment process, it improves everything else downstream, from feasibility estimates to response times to economics for suppliers and buyers.
7. Field Monitoring
Field management is perfectly suited for automation. Very similar to #5 above, Quota Management, sample companies should be using automation to achieve the same goal: making sure nothing goes wrong in field immediately. This nimble and fast approach means problems are less likely to slip through the cracks and can be addressed before they snowball.
8. Incentive Management
Instant rewards is the name of the game when it comes to respondent incentives. Automation of incentives allows immediate delivery, closing the loop with the respondent. This is another place where most suppliers are already using automation.
9. Fraud Mitigation
When it comes to fraud mitigation, there is much work to do to combat the growing menace of fraud. Traditional techniques are still in widespread use, but need to be coupled with more technologically advanced methods to fight the savvy fraudster. Using artificial intelligence to navigate billions of data points that include demographics, behavioral indicators and field experience can make great strides in detecting and combating fraud. A smart machine can instantly analyze anomalies and patterns in the data, such as user “surges” involving specific demographics, IP ranges or completion time frames. By using this kind of technology, sample companies can start to significantly mitigate fraud and meet new threats.
10. Profiling
Automation at this stage can use respondent profile data to enable precise sample targeting across hundreds of qualifiers. This can ensure that the right respondents are matched with the right study without the annoying requirement (for the respondent) of having to answer the same questions constantly! Automation can help ensure proper targeting, improve the user experience and yield better, more accurate data.
Conclusion
We hope this article has helped you understand two things. One is the real potential of automation in sampling. The other is that this potential is already within reach and utterly essential for suppliers to implement. It’s time for our industry to start using automation to its fullest. Instead of partial implementations that create broken processes, using this technology to its full potential can help to build true efficiencies that result in better quality from start to finish.

10 Ways Sample Companies Should be Using Automation

The latest <a href="https://www.greenbook.org/grit">GRIT (Q3-Q4 2017) report</a> states it unambiguously:
“We need to talk about sample quality. Issues surrounding accessing quality and representative sample is [sic] without doubt the single biggest individual challenge mentioned in the survey.”
If we sift through the open ends in the report for those relating to sample, we find three key themes: 
Feasibility and representativeness
<ul>
<li style="font-weight: 400;">“Obtaining a representative sample”</li>
<li style="font-weight: 400;">“Lack of probability samples”</li>
<li style="font-weight: 400;">“Decline in response and participation rates across ALL modes and methodologies”</li>
</ul>
Real, honest humans
<ul>
<li style="font-weight: 400;">“Frustratingly low quality”</li>
<li style="font-weight: 400;">“More and more professional respondents”</li>
<li style="font-weight: 400;">“Fraudulent responses, whether generated by individual respondents for personal gain, or on a larger scale by sophisticated bots”</li>
</ul>
Engagement
<ul>
<li style="font-weight: 400;">“Apathy, fatigue and disinterest makes it difficult to generate reliable, sincere and in-depth answers”</li>
<li style="font-weight: 400;">“Getting quality respondents that meet client criteria to stay engaged over the long term”</li>
</ul>
These issues will come as a surprise to no one, as they have occupied a meaningful part of the industry’s public discourse for years. 
The solutions proposed in GRIT are familiar as well: 
Better surveys
<ul>
<li style="font-weight: 400;">“We have to improve surveys.”</li>
<li style="font-weight: 400;">“Need to cap survey length at 15 minutes and make sure all surveys are mobile friendly.”</li>
</ul>
You get what you pay for
<ul>
<li style="font-weight: 400;">“More training for DIY platforms” </li>
<li style="font-weight: 400;">“Stop offshoring” </li>
<li style="font-weight: 400;">“Pay panelists more” </li>
<li style="font-weight: 400;">“Clients perceive research sample as a commodity, and almost always buy on price”</li>
</ul>
Let’s build a campfire and sing songs
<ul>
<li style="font-weight: 400;">“Come together as an industry”</li>
<li style="font-weight: 400;">“Research vendors should be more transparent with some of the potential issues”</li>
<li style="font-weight: 400;">“Industry initiative with the involvement of all stakeholders”</li>
</ul>
The best one can say about any of these solutions is that they have been unsuccessful. Despite massive amounts of education, including everything from research-on-research to parading real panelists into conferences to talk about how they game the system, long poorly-written desktop-only surveys are still too common. Bemoaning the advance of technology or changes in the market is like shouting at the sun for rising each morning. Anything approaching binding industry standards is a pipedream for suppliers and buyers: the businesses are too diverse and the stakes are too high. 
The worst aspect of these solutions is that none of them are forward looking. They arise from an antediluvian understanding (think: 2007!) of the research market. Equally troubling is that these comments don’t speak to ways in which sample suppliers share responsibility for the blame and solutions. 
<h3>Forward-looking solutions</h3>
As an industry, we do ourselves a disservice by not thinking about these problems in new ways. Here are some possible solutions which are both forward-looking and reason for hope. 
First, we need to do a much better job defining what we mean by a bad experience.
Time limits, though important, oversimplify the problem. For example, we have known for four years (my former NPD colleague, Inna Burdein, <a href="https://www.researchscape.com/blog/is-the-ideal-survey-length-20-minutes">presented this</a> at CASRO in 2013), that people will be happy with longer simple surveys than shorter harder ones. We also know that incentive levels, question format, response types, demographics, topical interest, day of the week, time of the day, a host of other elements impact the respondent experience&#8211;and this does not include very common issues with project setup, quota management, and programming! 
From field conditions to participant data to engagement tests, there are dozens of variables we can use to evaluate the quality of the respondent’s experience. We should also be allowing the respondent to comment as well. Survey ratings can capture this. If a respondent tells us a survey is great or it stinks, we should use that data. When all this data is combined with a supplier’s economics, actuarial techniques can bring more clarity to what constitutes a bad experience, which can continue to be shared with buyers.  
Second, we need to go all-in on technology and automation for sampling
Here’s the thing: the situation is bad enough that simply maintaining the status quo or taking a limited approach to automation is a losing proposition. Manual operations are now extremely fragile and relatively costly; limited efforts at automation (particularly those aiming to simply maximize the number of completes per panelist) can actually have detrimental effects on everything from feasibility to reliability to data quality.
Automation allows us to process this information in real time, allows for variation in inputs (the bazillion different questions someone could ask to a gajillion different types of people under a squintillion different environmental conditions), and&#8211;most importantly&#8211;react to it. In fact, automation and programmatic algorithms are the best hope we have to improve respondent engagement and data quality. They are the only way we can possibly evaluate complex processes at scale, project outcomes, and then take action in ways that would be computationally and logistically impossible for human operators. But yet, we can still have human operators&#8211;trained and well-equipped&#8211;reach out to clients and alert them of real issues that threaten the delivery of their study and the data they are counting on to make decisions. 
Third, for those who haven’t already, we need to migrate our business models to a new future. 
The first two points don’t really happen without people believing that they are experiencing business-critical events. The real trick is repositioning the business, which is like changing wheels on a speeding train. This is where those who believe sample is a commodity should be paying careful attention. For if you believe sample is a commodity, you are essentially saying that you are either unimpressed by your supplier’s commitment to these issues or are otherwise unaware of what is happening in the industry. Fortunately, neither of these is a permanent condition. 
<h3>Conclusion</h3>
The latest GRIT results put an even finer point on an old issue. There can be no disagreement about its gravity. But where we do ourselves a disservice (as an industry) is by imagining solutions that are rooted in the past. The very same technologies and market forces that carried us to this place are precisely those that hold the greatest promise for renewal. Suppliers already understand that this is a road that must be travelled. The faster buyers understand this, the more they will hasten a better future.

Let’s Stop Talking About Mobile Surveys

Sample has a significant impact on study outcomes, but there is no set process for researchers to choose a good sample. Programmatic sampling is one way that the consumer insights world identifies respondents, but there still exists uncertainty about what exactly programmatic means. Below are some key concepts.
<h2>Programmatic is about automation</h2>
One of the central benefits of programmatic sampling, as the name would imply, is the replacement of manual processes by computer programming. With machines doing the routine work of data retrieval and transmission, companies benefit from lower costs, greater efficiency, and reduced error rates. 
<h2>But without APIs, programmatic is not possible</h2>
Automation is a defining characteristic of data-driven businesses these days, but it is simply not possible at scale without an Automated Programming Interface. These interfaces specify how two different systems can communicate. Take the simple example of a panel database and a data collection platform. For the data collection platform to request a respondent, it needs to communicate what type of people (demographics) it needs. The panel database has these demographics defined in a certain way (the coding for different demographic variables), so the API will need to respect those parameters. Now imagine that same panel company is working with multiple data collection platforms. Each might have a set of standards, embodied in an API, that other companies can use to facilitate the exchange. 
<h2>Mapping data: the devil in the details</h2>
The ideas behind programmatic sampling are simple… deceptively so. One of the challenging realities in sampling is that each company stores data in different ways. From age to sex to geography to various behaviors, the “easy” part is building the interface. The hard part is mapping the data, i.e., knowing how each variable is coded and which codes in Company A’s system correspond to which codes in Company B’s. 
It is worth noting that many sample companies, including my own, have been talking about developing standards for years, dating back to the 2014 Samplecon conference. The competing interests make this highly unlikely, our approach to this has been to simply push ahead and build regardless of a partner’s complexity. Because of this approach, we have nearly 100 APIs operating across supply and demand partners.
&nbsp;
<img decoding="async" class="aligncenter wp-image-19640 size-large" src="https://greenbook.org/mr/wp-content/uploads/2017/12/P2Sample-1024x570.png" alt="a graphic from P2Sample of different groups of blue and yellow people flowing into each other to depict programming sampling insights" width="1024" height="570" />
&nbsp;
<h2>Programmatic sampling is less error-prone, not more</h2>
One of the stranger things we hear from clients is the belief that programmatic sampling increases errors. This is untrue. Fundamentally, a computer program that is properly “specific” and targeted is going to make fewer mistakes. In fact, it will actually expose weaknesses elsewhere in the process that are human-driven or less precise, e.g., errors in quota specifications or questionnaire programming. Moreover, an API will not compensate if the supplier’s feasibility model does a poor job of predicting actual completes, leaving the client short of fulfillment. 
<h2>Programmatic capabilities vary a LOT by provider</h2>
Given the technical nature of the topic and practically non-existent standards, there are huge differences in programmatic capabilities across sample companies. Most are limited to the basic transmission of data (limited demographics, project parameters, and field status). Most optimize their platforms to maximize yield, regardless of the conditions for the research participants. As a result, suppliers allow respondents to bounce from router to router for paltry incentives with no thought to the broader implications of future participation or their client’s data. 
<h2>Programmatic should mean better quality</h2>
Being programmatic does not HAVE to mean sacrificing data quality at the altar of speed and price. To this end, sample companies should be using technology to optimize the respondent’s probability of a great experience, from maximizing the expected reward (a function of the likelihood of successfully qualifying and the amount of the incentive) to ensuring that bad surveys, defined so by the respondents themselves, are pushed out of field. If more sample companies conducted these types of techniques, displaying a true commitment to a great respondent experience, I believe that quality will automatically improve.

Programmatic Sampling: The Basics

<img decoding="async" class=" wp-image-16201 aligncenter" src="https://wp.greenbook.org/mr/wp-content/uploads/2016/05/Screenshot-2016-04-15-09.44.46.png" alt="Screenshot-2016-04-15-09.44.46" width="645" height="552" />
By JD Deitch 
Our world of constant, on-the-go connectedness has changed the way consumers participate in survey research. Yes, yes, we all know this.
Why is it, then, that the majority of survey research can still be defined as ‘studies which require participants to remain immobile in front of a computer and focused for long periods of time while they undergo an exercise that hasn’t fundamentally changed its form since the turn of the century when questionnaires were written on multiple sheets of paper?’
What a poor showing the research industry has given for itself! We are, after all, an industry that prides itself on knowing how people feel and act! We’ve let down our clients, who are being driven by these very respondents to existential change, and we’ve let down the respondents themselves, who are indisputably voting with their feet to do something else with their valuable time.
But wait, you say. As an industry we have been signaling this for several years. Everyone knows we need to change. We’ve been telling people to move to mobile, to shorten their studies. We’ve shown them research on research. If only the {sample suppliers, panel suppliers, end clients, client service people} would get their acts together, we’d be fine. 
Setting aside the unhelpful finger-pointing (and the visible hypocrisy of many of these companies who continue to allow abusive and biased studies), there is a good deal of truth to this statement. Suppliers have been doing their darnedest to lead the horse to water, but they can’t make it drink. There is a veritable mountain of data, analysis, and guidance calling for change and specifically prescribing the shape of the new world.
Yet the core of the industry’s business—that which provides evergreen revenue streams for suppliers and sensitive KPI for clients— is still foundering. The changes we are seeing in markets, media consumption, behavior, and survey participation are now creating big headaches for tracking brand awareness, market share, competitive positioning, advertising effectiveness, satisfaction, and many other indicators that companies use to evaluate performance over time.
What suppliers have done well is to tell clients what their new research projects should look like. What they haven’t done, though, is help their clients successfully manage the transition. This isn’t easy work, nor is it a common skillset. There are very few people in the industry who have personally managed a change of this magnitude.  Trackers and normed studies will typically have at least six figures’ worth of corporate decisions riding on them. Bigger ones can have an impact two or three orders of magnitude greater. Then there are the personal implications in terms of performance assessments, and thus reputation and remuneration. There’s a lot at stake.
<h2>Our e-book on Tracker Transformation</h2>
Changing trackers and normed studies is challenging, but it is very do-able with the right preparation and plan. To help corporate researchers and their consultants manage this change so their brands continue to measure their markets and performance accurately, we’ve written an e-book entitled <a href="https://aytm.com/blog/research-junction/managing-change-trackers-normed-studies/">Tracker Transformation: Managing Change in Trackers and Normed Studies</a>.
The book has five chapters:
Chapter 1 digs into the seismic changes in consumer behavior that, in addition to disrupting global markets, have created such turbulence for tracking studies. This chapter also confronts the major organizational obstacles that are preventing research buyers and suppliers from embracing change. We make the case for change by asking four main questions:
<ul>
<li>Are we measuring the right thing?</li>
<li>Are we talking to the right people?</li>
<li>Are our methods and techniques fit for purpose?</li>
<li>Are we managing these studies efficiently?</li>
</ul>
Chapter 2 serves up a proven project management framework and steps to follow to ensure success. Change is possible, but it requires a plan that is pursued with diligence and care. We explain a five-step plan:
<ul>
<li>Phase 1: Investigation</li>
<li>Phase 2: Design</li>
<li>Phase 3: Testing</li>
<li>Phase 4: Analysis</li>
<li>Phase 5: Implementation</li>
</ul>
Chapter 3 speaks to the methodology issues one must confront when contemplating change for studies where being consistently wrong is sometimes preferable to being right. This chapter explores the questionnaire and sampling issues arising from the mounting evidence indicating that, despite consistent fielding practices, trackers are going off the rails due to changing respondent behavior. In this chapter we provide researchers with four different lenses through which they can contemplate questionnaire or sample changes.
<ul>
<li>Does the tracker need to be shorter?</li>
<li>Does it need to better reflect the market?</li>
<li>Do respondents understand and complete the survey as expected?</li>
<li>What sample size do we need to report at, and how does this impact frequency of tracker waves?</li>
</ul>
Chapter 4 offers advice for research buyers on preparing their organizations for change. As unsexy as tracking and normed studies are, the prospect of changing them raises anxiety levels and sets off alarms all over the organizational chart. Without empathy and constant attention, all will be lost. Managing the optics is a necessary—and very difficult—condition for success.
Chapter 5 addresses a variety of topics that don’t quite fit elsewhere. These are the big “but what about …?” questions that clients continue to ask, to which suppliers have remained largely silent. The book concludes with a wrap-up and some thoughts on the future of tracking studies and how they fit in the constellation of insights.
One final note: throughout this book we use the terms “tracker” or “tracking study” as a shortcut for all manner of studies which, through unchanged design and consistent execution, ensure comparability against previous periods or norms over time. Put differently, if the prospect of a trend break or shifting norms makes you nervous, this e-book is for you. Thanks for reading.
Download the e-book at AYTM: <a href="https://aytm.com/blog/research-junction/managing-change-trackers-normed-studies/">https://aytm.com/blog/research-junction/managing-change-trackers-normed-studies/</a>
&nbsp;

Tracker Transformation

<a href="https://wp.greenbook.org/mr/wp-content/uploads/2015/12/put-up-or-shut-up.jpg"><img decoding="async" class="size-full wp-image-15332 aligncenter" src="https://wp.greenbook.org/mr/wp-content/uploads/2015/12/put-up-or-shut-up.jpg" alt="put up or shut up" width="800" height="533" /></a>
&nbsp;
By JD Deitch (@JDDeitch) 
The entire research industry just took a thunderous body blow.
<a href="http://www.nytimes.com/2015/11/28/upshot/online-polls-are-rising-so-are-concerns-about-their-results.html">This article in the New York Times</a> explains the issue with minimal jargon and an implication that’s clear as a bell: there’s a big problem in polling and, by implication, in online research. It’s no longer an “<a href="http://greenbookblog.org/?s=quality">internal discussion</a>,” nor is it just a <a href="http://www.britishelectionstudy.com/bes-resources/why-did-the-polls-go-wrong-by-jon-mellon-and-chris-prosser/#.VmCHX_mDGko">British problem</a>. It’s mainstream news.
Do we all understand how serious this is? Do we under understand that the majority of people who read that article will conclude benignly (or self-servingly) that online polls are crap?
We’ve been wringing our hands for years and now the chickens have come home to roost. It’s time for the all of us—research buyers, research agencies, sample companies—to stop blaming each other and start taking responsibility for what is now an industry crisis.
<h1>This isn’t news</h1>
Bias and sampling issues are part and parcel of survey research. Properly trained researchers are indoctrinated in these issues and, if my own experience is any measure, spend their professional lifetimes trying to find ways of managing and mitigating these concerns. Steve Coffey, long-time innovator at the NPD Group and former Chairman of the ARF (and my mentor), had a saying that sticks with me to this day: you have to be a grownup to do survey research. There are real trade offs. Technology and behaviors change, sometimes dramatically, as they have over the past several years in the always-connected smartphone age.
<blockquote>There’s far greater diversity of methods … [that reflect] the still-developing science of Internet polling. No one is sure of the right way to do it.
Nate Cohn, “Online Polls Are Rising. And So Are Concerns About Their Results.” New York Times, November 27, 2005. <a href="http://www.nytimes.com/2015/11/28/upshot/online-polls-are-rising-so-are-concerns-about-their-results.html">http://www.nytimes.com/2015/11/28/upshot/online-polls-are-rising-so-are-concerns-about-their-results.html</a></blockquote>
In the modern era, concerns about panel quality are hardly new. But they are now boiling over into the public domain. Yet <a href="http://greenbookblog.org/2015/12/07/grit-report-for-q3-q4-2015-is-here-download-it-now/">as the most recent GRIT study indicates</a>, the industry is still a long way off from being mobile ready despite <a href="http://www.research-live.com/news/news-headlines/smartphone-ready-respondents-find-research-industry-unprepared/4007515.article">YEARS</a> of <a href="http://newmr.org/blog/do-you-understand-device-agnostic-research/">information</a> and <a href="http://www.pewinternet.org/2015/04/01/us-smartphone-use-in-2015/">data</a> and <a href="http://www.economist.com/news/briefing/21645131-smartphone-defining-technology-age-truly-personal-computer">analysis</a> and <a href="http://greenbookblog.org/2015/10/12/6-no-nonsense-tips-for-mobile-market-research/">commentary</a> <a href="http://greenbookblog.org/2014/12/02/disrupting-market-research-an-update/">calling</a> <a href="http://greenbookblog.org/2015/05/14/the-future-of-panels-five-investments-needed-to-improve-participation-and-reduce-bias/">for change</a>.
Research buyers are slow to change. It’s hard work, with increasingly smaller budgets, and they are struggling to discern the signal from the noise among substitute and complementary methodologies. Traditional suppliers, whose fees are a direct function of length of interview and who count on annual revenue streams from <a href="https://www.ama.org/publications/MarketingNews/Pages/your-brand-tracker-is-on-the-cusp-of-death.aspx">inflexible tracking and normed studies</a>, are still trying to stave off financial challenges (rather unsuccessfully) to buy time to retool. Panels are starved of respondents under the age of 35 because none of them wants to take long punishing desktop-only surveys, which as we’ve known for years cause them to <a href="http://issuu.com/casro/docs/casro_2014_online_journal/35?e=0">disengage and respond with bogus data</a>. These aren’t fly-by-night companies either. These are industry leaders. Meanwhile, the combination of new tools and perceptions (right or wrong) of the inadequacy of survey research is driving down willingness to spend.
It’s a terrible vicious circle that has all the elements of a prisoner’s dilemma.
<h1>It doesn’t have to be this way</h1>
Blame for the industry’s problems can be placed at every point in the chain. <a href="http://greenbookblog.org/2015/09/10/are-diy-platforms-ready-for-prime-time/">But it doesn’t have to be this way</a>.
It is an indisputable truth that survey research can be done far more inexpensively, far more quickly, and far more efficiently with online platforms. These platforms are device agnostic and mobile friendly by design. They are automated and don’t require a slow costly army of people to run.
It is an indisputable truth that shorter, well written surveys still yield actionable information and meaningfully reduce “bad behavior” whether in the form of real deliberate fraud or the far more frequent inattentiveness for which researchers only have themselves to blame.
It is also an indisputable truth that people who aren’t subjected to long, poorly written desktop-only surveys are better responders who complete more surveys more accurately on average in their “lifetimes”.
It is also an indisputable truth that the technology exists to automatically test and retest people’s answers, and to use analytics and game mechanics to reward accurate responders, to not over-invite them, and to eliminate those over time who give suspicious answers, to treat panelists fairly and reward them for their rich profiles while maintaining their privacy.
<h1>Taking responsibility</h1>
There’s really no reason for the current situation to persist. It no longer requires management courage to reach the conclusion that the time to change is now. If anything, those who aren’t articulating concrete plans should be challenged by their employers, boards, and shareholders. So let’s do that.
Buyers, sample companies, research agencies, listen up.
Research buyers, if you’re still running long desktop-only studies, you are a fundamental cause of this problem. Blaming your suppliers for the quality of their panelists is like blaming the bartender for your hangover. I get that the change is difficult, but unless you really don’t care about people under 35 or moms with kids or ethnic minorities, you’re increasingly buying junk. This has to be part of the 2016 plan.
Sample companies, really it’s time to stop. The education you’ve done on the conference circuit is important, as are your tools and metrics to help your agency or end clients adjust to the new normal. But if you’re still taking the work that’s killing your panels then you are part of this problem.
It is even greater hypocrisy to engage in the gratuitous theater of parading around live respondents who talk about cruising mindlessly through surveys and even deliberately lying in qualifying questions when you don’t even follow your own guidelines about when to let in mobile respondents. You know who you are. You know what the solution is to this.
Research agencies, please actually continue doing what you’re doing. Your clients are starting to abandon ship in numbers, and they’re coming to me and companies like mine for a better and cheaper solution. Let’s be equally clear: we’re getting better data because people aren’t tuning out and we aren’t missing big chunks of the population. Your lunch is delicious. Thanks for sharing.
<h1>A Challenge to the Industry</h1>
There are many things happening in and around the research industry that are out of anyone’s control. The pace of technological change is breathtaking. New sources of data and new methodologies are changing the way we think about the behavior factors that influence it. It’s a fantastic time to be working in this space.
We can, however, control our own actions. It is time for us collectively to “walk the walk.” We need to follow the principles we espouse even if they are commercially inconvenient.
<ul>
<li>We will be device agnostic and optimized for mobile by design.</li>
<li>We won’t subject research participants, ours or anyone else’s, to long crappy desktop-only surveys.</li>
<li>We will talk to clients and research users in plain language about what surveys are good at and what they aren’t and how that impacts their studies.</li>
</ul>
Consider the gauntlet thrown down. Over to you, industry.

Market Research: It’s Time To Put Up Or Shut Up

<a href="https://wp.greenbook.org/mr/wp-content/uploads/2015/09/ESOMAR-Congress.png"><img decoding="async" class="wp-image-14939 aligncenter" src="https://wp.greenbook.org/mr/wp-content/uploads/2015/09/ESOMAR-Congress.png" alt="ESOMAR-Congress" width="596" height="253" /></a>
&nbsp;
By JD Deitch
Day 2 of the ESOMAR Congress was as good, perhaps even better, <a href="http://greenbookblog.org/2015/09/29/highlights-from-the-2015-esomar-congress-day-1/" target="_blank">than day 1</a>. Here are my highlights.
<h1>Tuesday</h1>
Outrunning cancer with Vanessa Oshima: For those who don’t know her story, Vanessa Oshima (Coke’s VP Corporate Strategy &amp; Planning in Japan) has been running at least 5 kilometers per day every day for over three years to support and inspire her friend Caroline who was diagnosed with breast cancer. When the tweet went out before the Congress from Vanessa asking attendees to participate, I was hesitant. Market research conferences can be arduous, and six o’clock alarms don’t really work well with the usual evening festivities. Nevertheless, I threw my running shoes in the suitcase. I’m glad I did. She told us that was day 1113.
As I mentioned in my first post on this year’s Congress, I find ESOMAR a place where I never fail to serendipitously meet fascinating people. I met many on that run, including Vanessa, who is an amazing woman doing an amazing thing. I encourage you to <a href="http://everydayhero.com/posts/540408">learn more about Vanessa’s story and donate generously</a>.
Wow, You Do Research Where?: This session was a good as I had hoped it would be. The speakers were warm, passionate, intelligent, and funny as they told stories of their experience that gave those of us who operate mainly in developed markets a clear indication that the “issues” we normally face are distinctly first-world problems! From Marita Schimpl conducting interviews in wood huts in Myanmar to Fatima el-Khatib overcoming the strict codes of Saudi Arabia, we were given a very different and inspiring perspective of a very cool part of our industry.
The Path to Purchase: Stephen Needel did his best to kill the term “path to purchase” for its models that were overly complex and dissociated from outcomes. He proposed a return to a classic social psych model called the Theory of Reasoned Action (<a href="http://people.umass.edu/aizen/f&amp;a1975.html">Fishbein &amp; Ajzen</a>, 1975), which specifically allows for things like perception, beliefs, norms, and persuasion. It was smart stuff, interjected with a bit of levity that somehow involved a discussion of toilet paper consumption.
Hooked on Shopping: AOL’s Vicki Draper and Andrew Consky led off this truly revelatory session continuing the toilet metaphor when they shared that 51% of millennials indicate they have made a purchase while on the porcelain throne. On a more serious note, their data showed something vitally important. Online shopping meets all of our human emotional need states. With technology that provides us a limitless opportunity to shop, it’s no wonder this is happening. Their segmentation should stimulate marketers to rethink their content strategies and online retailers to pay far more attention to the emotional element of merchandising.
You Call It a Snack? Brett Ao, one of ESOMAR’s Young Researcher of the Year nominees, shared his research which, founded upon an understanding of Chinese culture, perceptions, and practices, helped a European snack brand position its product and design packaging in ways that would resonate with Chinese consumers. Brett was poised, genial, and confident, and his work was of high quality.
Less is More: Vanessa Oshima (The Coca-Cola Company) and Dave McCaughan (BIBLIOSEXUAL) reminded us how easy it is to fall back on tried-and-true methods which, when we turn to them reflexively, cripple our creativity. Using the simple idea of deprivation, in one case forcing an interviewer to not ask questions and just respond with nonverbal cues, they found that participants ended up creating their own narrative that led to far deeper understanding.
Client Talk Show/Brand Tracking Revelations: Whether you’re a research supplier or a client, few assignments are more perilous, more anxiety inducing, more politically charged and contentious, than changing trackers. This sort of change requires having adult discussions about tradeoffs that, frankly, few on either side are able or willing to have. It was thus refreshing to hear mature perspectives from the panel (like Alexis Vernon from Barclay’s) as they reflected on the evolution of their tracking programs in the digital age. Reed Cundiff from Microsoft perfectly encapsulated a key challenge when he said data democratization is here, and researchers have to focus on shaping the discussions with their internal stakeholders.
One aspect about this conversation that disappointed me was listening to them talk like they could wait on becoming mobile first and device agnostic. Having been through these changes as a client and a supplier in some very difficult contexts, I get that this sort of evolution is really hard, especially in big companies where compensation is tied to tracking results. It effectively corrupts any insight aimed at people under 35 and dramatically reduces sample availability. I got the sense that Orange, France’s largest telecoms operator, was lagging on this and I was taken aback. I’ve said this elsewhere: it’s not suppliers who are forcing this change, it’s consumers themselves. We don’t have a year or two to wait.
The Awards Dinner! I can’t say it any better than Lightspeed GMI’s Jon Puleston:
<blockquote>
<div class="stream-item-header"><a class="account-group js-account-group js-action-profile js-user-profile-link js-nav" href="https://twitter.com/jonpuleston" data-user-id="58780833"><img decoding="async" class="avatar js-action-profile-avatar" src="https://pbs.twimg.com/profile_images/1185496618/Jon_Puleston12_bigger.jpg" alt="" />Jon Puleston ‏<s>@</s>jonpuleston </a> <a class="tweet-timestamp js-permalink js-nav js-tooltip" href="https://twitter.com/jonpuleston/status/648890763602956289" data-original-title="12:03 PM - 29 Sep 2015">Sep 29</a></div>
50 submissions to the <a class="twitter-hashtag pretty-link js-nav" dir="ltr" href="https://twitter.com/hashtag/ESOMAR?src=hash" data-query-source="hashtag_click"><s>#</s>ESOMAR</a> research effectiveness awards! Hey I&#8217;d love to hear about all 50 of them great to see research in practice
</blockquote>
<h1>Wrapping Up</h1>
I missed a couple of sessions at the end of the day as I had a variety of meetings. And clearly my view here is limited by the fact that I couldn’t be in two places in one time. I have heard from others I spoke to throughout the two days that there other really good sessions. Have a look at Kristin Luck’s timeline on Twitter, for example. She seemed to catch more of the career and young researcher sessions specifically. It’s also worth perusing the tweets with the #ESOMAR hashtag.
Regrettably, I missed some good content on Day 3 but I needed to take a midday flight. But upon reflection over these past days, I am left with these parting thoughts:
I confirmed my theory of serendipity. I have not been to any other market research conference where I have met so many interesting, talented, diverse, and genuinely nice people. It really is a pleasure to be there.
ESOMAR is a great place to see important and relevant work. Kudos to Simon Chadwick and the organizing committee as well as Finn Raben and the ESOMAR team for making this the best of the recent ESOMAR Congress’ I’ve been to. The entire event was very well done, and Dublin was fantastic.
Multiple methods are the only way to build competitive advantage. In a world awash in data, the most thought-provoking and meaningful presentations in terms of business impact were those that sought to break free of the reflex to use rigid quantitative approaches to seek greater understanding and consumer affinity. It’s hard to do this. It’s probably a bit longer and costlier as well. But I believe fundamentally that this will distinguish winners from losers.
Hope to see you all next year!

Highlights from the 2015 ESOMAR Congress Day 2

<a href="https://wp.greenbook.org/mr/wp-content/uploads/2015/09/ESOMAR-Congress.png"><img decoding="async" class="alignnone wp-image-14939" src="https://wp.greenbook.org/mr/wp-content/uploads/2015/09/ESOMAR-Congress.png" alt="ESOMAR-Congress" width="577" height="245" /></a>
&nbsp;
By JD Deitch
Day 1 of the ESOMAR Congress is in the books. The conference is off to a great start. Here are my highlights from the past twenty-four hours.
<h1>Sunday</h1>
Survey Design for Mobile Devices: SSI’s Pete Cape and Jackie Lorch covered this important topic well, although it was poorly attended (only about 30 people). The added benefit was Pete’s thorough review on how to think about scale questions.
Initial networking and cocktail hour: The cocktail hour is the unofficial conference kickoff. It was great to socialize and catch up with colleagues and friends on the exhibition floor.
<h1>Monday</h1>
Warm-up act! Magician Andy James, Irish entertainer of the year, set money on fire and levitated a table. What a great way to warm up an audience!
Welcome to Ireland: Ireland put its best foot forward. Finn Raben, ESOMAR’s Director General, and Richard Colwell, ESOMAR’s representative for Ireland, showed videos about their country’s proud history. Anyone who has ever visited Ireland can speak to the warmth of the Irish people and the beauty of the land. All conference planners should take a leaf out of ESOMAR’s book and spend time introducing us to the places where we’ll be spending the next 2-3 days.
Two great keynotes, two strong women: Kudos to the Committee for selecting not one but two accomplished women as our lead speakers. Ireland’s Deputy Prime Minister (Tánaiste), Joan Barton, spoke about the importance of our industry influencing the political and social landscape. This was a wonderful and impassioned speech that evoked the need for ethical research and referenced Europe’s current migration crisis. The second was Susan HayesCulleton, the Positive Economist, who talked about the micro and macroeconomic impact of market research and showed how unequivocally powerful our industry is.
Fireside Chat with Unilever: Stan Sthanunathan, Unilever’s head of Consumer and Market Insights and his colleague BV Pradeep spoke about Unilever’s changing approach to insights. He believes it’s possible to do things better, faster, and cheaper. I agree.
Business impact / stories from the front lines: These were great presentations about insights being used in the real world where emotion was an equal partner of reason. The presentation I had circled before the conference about a research team’s decision to go “all in” devoting their marketing budget and putting their careers on the line using a single piece of research was good. Namita Mediratta from Unilever spoke of their work to change perceptions of an ice cream brand. They were indeed success and turned the brand around using an appeal that was based on evolving cultural norms in different countries.
The best of this bunch, though, was from beverage brand giant Diageo. They believe that consumers don’t relate to brands. Rather what’s really important is for brands to be authentic champions of things consumers actually care about. Using cultural insights and semiotics they gave numerous examples of how they have evolved across their portfolio. Most impressive was their discussion about the Snapp brand, an alcoholic apple-flavored drink targeted to women in Africa, which was the most successful launch in Diageo’s history. Their creative was really excellent.
Reliabilty and Predictive Validity in Neuroscience: Neuro is one of the most fascinating areas in insights today. From eye tracking to galvanic skin response to brainwaves, researchers are studying people’s biological response to advertising. Michael Smith from Nielsen’s presentation went right to the point: the science works. Tests show highly reproducible results and are stable even with small sample sizes. Best of all, he showed conclusively that ads that engage the brain are correlated with increased sales. The cherry on top was the finding that segments of television that are neurologically powerful are more highly shared on social media.
Stay tuned for day 2!

Highlights from the 2015 ESOMAR Congress Day 1

<a href="https://wp.greenbook.org/mr/wp-content/uploads/2015/09/ESOMAR-Congress.png"><img decoding="async" class="alignnone wp-image-14939" src="https://wp.greenbook.org/mr/wp-content/uploads/2015/09/ESOMAR-Congress.png" alt="ESOMAR-Congress" width="598" height="254" /></a>
&nbsp;
<blockquote>Editor&#8217;s Note: The 2015 ESOMAR Annual Congress begins this Sunday, September 27th in Dublin, Ireland. JD Deitch, (twitter:@JDDeitch) frequent contributor to GreenBook and COO of Ask Your Target Market sets up the event with this article and will be providing daily updates.</blockquote>
&nbsp;
By JD Deitch
The Annual ESOMAR Congress is always an event to look forward to on the conference circuit. The event is well-attended, content and networking are of high quality, and the locations are always terrific. Here’s what I’m looking forward to at the 2015 installment.
<h3>1: The Networking</h3>
ESOMAR brings together a broad swath of the industry in terms of size and geographic reach. Big research suppliers—global by nature and naturally well represented in Europe (which is what the E in ESOMAR stands for)—are always there in force. While it’s not “the” event to see clients, big clients are always represented. This year’s attendees list includes recognizable CPG, beverage, media, and fashion brands, among others from all over the globe. Finally, while it can sometimes be a budgetary stretch, many medium-sized companies (clients and suppliers) come as well.
What this means is that the opportunity for expanding one’s horizons is greater than many other conferences. Personally, I have found ESOMAR to be a great conferences for serendipitous connections.
<h3>2: The Exhibition Floor</h3>
I love to stroll through the exhibition floor at a Conference as it says a lot about the firmographics of the event. It also helps me stay up-to-date on what my suppliers and competitors are doing. By virtue of its broad reach, the ESOMAR Congress delivers on this front as well with a great global mix of big and not-so-big providers. I find it particularly interesting to talk to the smaller companies who are spending a meaningful part of their marketing budget for a dedicated 3 square meters of space. There’s a reason they’re there, and it’s interesting to hear their stories.
<h3>3: Good Content</h3>
ESOMAR’s content is always a broad and balanced mix. Specialists beware: this isn’t the show to go to if you are trying to get real lay of the land of what’s new in the industry. IIEX holds that distinction for me. Nor is it the place to go for a more detailed look at methodology, as ARF and CASRO do this better. But there is something of interest for everyone, which is a credit to the organizers especially. This year’s event blends both new and traditional approaches from methodology to application.
Here are some sessions that look promising to me:
<h4>Sunday (Workshop day)</h4>
<ul>
<li>Survey Design for Mobile Devices (3pm) – presented for free by SSI. This is a must for anyone who writes surveys or designs survey research who has been living in isolation for the past two years or otherwise has ignored the bell that’s tolling for long, punishing, non-smartphone friendly questionnaires. SSI doesn’t have any particular insight into this topic that distinguishes it from other panel and research companies. They do, however, have motive. Regardless of their interest, the hands-on opportunity to get free advice from a smart researcher (Pete Cape) is worth it.</li>
</ul>
<h4>Monday (Day 1)</h4>
<ul>
<li>The Chance of a Lifetime (2:40pm Room 1) – presented by Namita Mediratta from Unilever UK’s market intelligence group. A client side researcher puts her career on the line with one piece of research? I hope the presentation is as good as its abstract. If it is, then this is a must for all research suppliers. You can’t possibly hope to deliver anything meaningful without empathizing with what your client is really going through.</li>
<li>Reliability and Predictive Validity in Consumer Neuroscience (3:50pm Room 1) – presented by Michael Smith from Nielsen’s Consumer Neuro group. Neuroscience is one of the most fascinating new branches of marketing science out there. Its detractors, however, point out that typical sample sizes raise issues of generalizability, reproducibility, and explanatory power. This is the first work I’ve seen that addresses these issues head-on. If Michael has, as his abstract states, found that neuro results are reliable and predictive, then it would be a huge affirmation of this technique.</li>
<li>Leveraging Passively Monitored Communities for Ongoing Insight (4:10pm Room 2) – presented by Chad Maxwell and Dave Choate from Starcom Mediavest. I like the look of this for two reasons. First, it’s great to see an advertising agency take on research as they’re going to be laser-focused on results without the fluff. Second, they’re blending data sources (survey and social), which I believe is going to define the industry in years to come.</li>
</ul>
<h4>Tuesday (Day 2)</h4>
<ul>
<li>Wow, You Do Research WHERE? (10am Room 1) – a panel moderated by Jon Puleston from Lightspeed GMI. I’ll confess that I’m jealous of Jon’s spot on this panel. Anyone who tends to focus on the highly-developed world should attend this just on general principle. I’ve had the opportunity to work in both big and small markets and it has made me indisputably better as a researcher and a business partner. The origins of the participants on the panel should make this a can’t-miss conversation.</li>
<li>Hooked on Shopping (11:45am Room 1) – presented by AOL UK, USA, and Canada. The folks at AOL observe that people are, in effect, constantly shopping. Their work aims to uncover these motivations from which advertisers can develop creative and editorial content. It doesn’t sound terribly new, but I am curious about the underlying motivations, and the list of presenters suggests a multinational focus. But there’s a time conflict with the presentation below.</li>
<li>You Call it a Snack? (11:55am Room 2) – presented by Brett Ao, Labbrand, a candidate for Young Researcher of the Year. A European entrant in the Chinese food market tries to understand the country’s rich diversity while maintaining brand authenticity? This sounds like a “boil the ocean” type of project, but I suspect it’s on the Young Researcher of the Year shortlist because Brett has been able to find clarity for the client. There’s a time conflict here with the AOL shopping session above though. Right now I’m leaning toward Brett’s session.</li>
<li>Brand Tracking Revelations (2pm) – roundtable moderated by Infotools. If there is one body of work that is in desperate need of overhaul, it’s brand tracking studies. The traditional industry is hopelessly stuck in a vicious circle of economic self-interest that prevents meaningful change short of disaster. I’m really hopeful this is a provocative discussion and not warmed-up platitudes about disruption and change.</li>
<li>Insight to Action: Using Survey Data to Target Customers and Increase ROI Through Digital Media (5:05pm) – presented by TNS. TNS has gotten a lot of mileage out of the Jan Hofmeyr’s Conversion Model, but here they claim to have connected it to actual programmatic marketing operations. In case anyone has any doubt, this is what it means for research to create real value: it needs to be tied to execution. Firms competing for advertising business that can’t do this don’t have a chance.</li>
</ul>
<h3>4: Catching Up with Former Colleagues &amp; Friends</h3>
I genuinely like the research industry and the people who work in it, and there is no better time to reconnect with colleagues and friends than the nightly cocktail hours and festivities. After a long day of presentations and meetings, it’s really pleasant to catch up with people you haven’t seen in a while, trade stories, and just relax. We’re a group that knows how to have a good time and is honest enough to admit it, though I think it’s fair to say we’re all glad these shows don’t last more than a few days.
<h3>5: Dublin!</h3>
Long may ESOMAR reign for its selection of great destinations! The Congress always ends up on my short list for this alone. This is not an airport-hotel-in-a-nondescript-city show. Dublin adds its name to the list of fantastic ESOMAR destinations—Berlin, Istanbul, and Nice, the previous three—that any traveler would be happy to visit.
Resist the urge to go the cliché tourist route. The Guinness and Jameson’s factory tours are nice, but not where you should spend your limited time. Give Temple Bar, Dublin’s version of Bourbon Street, the swerve as well.
It’s a twenty minute walk to Trinity College and the eastern end of a huge concentration of great restaurants and pubs, Dublin Castle, its glorious cathedrals, and St Stephens Green. Dublin has a great food scene. As you might expect, you’ll be able to find top-notch Irish cooking, traditional and modern dishes, at all price points. If you’re seeking something more cosmopolitan, there are sushi bars, wine bars, a very good Mexican restaurant, and dozens of others. I typically triangulate through TripAdvisor. Email or tweet me if you’re looking for ideas or a companion!
Stay tuned to the GreenBook blog for daily updates. See you there!

Five Things to Look Forward to at the ESOMAR 2015 Congress

<a href="https://wp.greenbook.org/mr/wp-content/uploads/2015/09/prime-time.jpg"><img decoding="async" class="wp-image-14850 aligncenter" src="https://wp.greenbook.org/mr/wp-content/uploads/2015/09/prime-time.jpg" alt="prime-time" width="649" height="365" /></a>
&nbsp;
By JD Deitch, Ph.D.
In 2013 I attended the CASRO Online Research Conference (now CASRO Digital) in San Francisco. One of the sessions that caught my eye was a roundtable discussion on do-it-yourself (DIY) survey tools. The program notes set the stage as follows:
“Much has been written about the recent proliferation of companies offering DIY/Instant Research methodologies, which are typically inexpensive. Are these methodologies able to produce representative results? What conditions are necessary to use such offerings effectively?
Representing the traditional industry on the panel were two respected and well-known researchers. The two biggest DIY firms (then and now) were at the table represented by senior officers.
The typical firmographic composition of a CASRO event meant that the DIYers would be heavily outnumbered and ensured skepticism from the off. It was therefore hardly surprising that the discussion never got beyond topline rhetoric. The researchers (on the panel and in the crowd) observed that the existing DIY solutions didn’t offer quality controls or oversight that one got from full service companies, which thus intrinsically and greatly increased the likelihood that people would do bad research. That these tools were being deliberately targeted at nonresearchers was, to some, like giving kids booze and cigarettes; it was morally reprehensible and threatened to bring the whole industry into disrepute. The DIYers rather unapologetically replied that people were already making choices, and that these were neither spendthrifts nor the feeble-minded but rather some of the traditional industry’s best and most astute clients. There were a few questions from the audience that tried to strike more thoughtful chords, but there was little room for common ground.
Two very momentous years have passed. What used to be a tiny insurgency has become a <a href="http://greenbookblog.org/2014/12/02/disrupting-market-research-an-update/">viable and consequential sector of the market</a>. There are those who would view commercial success as the surest sign that self-service tools are ready for prime time. Are they though? Are they good enough? The answer is an emphatic yes.
<h3>Their tools are complete, integrated, automated, and fast.</h3>
While there’s variation in both form and function across the space, DIY solutions are built from the ground-up to be maximally automated and fast. The best are horizontally-integrated and offer an all-in-one package of questionnaire programming, sample supply, data processing, and reporting that just leaves the industry’s traditional solutions in the dust. They are cheaper to run and don’t require days or weeks to produce ugly PowerPoints. They are indisputably better.
<h3>Their question formats are better.</h3>
There. I said it. With much of the industry desperately late in transitioning to a mobile-first, device agnostic world, the respondent experience for most DIY tools is downright refreshing. Many of these companies started as technology plays and thus understood implicitly that mobile friendliness was nonnegotiable. With no installed base of trackers to worry about or existing factories that needed to be retooled, they faced neither of the major obstacles that traditional firms did/do. Most have spent time on user experience and understand its implications for data collection. Some go so far as to hard-code character and response option limits so as to prevent the sort of questionnaire design problems that even the traditional industry seems unable to regulate with its clients. Even the interfaces for the survey creator are simple and fluid. Are legacy firms making progress? Yes, their tools are getting better, but their execution remains <a href="https://www.ama.org/publications/MarketingNews/Pages/your-brand-tracker-is-on-the-cusp-of-death.aspx">mired in the massive number of projects that are still desktop only</a>.
<h3>They are tackling and solving sampling problems.</h3>
Yes, you read that correctly. DIY companies are leading the way into programmatic sample, and in doing so taking out needless cost, enabling greater diversity, and still using reputable sources. That the questionnaires are short, well designed, and device agnostic means even broader coverage of audiences that are <a href="http://greenbookblog.org/2015/05/14/the-future-of-panels-five-investments-needed-to-improve-participation-and-reduce-bias/">abandoning the traditional industry’s studies and panels</a>. Some have their own panels, which they’re protecting precisely by not fielding the long, boring, desktop-only surveys that plague the legacy industry.
<h3>They are educating…</h3>
Whether they’ve come to the conclusion for commercial reasons or out of a belief that they need better quality (probably both), DIY companies are acquiring research competence and putting it to work. They’re hiring people who know research. They’re offering pre-packaged question modules and sample design recommendations to address the areas of greatest risk.
<h3>…and are attracting conscientious clients.</h3>
Amongst the researchers at the CASRO conference, the prevailing wisdom was that the potential for harm was great, especially for nonresearchers. Implied in this concern was the belief that (a) DIY firms were going to prioritize business growth over quality and (b) their clients were lemmings blithely jumping off the cliff. My conversations with people across the industry—with clients, DIY firms and observers—suggest that this hasn’t happened. To the contrary, it appears that clients are not only conscious of issues, many are asking for help. This is driving DIY firms to adapt and offer consultation.
<h3>Is there still a quality risk?</h3>
Are there people still doing bad surveys on DIY platforms? For sure, the answer is yes. But two things are abundantly clear. First, either by design or through deliberate action, DIY companies are taking clear steps to ensure quality while retaining their attractive price and speed. Second, they have resolved the quality problems that continue to vex legacy research firms, namely the glacial transition of long, punishing desktop-only surveys that are stressing panels and failing to cover a large swath of very desirable respondents. Put differently, there are plenty of quality risks that remain in working with traditional firms.
<h3>All grown up</h3>
In my <a href="http://www.research-live.com/features/big-bang-disruption-%E2%80%93-and-five-keys-to-survival/4011192.article">original article on disruption in the insights industry</a>, I stated that the winners of the revolution that continues to sweep through the sector would have a few central characteristics. They would make their clients’ money by getting closer to their execution. They’d build new assets. They would breed quality by design. They would ooze efficiency. Finally, as a result of these factors, they’d be able to create a new model to achieve profitability.
As the saying goes, I now have a horse in this race. After many years in the traditional industry, I’ve joined <a href="http://aytm.com">one of these DIY companies</a> because I think this is the future of research. Don’t take my word for it, though. See it with your own eyes. DIY firms are solving problems that research agencies are not, or are slow to solve. They’re starting to change the way people think about the impossible “research triangle” where, given the essential components of quality, speed, and cost, optimization is supposedly only possible across two. They face their own challenges about building and sustaining business, but they’re fundamentally doing what Clayton Christenson said in his seminal <a href="http://www.claytonchristensen.com/key-concepts/" target="_blank" rel="noopener noreferrer">article about disruption</a>. Because they’re not locked into legacy business models that militate against big change, they’re building the next generation and, in turn, leapfrogging the incumbents. They—we—are here to stay.

Are DIY Platforms Ready for Prime Time?

In the dark corners of the bar after a long day of conference presentations, back in prehistoric times before online panels but even as recently as pre-social media, conversations among researchers always turned to one subject: The disasters that had befallen each of us. As a former colleague once quipped, when you’ve worked in the business long enough, you know where all the bodies are buried.
Nowadays the conversations turn to a different subject: The trouble with panels. To be clear, nobody believes there’s a problem acquiring and retaining people who came of age before the Internet and time-shifted TV. The problem is everyone else. Scour the panel companies (or full-service shops with panels) for people under 35 or the parents with kids and you’ll find precious few. Racial and ethnic minorities, too. They aren’t joining traditional research panels any more.
The adoption en masse by these subpopulations of portable, always-connected devices is only a symptom of the industry’s pathology, one which device-agnostic surveys helps to alleviate.
The real disease, though, is that in general panels don’t offer compelling value to these people. The situation is critical in the U.S. and many countries in Western Europe (including the UK) and is becoming so elsewhere.
The impact of this crisis will be a real change in how sample is sourced, how much it costs to access real respondents, and on the future of panels themselves.
&nbsp;
<h3>Causes</h3>
The problem is easily summarized. Research participants who have even a modest interest in rewards quickly figure out that, to earn enough points, they have to take way too many surveys, most of which aren’t compatible with their preferred device or the amount of time they’re prepared to dedicate for meager nonfungible compensation.
Those who feel more altruistic or simply want to speak their mind have plenty of direct outlets to engage with their favorite brands or vendors. These outlets often provide recognition in the form of return contact. Likewise, most Internet polls show results immediately. Users get the instant gratification of knowing if their answers are “right”, or at least how others have responded. The research industry rarely offers this. Studies are blinded for fear of self-selection bias. Results are hidden, the confidential property of our clients.
If the slow and steady decline of panel participation is an infection, the industry’s response has been to continue to apply leeches. Acquisition techniques are evolving, but panel management and retention practices have remained static. Engagement is still largely transactional. Competition grows fiercer by the day. Attrition is a big problem.
In fairness, panel companies are caught in a Bermuda Triangle of sorts. They face a sea change in respondent behavior driven by constant connectedness, one they are powerless to control. Their clients understand the need for change but many are unprepared or slow to accept it. Their managers (or shareholders) are concerned about financial risk given the high levels of uncertainty. Like a drug-resistant disease, no existing medications will help.
&nbsp;
<h3>Building the panel of the future</h3>
Panels will decay into mere email lists (or social media “likes”) without new thinking and action around every aspect of the respondent lifecycle, from acquisition, to participation, to engagement and retention.
&nbsp;
<h4>1: Reach and diversity are essential</h4>
Having fished the same waters for too long, panel companies have been forced to look to other sources to expand the top of the recruitment funnel. One of first was incentivized networks, or the “river”. Without a doubt, river/non-panel sources will become more important, which requires rebooting the conventional wisdom for many researchers.
Managed properly, the only real difference between a panel respondent and a river respondent is that the latter has decided not to join a rewards community. This is from whence we come methodologically, yet there are clients who consider river respondents to be of lower quality than panelists when, methodologically, the sources are equivalent: neither can be linked back to the population with a known probability of selection. It is ironic that these same people ask what suppliers are doing to combat “professional” panelists. Nobody has found the Goldilocks respondent yet, though we’re all looking. At least in the short term, and probably for longer, the river will continue to supply audiences that are increasingly shunning panel.
Firms will need to invest heavily in making programmatic acquisition work as well. The challenge in a programmatic world is not so much the right demographic profiles, but finding those willing to participate. Buying on impressions (CPM) is an easy way to lose a lot of money if the people one acquires don’t take surveys. The ability to recruit richly at scale and evolve as the market evolves will be an asset that provides competitive advantage.
Implied in this greater attention to reach and diversity is the need for new measures of quality. Panel size and response rate are becoming meaningless. Both can be artificially manipulated in ways that do little to really help us understand bias. Response rate simply isn’t applicable in a non-panel environment. What matters now is feasibility across an ever-evolving panoply of sources. Reach and diversity are the only ways to increase feasibility and reduce coverage bias.
&nbsp;
<h4>2: Incentives and engagement must be meaningful</h4>
Reach and diversity help with the question of “who”, but not “why”. Panels are in desperate need of a new value proposition.
The easiest target for change is the proprietary incentive system, that is, those rewards that only have value through single panel participation. Common sense suggests these are of little interest to the audiences the industry needs the most. It seems safe to conclude that the level of participation required to earn a meaningful reward is too high. Companies that solve the incentive problem will allow respondents to accumulate points in multiple diverse ways and have fungible currencies. There are panel companies that do this already. Incentivized ad networks that form the backbone of many river sample offerings do this as well.
There are more interesting challenges on the incentives and engagement front, though. One is the impact that shorter, device-agnostic studies will have on retention. The burden on the panelist will be less onerous, but the incentive will be lower, too. This seems to still point to the need for transferable currencies.
Another is the possibility of differential incentives by respondent type. The only relationship in the value chain where payment is not a function of demand is between the panel and the general population respondent. Automation and access to data on costs by source could easily lead to highly personalized rewards by profile that improve participation, with the only caveat that companies will need to pay greater attention to fraud.
The third, and perhaps the toughest, challenge is providing the intangible engagement that is generally lacking in most panels. While panels have become more “social”, they don’t compete with other networks for stickiness. Whether it’s sharing how the data were used or permitting comparisons with a peer group, or offering interesting content, panel companies need to appeal to the latent feeling of “interestedness” respondents have that leads them to participate in the first place.
In his <a href="http://greenbookblog.org/2015/04/30/how-brands-really-grow-part-1-its-not-just-about-acquisition/">recent Greenbook article</a>, Jan Hofmeyr reminds marketers of the importance of working on both loyalty and acquisition. This applies in equal measure to panel companies. They need to acquire and reacquire/retain participants in ways that will force them to pay attention to—or to simply pay more for/to—the people they need.
&nbsp;
<h4>3: Suppliers need to stop putting up with abusive buyers</h4>
More has been written and said on the move to device agnostic questionnaires in the past year than any other topic related to sample.  Panel suppliers are shouting the loudest, exhorting researchers to move at a less-than-glacial pace to overhaul their designs.
Panel providers will, of necessity, start protecting their respondents. They will be able to measure survey pain by and construct “buyer ratings”, which, thanks to automation, will be easily calculable as a function of length of interview, device compatibility screenouts, abandon rate, and perhaps others. These pain measures will affect pricing as one would expect: buyers whose questionnaires fall short of the panel company’s standards will pay higher CPIs. At some level, suppliers will turn away revenue, either implicitly through pricing or explicitly by refusing to bid. This is happening already, and frankly it’s long overdue.
Similarly, companies will improve the routing experience, the process of shunting respondents hither and yon from study to study in search of a match, which still largely stinks. In part, it’s because the technology doesn’t move fast enough. In part, it’s because the industry has as yet been incapable of creating standards for passing basic demographic information, thereby requiring respondents to indicate their sex and age over, and over, and over. Many in the industry recognize the importance of standards. Work began at Samplecon and will continue at the Online Sampling Forum prior to the CASRO Technology Conference in late May.
&nbsp;
<h4>4: Respondents will be deeply and respectfully profiled</h4>
Companies that broker survey respondents continue to explore ways in which they can add value. Profiling is one nut that many hope to crack. From micro-surveys to data enrichment to analytics that enable suppliers to infer behavioral and attitudinal propensities from survey responses, there are many exciting prospects.
Having said this, it will surely become more and more difficult to collect profile information directly. The marketing and research industries will be forced to take a far more serious approach to privacy. Privacy is a throwaway thing for researchers. The fact that panel companies generally collect this information without meaningful compensation is proof. Researchers will need to carefully watch the trends in personal data security. Mounting concerns over privacy suggest a future in which, at minimum, there’s little trust for unknown data collectors.
More intriguing are the embryonic marketplaces for personal data, one of which explicitly speaks about cutting out the market research middlemen. Entrepreneurs are helping individuals become aware of the value of their personal information and facilitate its exchange. It’s too early to know whether this will take off, but it needs to be watched.
&nbsp;
<h4>5: Quality practices will need to evolve</h4>
Any research company of appreciable size has automated speeding and straightlining detection methods. These algorithms arose as reactions to clients’ punishing research designs. In a grid-less, device-agnostic world with shorter questionnaires, speeding and straightlining are no longer meaningful problems.
As the recent UK elections have proven, extant practices for ensuring representativity and response quality are insufficient. That the pollsters have suffered this massive a setback should strike fear into the hearts of other researchers, as the pollsters are generally more stringent in their methodology and write simpler, shorter questionnaires.
Panel companies can demonstrate additional value and potentially charge a premium by developing bias correction methods. Two avenues are worth exploring.
One is examining the relative consistency of responses. The technology exists to be able to automatically detect whether Panelist ABC’s responses are consistent with others of her demographic, attitudinal, or behavioral cohort and then decide what to do. Parameters could be set based on tolerance for some numerical deviation, at which point the respondent and his data would be eliminated or quarantined.
Another avenue of interest is asking questions that have gold-standard references. Standard practice is to weight data to correct for demographic imbalances, but more is possible. The essence of this type of bias correction would be to benchmark against gold standard data and calibrate responses to reduce bias.
&nbsp;
<h3>Cost and conclusions</h3>
<a href="http://greenbookblog.org/2014/12/02/disrupting-market-research-an-update/">I’ve written elsewhere</a> that technology will, in the short term, drive down the supplier’s costs by eliminating manual labor. It will expose market liquidity and reduce the information asymmetry between buyers and suppliers, creating greater pricing parity.
But make no mistake, the marketplace that panel companies and the industry have taken for granted is that of general population research participants. The implication of the above points is that the cost of securing an in-demand respondent is likely to rise. For people with no relationship with the participant, the cost will be direct and steep. For panel companies, the cost will be the investment needed to build compelling value. They will be forced to compete like brands for loyalty and interest, which for many will be an existential challenge. This is one reason why brand-based social communities are flourishing: they are able to capitalize on pre-existing affinity.
It’s not outside the realm of possibility that the market for sample will evolve like the market for bonds. Bond quality is inversely proportional to risk: the higher the quality, the lower the risk. The only hitch in the metaphor is that higher quality bonds have lower payouts down the road, while higher quality sample will cost more up front. As mentioned above, there is (or will be) enough information being automatically transmitted to permit this sort of calculation.
It’s impossible to predict how the factors that are driving sample challenges will resolve themselves. What’s sure is that the forces are inexorable. The worst assumption the industry can make is assuming the future looks like the past. Research companies of all stripes, private and public, need to understand that they are not facing marginal cost decisions. Their actions require new thinking upon which sustainable futures can be built.
Photo by <a href="https://unsplash.com/@morganhousel?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Morgan Housel</a> on <a href="https://unsplash.com/?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a>.

The Future of Panels: Five Investments Needed to Improve Participation and Reduce Bias

<div id="attachment_13717" style="width: 1071px" class="wp-caption aligncenter"><a href="https://wp.greenbook.org/mr/wp-content/uploads/2015/03/RIDLLR.jpg"><img aria-describedby="caption-attachment-13717" decoding="async" class=" wp-image-13717" src="https://wp.greenbook.org/mr/wp-content/uploads/2015/03/RIDLLR.jpg" alt="Tshirt worn by Chaordix CEO at IIeX in Philadelphia, 2013. Photo Courtesy of Nelson Davis. " width="1061" height="707" /></a>Tshirt worn by Chaordix CEO at IIeX in Philadelphia, 2013. Photo Courtesy of Nelson Davis.</div>
&nbsp;
By JD Deitch, PhD
Regular readers of Greenbook need no introduction to the companies or new techniques that are regularly on display at the Insight Innovation eXchange (IIeX) conferences. Their stories are told on this very blog week in and week out. Nevertheless, as with any good sporting or cultural event, there is something unmistakably better about being there, live.
IIeX is something special. Next to most other industry conferences, IIeX meetups are a breath of fresh air: invigorating, provocative, and unique. Thus it was in Amsterdam about two weeks ago.
<h2>Three takeaways</h2>
There are three big things I took away from an intense 48 hours of presentations and conversations.
First: research as it is done today—limited in approach, cumbersome in execution, and detached from how companies actually operate—is taking its last breaths.
The impassioned rhetoric of disruption, a fire whose flames I myself have fanned, is now totally mainstream. IIEX was my third conference already this year, and at each I’ve heard choruses of voices singing the same tune.
Amsterdam took that one step further. I thoroughly enjoyed watching Jeff Reynolds from LRW bash a giant nail in the coffin of how we approach brand tracking, and in doing so offer a new vision of doing it better, complete with a P&amp;L that was more attractive for both the agency and the client. (See my own similar piece in the March edition of Marketing News.)
Likewise when Eric Salama spoke about driving cultural change in the vast Kantar organization, there was unmistakable gravity in his frankness. My own employer, Ipsos, is actively engaged in the same process of renewal across the breadth of its business.
Second, the companies and techniques that are the vanguard of the “New MR” are starting to prove their worth. 
2015 will be the year that measuring the impact of social media and constant connectedness comes good.
The week before Amsterdam, at CASRO Digital in Nashville, I led a discussion during which my expert panelists agreed that research needed to become source agnostic. Joel Rubinson, one of the discussants, talked about his work showing the impact of social influence on purchasing. He argued forcefully for a better connection between research and clients’ execution, particularly with programmatic advertising. (Joel will be at IIeX Atlanta in June.)
Then, in Amsterdam, I chaired a track of presentations, two of which provided proof points about social media measurement. Fran Cassidy shared results of the IPASocialWorks/MRS project, through which they collected case studies to create guidelines and practices around measuring the impact and ROI of social media. Preriit Souda, ESOMAR’s 2011 Young Researcher of the Year and clearly one of the industry’s bright young stars, shared a multi-source study that blended survey and social media data to evaluate Scotland’s independence referendum.
Another area which, to my mind at least, is on the cusp of great discovery and impact is nonconscious measurement. I was impressed by Jeremy Sack’s (also from LRW) work on Identity Overlap driving brand affinity. I was blown away by Cristina Balanzó’s (Walnut Unlimited) presentation on subconscious human insights, in particular the stunning visuals from her company’s neuroscientific approaches to ad testing. Both speakers articulated cogent frameworks reflecting an undeniable maturity of thought.
And I’d be remiss if I didn’t speak to the rapidly evolving tools that make it easier to do good research. Whether it’s IIEX alum Zappistore blazing a trail for what I’ve come to call “ready to wear research”, or this year’s Insight Innovation Competition EU winner and mobile sample provider Dalia Research, or any of the myriad qualitative and text analytics solutions that just continue to get better, it’s a sure thing that we will see companies like these start to break through to bigger success in 2015.
Third, there is no better place to see the range of techniques and new ideas than IIeX. 
Our industry associations are visibly evolving to stay current amidst the sea of change, but they haven’t a patch on IIeX when it comes to providing a forum for the new. The presentation track I chaired was a microcosm of the range of techniques that, but for neuroscience, have captured the industry’s interest. In addition to Fran and Preritt, Mary Meehan (Panoramix) gave a very structured approach to studying broad cultural trends that reflected the best of what the industry aspires to when it talks about storytelling. Frank Kelly (Lightspeed GMI) spoke about the research they’ve done on using voice—both text-to-speech and speech recognition—in survey research which, one must believe (I do), will replace the keyboard in the future.
The conference has the right combination of tempo, curation, and people to create a very high signal to noise ratio in the sessions, on the exhibition floor, and especially in casual conversation.
<h2>Looking forward</h2>
IIeX is one of the only conferences that stirs my optimism in the industry. What makes it provocative and unique is that it gets beyond rhetoric to the real world. The techniques on exhibit are available now and are maturing rapidly. The demos are worth the time spent.
The rest of this year’s conferences will no doubt highlight the change underway in our industry. Talk of disruption will become the obligatory first slide of presentations in which companies who may or may not have new arrows in their quiver aim to convince listeners that they’re blazing new trails. IIeX is the real deal though. It is part-showcase, part-crucible for people trying new things explicitly designed to produce better research in front of an audience that is appreciative and imaginative.
Research is dead. Long live research.

IIeX EU 2015: Research Is Dead, Long Live Research

<a href="https://wp.greenbook.org/mr/wp-content/uploads/2015/02/Marissa-Mayer-Yahoo.jpg"><img decoding="async" class="alignnone size-full wp-image-13618" src="https://wp.greenbook.org/mr/wp-content/uploads/2015/02/Marissa-Mayer-Yahoo.jpg" alt="Marissa-Mayer-Yahoo" width="475" height="356" /></a>
&nbsp;
By Jonathan Deitch, Ph.D. 

<blockquote>This essay represents my personal point of view about the challenges we face as an industry that increasingly struggles to acquire its essential raw material: respondents. My belief is unambiguous: we are the problem, and we need to change to survive. We must re-imagine our offer, which will only happen when we understand what the online survey research experience looks like in today’s context from their point of view, from the moment they decide to spend time with us through survey and reward, from environmental context to screen size, from comprehension to duration. We need to re-engage them—and make them want to be engaged— in our journey. 
</blockquote>
The fast-paced multi-billion dollar world of internet search, advertising, and content is not easy to follow. Tools and practices change quickly. To continue to exist, companies must innovate routinely and quickly. If they don’t, someone else will. A stale experience means fewer eyeballs. When nobody returns, advertisers flee. When advertisers flee, there is less funding for innovation, and so on…
This is effectively what happened at Yahoo and what ultimately led to the appointment of Marissa Mayer. I’ve been following her appointment at Yahoo because, though our industries are very different, I believe we face essentially similar challenges.
<h1>The Rise and Decline of Yahoo</h1>
Not quite two decades ago, Yahoo started as a small site that aimed to “organize the Web”. This seemingly grandiose mission was possible in those days; the web was still small enough to manually curate. Websites were brochures and offered little to actually do. As a portal for discovery, Yahoo was the unequivocal leader in those ancient times.
A few years later, an upstart company began doing the same thing, except they had automated algorithms that served to weigh a site’s importance by the extent to which others thought it was valuable. They began to lure away Yahoo’s users as their results were seen as more relevant, complete, and even democratic. Yahoo’s importance dwindled, and Google rose into prominence.
History clearly shows who won this battle. Yahoo’s response to Google was insufficient. Google launched new products (like Gmail and Google maps, two products Marissa oversaw) while continuing to improve and capitalize on their core search product. They attracted more visitors, and with them more advertising dollars. Yahoo failed to innovate and steadily lost ground. Google’s market value went up. Yahoo’s went down. Over the last several years Yahoo has gone through one CEO after another, but none have been able to reverse the decline.
FIGURE 1: STOCK PRICES INDEXED. 
GOOGLE IS THE RED LINE. YAHOO IS THE BLUE LINE.
&nbsp;
<a href="https://wp.greenbook.org/mr/wp-content/uploads/2015/02/yahoostock1-e1424098803323.png"><img decoding="async" class="alignnone size-full wp-image-13625" src="https://wp.greenbook.org/mr/wp-content/uploads/2015/02/yahoostock1-e1424098803323.png" alt="yahoostock" width="1075" height="462" /></a>
&nbsp;
Enter Marissa Mayer. An early Google employee, Marissa’s strength was her ability to blend a real understanding of technology with strong vision about how to mold it into appealing products. Her success with Gmail and Maps made her one of the “superstars” of Internet business and Yahoo sought her out because of this. The fact the Yahoo Board chose her over more experienced general managers is all the proof we need to conclude that, to them, perspective mattered more than tenure.
While Marissa’s profile as a leader is thought-provoking, what interests me more are the parallels I see between her mission at Yahoo and ours in the MR industry. Let me be clear here: I don’t believe we have lost our way like Yahoo did. Nevertheless we face similar challenges, the biggest of which is our ability to captivate and retain our audiences.
<h1>Marissa’s Plans</h1>
So what is Marissa going to do to reverse Yahoo’s fortunes?
The trade press suggests she’s focusing on two things. One of them relates to key infrastructure; the other, to Yahoo’s users themselves.
Yahoo is losing out on advertising revenues in part because they lack top tools. For Yahoo to attract advertisers, they need to be able to deliver highly targeted ads that maximize ROI for all parties. With a new engine to run the business, they hope to revitalize the key driver of Yahoo’s revenue. Yahoo is also betting heavily on mobile. The conventional wisdom is that Yahoo has no mobile products that are on par with Google or Facebook. To think that this is simply an “app” problem misses the point.
For Yahoo, mobile means more than just shrinking content to a smartphone screen. It means understanding that the environmental context, needs, use cases, and methods of interaction exhibited by a person on the go are different from when s/he is sitting at a desk. To be successful at mobile for Yahoo means no less than reimagining their offer by taking their great content and delivering it to people in a way that’s useful, accessible, meaningful, and engaging. For Yahoo, mobile is neither the end nor the means. It’s a change of paradigm, a new way of thinking that is as fluid as the name implies. The endgame is to keep users coming back.
<h1>Parallels for MR</h1>
What Marissa is doing at Yahoo is exactly what we need to do ourselves.
We are in the middle of our own change in industry infrastructure. Sampling has evolved from the utilities of panels to being the heart of survey ecosystems and routers that are integrated with field and project management platforms. Technology is allowing us to not only leverage the latest in online research and sampling methodologies, but to also more efficiently manage the flow of respondents. These integrated platforms are becoming our new engine and a necessary but not sufficient component of our broad strategy.
What’s our analogy, then, to Yahoo’s quest to become more mobile?
Some would argue (I’m one of them) that we need to (quickly) become more mobile in the literal sense of the word.
Using the Expanded Definition by ESOMAR as the basis for calculation, online only approaches account for 55% of global spend at $33.5B. According to Comscore in 2014 60% of all internet traffic of any type originates on a mobile device. That means that $20.1B of our industry revenues are likely driven by mobile as well.
One simple thing “mobile” means for us is having a survey that displays properly on a small screen. The grid question format, one of our most widely-used (yet indisputably painful for respondents even on big screens), simply won’t work. Fortunately, everyone on the sample and data collection side of the business see this and are pushing to become mobile friendly. The work now lies with helping clients manage through this transition.
Mobile success requires more than new interfaces though. Marissa knows—and Yahoo hired her because—she understands that product success is first and foremost dependent upon understanding how users will engage with that product. With each revolutionary wave of how people engage with technology, Marissa’s challenge is to make sure Yahoo remains connected with its users and engages them with compelling and relevant offers that elegantly package content and context.
If Yahoo can’t make this work, it will never return to its former glory. Indeed Yahoo’s continued existence probably depends on her success. I think it’s no exaggeration to suggest that ours may very well depend on these same factors.
<h1>Unlocking Engagement</h1>
Our industry faces a very stark future unless we figure out how to reengage with our respondents.
Low response and high attrition rates are the symptoms. In an attempt to cure the disease, we developed new methodologies like routers and river sampling. While these give us greater reach and efficiency, they are mechanical solutions that still treat people as if they were an inexhaustible resource. To use an analogy, it’s like we’ve treated a water quality problem by simply opening the taps wider hoping more water will dilute the contaminants. We still have dirty water and we haven’t done anything to stop the pollution at its source.
To address the real cause we have to understand what our offer looks like from the perspective of our users. This means two things.
First, we must understand that our competition isn’t from other other research firm. It is Facebook, YouTube, Angry Birds, the television, the telephone, the kids, email, text messages, apps, the loud person next to us on the train, and so on. We ask for someone’s valuable time and their full attention when both of these things are in increasingly short supply. We need a better understanding of the constraints when we do this, and we need to change our expectations.
Second, we need to stop treating the respondent as a commodity and his actions as a one-off transaction. The answer here lies in trying to create relationships like those that exist in community panels. Community panels (we used to simply call them proprietary panels!) are making a big comeback in these days of extreme social connectedness. This approach leverages the natural affinity panelists have for the brand sponsoring the panel. Through new types of research-centered participation, people keep coming back. This is exactly what we as an industry need to do more of.
Imagine if our panelists were proud to be a part of the research process? Imagine that, through a satisfying survey experience and the feeling their time was well spent, the average panelist who might ordinarily only complete 5 surveys in her life gave us just one more? The benefits would be dramatic.
<h1>Our Actions</h1>
The implications of the above reasoning are manifold.
Our survey experience must change. It needs to be top drawer. It must be modern, interesting, and engaging. It shouldn’t be clunky or abusive or difficult to do. Rightly or wrongly, our panelists are comparing us to sites crafted by a corps of super-skilled designers, engineers and developers. They are the benchmark, and we need to put out a much better product, one that feels more welcoming than an application for a bank account.
We need to truly understand what our offer looks like through the eyes of our potential respondents. From the first ad they see enticing them to take our surveys, we then need to maximize the likelihood that they respond accurately and completely, whether they’re on their laptop, or on their smartphone on the bus, or on an iPad while watching television while instant messaging with a friend.
We need to think like brand managers. Retention isn’t just about incentives. Saatchi and Saatchi sell their clients on a vision that says if you give people an experience they love and you earn their respect, they will show you loyalty beyond reason. Is it ambitious to think respondents show us loyalty beyond reason? Maybe. But the fact is we have to aim for this peak. If we don’t, our fate is surely sealed.
We’ve taken the first step in this journey by hiring a senior leader to oversee our engagement efforts.
<h1>Conclusion</h1>
Marissa Mayer’s challenge is to change Yahoo: to transform it from being a “legacy” Internet company to a modern one, one where users can engage with the company however they want, wherever they want. This is an article of her faith. Yahoo’s existence depends on it.
Though our situation is not (yet) as dire as Yahoo’s, we face the same challenge. We need to transform our industry from being a “legacy” data provider to a modern one. We must reengage our respondents. We need to see the world from their eyes. We need to give them an experience that is more satisfying for them. We need to treat them and their time with respect. This is an article of my faith, and I believe our existence depends on it.

Monitoring Marissa: How MR And Yahoo! Face The Same Uncertain Future

<a href="https://wp.greenbook.org/mr/wp-content/uploads/2014/12/disruption-1.jpg"><img decoding="async" class="alignnone size-full wp-image-13254" src="https://wp.greenbook.org/mr/wp-content/uploads/2014/12/disruption-1.jpg" alt="disruption-1" width="550" height="302" /></a>
<blockquote>&nbsp;
Editor&#8217;s Note: Earlier this year JD Deitch wrote what I consider to be one of the best deconstructions of the systemic challenges facing the market research industry I have ever read. I was so impressed I asked him to come to IIeX in Atlanta to <a href="http://insightinnovation.org/blog/2014/08/13/big-bang-disruption-and-keys-to-survival/" target="_blank" rel="noopener noreferrer">present his thoughts as well</a>. Last week JD asked if I&#8217;d like to post a follow-up, and I jumped at the opportunity. As we close out 2014 and turn our eyes towards 2015, this should become a foundational POV to inform your strategy for the foreseeable future.</blockquote>
&nbsp;
Jonathan D. Deitch, Ph.D. (Connect on Twitter <a href="https://twitter.com/JDDeitch" target="_blank" rel="noopener noreferrer">@JDDeitch)</a>
About ten months ago I wrote an article entitled <a href="http://www.research-live.com/features/big-bang-disruption-%E2%80%93-and-five-keys-to-survival/4011192.article">Disrupting Market Research: Five Keys to Survival</a>. In it, I observed that, like most sectors, traditional market research is being disrupted by new techniques and environmental factors that pose an existential threat to laggard firms regardless of size. I concluded that, to merely survive, research companies needed to stop talking about value and start delivering it, and that this would only happen if they created game-changing assets and approaches that gave clients the tailored services they wanted.
My argument was predicated on changes in spending as data penetrates the business world, changes that have effectively ended the monopoly on budget and methodology traditional researchers once enjoyed. From 2010-2012, spending on traditional MR remained virtually flat, while spending on new techniques grew at a healthy double-digit rate.
With the recent release of 2013 estimates from ESOMAR (as estimated by Outsell) I thought I’d update the original article and reevaluate its themes. The data continue to point to the same conclusions. The industry is innovating and incumbents are facing stiffer headwinds. Worst of all, research participation is now at the edge of an abyss.
<h1>2013 Spending: Traditional MR still losing ground, online trends down</h1>
Spending on traditional MR in 2013 grew by a scant 2% compared to 8% growth in what ESOMAR calls the ‘expanded market’. Traditional MR thus continues to lose ground to new techniques at what is roughly the same pace as previous years. Moreover, the share of spending on online research declined by 1 point, meaning about $600 million shifted to other modes. ESOMAR attributes these factors to price competition in online research and competition from other methodologies, explanations which are entirely consistent with my observations in January’s article.
For sample and panel companies, one would expect similar topline headwinds as traditional MR companies are their core clients. Nevertheless, that sector grew by about 5%. There is evidence that sample/panel companies are diversifying revenue sources, e.g., by going directly to end clients or otherwise leveraging their technology investments.
Meanwhile, new sectors of the insights industry continue their strong growth. Online analytics, web and social media research, social media communities, and marketing reports and research grew by at least 10%. 2013 saw a very healthy 13% increase in the survey software sector as well, which seems a clear vindication of DIY.
<h1>2014: More of the same?</h1>
In my original article, I postulated that industry growth would top out when clients, recognizing they were spending ever more money on insights, would start making zero-sum decisions, i.e., they would redirect budget from traditional techniques to new methodologies. While the “legacy” MR industry may have staved off the worst in 2013, there are ill winds blowing in 2014.
<h2>Dark clouds for traditional MR firms</h2>
In July, two of the biggest traditional MR firms announced first-half revenue shortfalls and warned of full-year consequences. The capital markets delivered swift judgment. In October, one the biggest conglomerates showed a revenue decline in Q3 and warned of weaker sales to come. The weakness appears to be hitting hardest in mature markets.
While they don’t report their finances like publicly-quoted companies, the “upstarts” show no signs of flagging. The leaders of the insurgency might well be not one but two DIY (do-it-yourself) survey software companies, both of which have putative valuations of over $1 billion, making them worth more than all but the biggest players in the industry. At what point do they “acqui-hire” a full service firm for some additional research bona fides for clients who might need higher-level consulting?
<h2>Trends to watch: Research “off the rack”</h2>
Keeping with the DIY theme, for years now insights clients have been complaining about the haute couture world of full-service research with its high prices, slow delivery, and methodological ostentation. In response, researchers have argued back that DIY solutions are like buying factory seconds: you get what you pay for.
Enter “Ready-to-Wear” Research. A UK-based company offering services in Europe, North America, and Australia, now allows customers to buy, with a credit card, the industry’s leading intellectual property for concept testing, copy testing, and brand measurement (with other products coming soon) in explicit partnership with the IP owners. By buying “off the rack”, clients don’t get every bell and whistle, but they do get reliable results using proven techniques, faster and at a lower price, which is exactly what they have been saying they want. This hybrid approach—not exactly DIY but not exactly full-service—gets sample via an API from a large global sample company. The whole thing oozes efficiency, explicitly addresses the issue of quality, and appeals to clients’ own personal consumer experiences of getting just what they want while executing the transaction quickly and conveniently.
This company is not alone in its efforts to create a more templated solution. Many research companies are trying to find the sweet spot between their full service offerings and “lighter” designs that are faster and cheaper. The limiting factors are twofold. One is the operational challenge of simplifying. The other is the deep concern that the lighter product will cannibalize revenue.
<h2>Trends to watch: Automation</h2>
With sample and panel providers leading the way, the industry is now headed full speed into programmatic execution. From lower operations payrolls to highly efficient and cost-effective matching and delivery of respondents to studies, automation holds a lot of promise.
Suppliers are now offering pricing advantages for using self-service APIs instead of their human project managers. Automation may be both blessing and curse, though. In the longer term, automation must surely reduce the information asymmetry between sample buyers and suppliers (which currently favors suppliers) as it promises far greater price transparency and liquidity. This, in turn, should promote greater pricing parity and create both top- and bottom-line pressure. Of course there is more to sample than just price—respondent quality and a firm’s reliability are important considerations, too—but at some point this information will become equally available and get priced in.
On the acquisition side, panel builders continue to recruit in the fully automated and magnificently intricate digital media ecosystem. The process is entirely data-driven and managed in real-time: predictive models determine, in a matter of milliseconds, whether the person currently looking at the screen (whose demographics are inferred and appended from big data) is likely to be a panel joiner to determine whether they should bid for the ad spot and show creative that has been optimized for the body viewing the screen. The competition for “eyeballs” is fierce. The advertising and affiliate networks and co-registration marketing companies who have traditionally provided the industry’s panelists are also showing a willingness to experiment with new techniques to drive conversion.
<h2>Tipping Point: Smartphones and Young Respondents</h2>
Despite all the talk by both suppliers and end clients, and despite more than adequate research-on-research to provide guidelines, the industry is moving at a snail’s pace in its adoption of mobile-first, device-agnostic surveys—and it’s starting to materially affect feasibility in younger audiences.
Even a casual review of <a href="https://www.greenbook.org/market-research-companies/malaysia">research company</a> websites and apps provides ample evidence that there isn’t nearly enough inventory of smartphone-friendly completes. Users give middling ratings to mobile apps and remark that “there are never any surveys available”. In app, users still get diverted to the big-screen versions of surveys and websites. They watch as the “please wait” icons spin, only to finally deliver the frustrating news that the survey is closed.
The industry has reached tipping point particularly in the US, where somewhere between 30-50% of new panel recruits (perhaps more depending on the weight of mobile in the media plan) are joining from their smartphones. The consensus is that better than half, and perhaps as much as two-thirds, of all emails are opened on mobile devices, which is important as email remains the primary method of informing people about surveys. The numbers skew higher for those under 35 as smartphone penetration is higher.
This is a massive slice of the available universe, one that <a href="https://www.greenbook.org/market-research-firms/proprietary-panels">panel companies</a> have paid for and their clients would love to speak to, that is simply unavailable for projects that are not adapted to smartphones. Being tablet-friendly isn’t enough either: both the user experience and demographics are different. In the US at least, tablet penetration is around 40% and is skewed toward higher incomes, whereas smartphone penetration is 20 points higher and, importantly, shows no significant difference with racial minorities and less important differences by income.
This problem is of particular concern for trackers and normed studies. While one can understand their reluctance to change, it is no longer avoidable. There are only two sustainable options: they either (a) get shorter and adapt their data collection to be device agnostic or (b) they tolerate feasibility declines that lead to trend breaks and bias. Researchers and clients need to confront these issues so they can plan and act deliberately.
If the industry is fortunate, the need to adapt to smartphones will drive greater attention to the user experience and lead to shorter surveys, more reasonable designs, more focused questions, less cumbersome response mechanisms, and ultimately higher participation rates and better data. There really is no other alternative.
<h1>Five Keys to Survival: They Still Make Sense</h1>
In the context of the above developments in the industry, the Five Keys to Survival from my original article remain relevant.
<h2>1: Make money for clients</h2>
At the moment, this appears to be taking the form of leveraging technology and tools for greater speed. With data and analytics continuing to be built into execution, the need to undertake a distinct “research project” is shrinking. While this need won’t disappear, it is clear that understanding how clients execute and developing offers that fit their execution is essential.
<h2>2: Create new assets</h2>
At the moment, this is all about automation and tools. From “ready to wear” research to sample automation to horizontally-integrated field/project management platforms, insights firms are investing in technology that enables them to streamline their own execution to save time and money. If there is nothing else certain about the future of the industry, it is that it will be far less manual than it is today.
What remains to be seen still is whether firms will be able to develop new data assets either from the data itself or the analytics on top. A quick scan of the job boards shows a healthy market for data scientists, particularly in the new expanded sectors of the market. Yet while big data is interesting and necessary, it is not sufficient. Even Google is using survey data to inform its own models. It’s hard to see how the insights of the future would not be driven by a combination of techniques, both innovative and traditional.
Equally uncertain is the state of consumer panels in mature markets. The situation is especially disquieting among young audiences. The industry and its clients would be wise to remember that, still, the decision to join a panel or to participate in research studies remains a function of the participant’s perceived value given the cost. All the technical sophistication in the world will not overcome a poor experience, and the bar for what constitutes “acceptable experience” is always rising.
<h2>3: Embed Quality</h2>
The industry continues to make great strides in terms of understanding the drivers of quality in research design and sample management, particularly on connected lifestyles and <a href="https://www.greenbook.org/market-research-firms/mobile-phone-surveys">mobile research</a> experience. The work is as good as it’s ever been. Self-service tools are getting better, too.
The real issue here though, still, is around translating the research-on-research into action to improve research participation. The solution is a tricky one for incumbent firms. End clients need to agree to more sensible designs, yet these typically imply not only lower price points but also difficult change. Suppliers aiming to avoid big change may, for a time, be able to maintain revenue, but will unquestionably see margins shrink as studies become increasingly costly to field. It is a quintessential prisoner’s dilemma, and escaping it is the biggest challenge the industry and its individual firms face.
<h2>4: Ooze efficiency</h2>
I’ve mentioned several examples above which illustrate this point. Efficiency via automation is clearly the focus for many companies as they seek to improve speed and cost. These efforts will surely continue in 2015. Expect to start seeing real differences (and consequences) between those who do this well and those who don’t.
<h2>5: Reboot the business model</h2>
Are we on the cusp of this heading into 2015? It seems so. News has just broken of a merger between two analytics and insight firms. These companies combine a variety of assets, experience, and global presence to form a wholly integrated, and quite innovative looking, global top 10 competitor. It will be interesting to see if they get traction.
<h1>Conclusion</h1>
It is tempting to try to reduce the industry’s challenges to bite-sized pieces, as if using big data, understanding millennials, or telling better stories (all big topics in the past year’s conferences) might turn the tide. These things need to be done, but the issues are bigger. I noted in the original article that the biggest mistake firms being disrupted can make is to think marginally when more profound action is needed. Firms need to act strategically by envisioning the business they would build if they were starting from scratch.
&nbsp;
Copyright © 2014. Jonathan D. Deitch, Ph. D. All Rights Reserved.

Disrupting Market Research: An Update

Market research is speeding up. Demands on researchers are rising. Pressure is increasing for granular, quality insights. We know these things to be true and yet many still struggle with implementing the technology needed to meet this new reality head-on.
Too often, there is an internal disconnect at many companies. This is one of the first gaps, if not chasms, that crop up along the way. Business leadership may have a long-term vision for efficiency and future-proofing that is not aligned with the operations and IT teams’ interpretation of how best to fulfill this vision. For the latter two teams, the most pressing, immediate issues tend to be prioritized and, consequently, automation solutions tend to be designed and architected to meet relatively short-term goals. 
This sets in motion a journey that may ultimately be destined for catastrophic failure and major expense. And this journey could quietly cost the company its long-term ambition and ability to compete, as innovation dies under a mushroom of technical debt.
<div id="attachment_32452" style="width: 1034px" class="wp-caption aligncenter"><img aria-describedby="caption-attachment-32452" decoding="async" class="wp-image-32452 size-large" src="https://www.greenbook.org/mr/wp-content/uploads/2021/12/Cint-1-1024x576.jpg" alt="sleek sports car" width="1024" height="576" />KEVIN BHAGAT, <a href="https://unsplash.com/photos/3cLpiv8h5so">UNSPLASH</a></div>
In our industry, we see this happening most frequently in the large, broken, and largely unresolved piece in the middle of the market research process, having to do with supplier and fieldwork management. While many are using the power of automation to tackle challenges in steps such as survey programming and design all the way through to data analysis and reporting, the data collection piece often becomes the bottleneck. This is the second gap, and it’s one that promises to become even wider as time goes on.
When companies try to solve the data collection sticking point internally, the focus is taken away from the company’s primary vision and goals. What many fail to realize initially is the massive complexity of this stage. In order to find an acceptably-sized pool of respondents to achieve quotas, feasibility, and data quality, most need to tap into multiple suppliers. There is no standardization among these suppliers, which makes combining them a full-time, specialist task. And that’s just a tiny portion of the critical components that must be considered. Companies must also manage financial and process impacts, effects on stakeholders, efficiencies of the entire supply chain, human resources strain, and much, much more. It’s easy to see how maintaining this piece of the process quickly takes over and introduces gargantuan disruptions to daily workflow. Long-term implications abound.

 <div class="post-related-post">
 <h3>Related</h3>
 <h4><a href="https://www.greenbook.org/insights/understanding-the-scope-and-pitfalls-of-end-to-end-insights-automation">Understanding the Scope and Pitfalls of End-to-End Insights Automation</a></h4>
 </div>
 
The truth is, however, that solutions are out there. Software, APIs, and algorithms now present a better way. Technology allows companies to tap into the power of existing solutions and partners that can handle all the moving pieces for them, in an adaptable, seamless way. For example, solutions already exist that automatically standardize all of the considerations from a vast number of sample suppliers, panels, and third-party vendors. This takes the impossible burden of standardizing respondent profiling and other supplier-centric complexities off internal shoulders. This alone can allow companies to renew their focus on being innovative, agile, and fast-to-market, which is becoming more and more important in today’s ecosystem. 
<div id="attachment_32454" style="width: 1034px" class="wp-caption aligncenter"><img aria-describedby="caption-attachment-32454" decoding="async" class="wp-image-32454 size-large" src="https://www.greenbook.org/mr/wp-content/uploads/2021/12/Cint-2-1024x575.jpg" alt="turtle chasing a cheetah" width="1024" height="575" />MAROS_BAUER, <a href="https://www.istockphoto.com/photo/running-cheetah-gm466996757-26054183">ISTOCK</a> x YOD67, <a href="https://www.istockphoto.com/photo/image-of-turtle-isolated-on-white-background-wild-animam-gm664908914-121079653">ISTOCK</a></div>
Automation truly is a business-critical issue in the market research and consumer insights industries, especially at a time when pressure for agility and efficiency is increasing all the time. The decisions insights companies make today regarding automation will impact future success or failure. We’ve outlined these complexities in a recent white paper, <a href="https://info.cint.com/whitepaper-successful-insights-automation">Transforming data collection without paralysing your insights business.</a>
Today, success means employing genuinely scalable and future-proof solutions that are equipped to address the unique challenges faced by market researchers, preparing them for the next phase of change. On this front, huge wins can be realized by bridging that data collection step with smart automation &#8211; but only when it is done in a way that doesn’t take the proverbial eye off the prize: a company’s long-term vision and goals.

Bridging the Automation Gap to Meet Today’s Insights Demands

There is an ongoing quest to reach audiences in our multi-channel world and marketers, advertisers and brands have had to adapt to the rapidly changing landscape in the digital marketing space. No longer are marketers able to target consumers just through television or magazines. Instead, these experts have been compelled to embrace new ideas, technology and strategies. With the consumers’ shift to mobile, ensuring the right audience for a brand’s message has never been more important.
The emergence of ‘People Based Marketing’ has been a welcomed and boundless innovation, gaining immense traction thanks to businesses like Facebook and Google. With its cross-platform user profiles, Facebook has undoubtedly become the king of self-declared data.
<h3>Where are we now?</h3>
Reaching the right audience is the Holy Grail for advertisers, and web publishers in particular rely on this as their source of revenue. To succeed in attracting advertising spend, publishers need to drastically expand their ability to provide the self-declared data that Facebook, Google – and even players such as Amazon – have made commonplace.
Most web publishers have long relied on inferred data, licensed from a third-party provider running a Data Managed Platform (DMP), as they do not necessarily have the ability to regularly uncover rich, self-declared data from their current audience. Inferred data is an acceptable way to audience target, but questions about recency and accuracy persist. In fact, a study conducted by Atlas found that there was only a 65 percent accuracy rate in demographic targeting when using cookie-based measurement – that is a 35 percent error rate! This begs the question – is it even possible for publishers to compete with the current data giants?
<h3>Help is at hand</h3>
During their data quest, some publishers have turned to third-party solutions to provide a viable way to passively track audiences to gain inferred data via surfing behavior. Unfortunately, this dynamic inevitably creates conflicts. To take one example of many, Business Insider found itself in a dispute with cross platform measurement company comScore. Business Insider published a data breakdown illustrating a monthly audience of 300 million – more than three times its comScore estimate.
Instead of relying solely on measurement companies, publishers are in the perfect position to use a lucrative source that is right at their fingertips – their own audience base. Publishers have always known that they have strong brands and credibility with their audiences. But the legacy of the print mind-set meant that many publishers have not known how to make the leap from a strong brand to premium ad rates. The answer is surprisingly simple: just talk to your readers, who are usually more than willing to divulge demographic information to support trusted news outlets. By inviting their audience to become survey panelists, using email, social media and the website, publishers are enabling their users to supply self-declared data which can be combined with inferred, to provide richer, more insightful data.
For a web publisher such as the New York Times, with around 30 million users, panelizing a relatively small percentage of that base would supply a large enough sample size to extrapolate data and create accurate insights. The benefits of creating a panel of users specific to a publisher are infinite. Uncovering more audience insights will not only benefit the publishing platform, but also help generate further revenue from the increased advertising sell. Similarly, there is the potential to create a panel marketplace, thus generating added user engagement – as both panel owner and panelist are rewarded.
<h3>Does size matter?</h3>
While publishers with millions of users can quickly generate this level of data, smaller brands may believe that their audience base will not yield substantial results. This is not the case and even modest publishers can obtain lucrative data from their users. In fact, consumer loyalty is frequently stronger with smaller brands as they are able to better connect with the audience and cultivate ongoing relationships.
Rewards are a key area where publishers can generate benefits for their audience. A publisher might attract or retain a panelist based on incentives and can create further brand affinity during this process. For a subscription-based publisher, a panelist could be incentivized via a subscription credit or several free article reads. By using a virtual currency that will benefit both parties, ‘stickiness’ and enhanced user engagement are created for the website, resulting in incremental page views and ad impressions. This in turn, feeds the cycle of creating data that will validate effective audience targeting to advertisers.
<h3>Where do we go from here?</h3>
Surveying a portion of an audience base may seem like a daunting task while also being time-consuming and expensive. The truth is, technology is a publishers’ most trusted ally. Generally, publishers are unaware of the options available to them to generate their own self-declared data.  They are sitting on a gold mine of insights and just need to implement the right mining tools to unearth the benefits. When first-party, self-declared data insights are combined with inferred data sets, the result is more relevant audience profiles that can be used by brands for accurate ad targeting – and that is how publishers can compete with the data giants!

The Publisher’s Secret Weapon: Using Self-Declared Data to Build Audience Profiles

<h3>CHALLENGE</h3>
Werkspot.nl is a Dutch online platform that connects homeowners in need of home improvements, to service professionals that can get the job done. After consumers post their job needs (anything from a small fix in the kitchen to full home renovations), service professionals who are interested can reach back with their proposals and answer any possible question from the users. Then it’s up to the users to compare professionals, check their customer reviews and decide who they want to hire for the job.
Werkspot launched its international label, Instapro.it, in Italy in 2015. Werkspot wanted to boost their company awareness in the Italian territory with relevant PR campaigns but needed help in understanding what were the home-improvement trends among homeowners in Italy. This is when they approached Cint to set up what they call a “trend-barometer”.
<h3>SOLUTION</h3>
Access by Cint was the perfect tool for the job, giving Werkspot the opportunity to reach 757,000 consumers in Italy alone with precise and recurrent survey targeting across a period of six months. Each survey ran for about a week, targeting home owners aged between 30 and 65 spread across the whole country.
Access by Cint is a survey targeting tool (SaaS) that allows users to target and field to/interview 40 million consumers worldwide, all while getting your project’s feasibility and costs in real time thanks to our advanced algorithms. With Access by Cint, you can also target and create/manage any type of quotas, such as: region, income, type of education, age of children, car brands, and more.
<h3>OUTCOME</h3>
Werkspot received regular datasets from Cint that were used to get an overview of trends in the home improvement market in Italy. Werkspot was finally able to answer questions like: how much money do Italians spend on home improvement, which type of home improvement is most popular, etc.
Additionally, each survey also had questions built around a specific theme (such as, postponing jobs in the house and how this could affect people’s relationships), so that they could use these answers to create a selection of thematic press releases that were then distributed to journalists and bloggers to create awareness around the Italian Instapro platform.
“Cint’s ‘Trend barometer’ research data is one of the pillars of our PR strategy. We have used it as a basis to create press releases, organise media events for journalists, create infographics, and more – all of which help our goal to become the go-to authority within the home improvement world. The resulting PR content generated from Cint’s data has been published all across the Italian TV, newspapers, lifestyle magazines and online media and has been vital to our brand building in the region.”
– Elize Brolsma, International 
PR &amp; Communication Specialist
Visit Cint at <a href="http://www.cint.com">www.cint.com</a>.

How Werkspot Uses Cint to Approach New Markets with Effective Data-Driven Brand Awareness Campaigns

Consumers are complex. Digitalization provides us with more and more consumer data to explore &#8211; and proving that we can no longer make assumptions about the behaviors and sentiments of any specific population. At Cint, we’ve presented at conferences (and even written a white paper) about how critical it is to <a href="https://info.cint.com/connected-data-whitepaper">connect all this data together</a> in order to gain a holistic view of audiences. And primary research must be a part of this mix. There is no replacement when it comes to finding out exactly what makes your target audience tick. It gives you that all-important &#8220;why&#8221;.
We decided to do a little research of our own, accessing 3,000 respondents in the U.K., U.S. and India &#8211; split evenly among markets and generational groups &#8211; using our global sample exchange. We wanted to find out just what kind of balance people were looking for when it came to conducting their daily activities online versus in real life. We asked them questions about online fatigue, preferred shopping activities, and their expectations from brands for the future. While there were some similarities across generations and markets &#8211; such as conveniences offered by more streamlined digital shopping translating into face-to-face experiences in the future &#8211; we did find significant nuances among the various audiences.
While the full results of the study are fascinating, and can be found in our new white paper <a href="https://info.cint.com/online-convenience-versus-fatigue">Convenience versus fatigue in a digital world</a>, it is the challenges survey research faces today that I’d like to discuss here.
First, as I mentioned in our <a href="https://www.cint.com/blog/cint-weighs-in-what-does-2022-hold-for-the-consumer-insights-industry">2022 predictions blog</a>, the demand for consumer insights has never been higher. As companies scramble to keep up with a consumer (and a marketplace) that is in constant flux, they need answers fast. This means a lot of surveys, and a lot of fast turnarounds. For our recent survey, we were able to turn the data around in just three days, and it is this kind of speed that the industry is seeking. But it really isn’t possible inside the constraints of our traditional primary research process, and we must begin to implement and rely on increasingly advanced technology to achieve the accuracy and speed the industry requires.

 <div class="post-related-post">
 <h3>Related</h3>
 <h4><a href="https://www.greenbook.org/insights/attracting-the-right-members-for-a-successful-community">Attracting the Right Members for a Successful Community</a></h4>
 </div>
 
Second, this huge level of demand also means that we are experiencing an industry-wide constraint on sample, especially in the United States. Finding the right people for primary research projects is the first step toward greater understanding. And in today’s sample landscape, finding them means first accessing a large enough pool of highly profiled potential respondents in order to find those that can provide the best insights for your audience and your brand.
With all of us trying to adapt to our new normal, everyone is rushing to keep up with a changing landscape. Consumers are not behaving in predictable ways, and brands cannot afford to make assumptions about their audiences. Primary research is the only way to really access the &#8220;voice of the consumer&#8221;. I foresee this demand continuing, and the need for unique supply to be a key challenge in the future as well.

Not a Small World after All: Are Your Surveys Reaching the Right Audiences?

Cint is a global software leader in digital insights gathering. Stockholm, Sweden.

HIPAA Compliant

Full Service

Fast, Reliable Answers for Insightful Decisions. OvationMR is a trusted place where research practitioners find their target population via engaged global survey panel audiences and significantly improve outcomes on B2B &amp; Consumer studies.QualibeeOE is our AI-enabled online survey question probing tool that gives you deeper, more meaningful insights from regular quantitative surveys. Qualibee also provides response scoring for better survey data quality.EthOS is our Mobile First UX, CX, and Employee Experience Platform for conducting Ethnographic Research, Journey Mapping, Concept Testing, Qualitative IDIs, and Video Surveys. Use OvationMR Solutions for...Brand TrackingConcept TestsMobile EthnographySegmentation Studies and more OvationMR delivers Global Online Survey Audiences and Services to both Marketing and Social Science Researchers. An experienced team will build your sampling plan using probability and/or quota sampling methods and find feasibility for real respondents at the best value compared with any provider in the marketplace.Try our Sample Size Estimation ToolWe deliver quality products...B2B Online Sample - Most Feasibility - Over 140 Industry Verticals.Consumer: Low Incidence &amp; Hard to Reach AudiencesChildren/Teens/Young Adults: GENZ/Millennials - GENX/Boomers &amp; SeniorsBrand and Wave Tracking Sampling for B2B and Consumer Audiences.Public Opinion &amp; Political PollingPassive Metering Capabilities for tracking behavioral data.Committed to our Client's success and your ability to win in the marketplace with reliable data &amp; products. Our service model offers consistent value and the ability to expect full deliverables; on time and to meet or exceed your expectations.Every project, whether it's a sample only project, or we program your survey and translate into multiple languages, is handled with precision and confidence by a professional who works with you as a partner.

OvationMR

IT'S A GREAT DAY FOR DISCOVERING WHY... Connecting great Audiences, Insight Practitioners, and Research Tech—you tell your exceptional story. 

Cint

About

Research Solutions