Panels / Communities

Qualitative Research

Quantitative Research

DISQO is the CX intelligence platform for understanding every customer experience. Businesses trust DISQO to power better decisions for every customer, touchpoint, and outcome. 
<br>
<br>
Millions of consumers power DISQO’s insights, agile testing, and advertising measurement products on the industry’s largest opt-in consumer data platform. DISQO is recognized in Deloitte’s Fast 500 and is a Top 100 Next Gen Workplace. Follow @DISQO on LinkedIn and Twitter.

DISQO

For businesses that want to build awareness and maximize lead generation performance

Advanced

United States of America

Canada

DISQO delivers unprecedented data and analytics, connecting what people say and do via its high-quality, first-party audience.

READ CASE STUDIES

<p>Demand for insights about customer experience is more urgent than ever. Whether it&#8217;s in-depth strategic research led by insights professionals or customer experience feedback collected by product managers, marketers, or customer care teams, clients need trustworthy knowledge to make data-driven decisions.</p>
<p>Look at David Zaslav, the newly minted CEO of Warner Bros. Discovery. <a href="https://www.wsj.com/articles/warner-bros-discovery-david-zaslav-11652822285">According to The WSJ,</a> he emailed this to his leadership team, “As we build this new company, we need to be guided by data and insights to understand what’s working and what’s not.”</p>
<p>A media merger like the one he’s stewarding is the highest of stakes. However, today’s inflation and supply chain shortages mean that bad decisions by any size or type of business are more expensive and difficult to rectify.</p>
<p>Data quality must be a primary focus; and, our industry must invest more and innovate faster in fraud-fighting technologies.</p>
<h2>First, what is Data Science?</h2>
<p>Data science combines math and statistics, specialized programming, advanced analytics, artificial intelligence (AI), and machine learning with specific subject matter expertise to uncover actionable insights hidden in an organization’s data. These insights can be used to guide decision making and strategic planning.</p>
<p>Deepening industry-wide investments in data science, artificial intelligence, and automation – and sharing our learnings – are imperative to raising the bar on the quality of the data collected.</p>
<p>&nbsp;</p>
<h3>What can be done about data quality?</h3>
<p>Given the volume and rising sophistication of research fraud, we simply won’t outpace the thieves without technological solutions. And it’s equally important that we use these tools to preserve the experience of good respondents.</p>
<p>&nbsp;</p>
<h4>Consider a tiered approach to identity validation</h4>
<p>What leading providers find works best are tiered approaches to identity validation that combine passive and active measures. In this type of approach, validation requirements can ramp up based on the likelihood that a potential audience member may be invalid.</p>
<p>Machine learning models are key to this. With these tools that can improve over time, a provider can canvass thousands of data points, including respondent-supplied information, third-party data sources, and later… the behavior within the survey platform, scrupulously looking for suspicious signals.</p>
<p>By flagging suspicious activity, the machine learning model can tell you how vigorous to be in verification and very importantly &#8212; when to usher someone authentic and sought after by clients into their projects. Technology can help us to nurture the “babies” while discarding the bathwater and the “soap scum.”</p>
<p>&nbsp;</p>
<h4>Use machine learning across the respondent’s life cycle</h4>
<p>In a “trust but verify” approach, you can also check multiple fraud indicators at the outset and assign tags to respondents for use in other models later. With a combination of supervised and unsupervised learning models, you can efficiently monitor a respondent’s full journey within your environment. When done right, the models learn emerging and suspicious behavioral patterns and prompt intervention. This empowers you to catch a fraudster who may have made it through your initial defenses.</p>

        <div class="post-related-post">
            <h3>Related</h3>
            <h4><a href="https://www.greenbook.org/insights/market-research-fraud-is-on-the-rise-lets-conquer-it-together">Market Research Fraud is on the Rise — Let’s Conquer It Together</a></h4>
        </div>
    
<p>When you add automation on top of machine learning, this empowers you to speed intervention when required, and to ensure continuous audience grooming. This must be done very carefully to protect the respondent&#8217;s experience. Some respondents may simply need to take additional steps, like biometric identity verification, to prove they are who say they are… and that they are qualified for a particular study.</p>
<p>Technology solutions are simply imperative to meet the sophistication of today’s bad actors. For example, we’ve seen groups of synthetic profiles with similar behaviors that targeted specific surveys and manipulated the links. Providers have also seen fraudsters signup with hundreds of duplicate accounts predicated on different disposable domains.</p>
<p>&nbsp;</p>
<h4>Stay vigilant</h4>
<p>I talk to clients about fraud every day. It’s top of mind for them. Not long ago, there were daily headlines about fraud in programmatic advertising. Clients learned then that the “people” on the other side of an ad impression were not all the same – and that investing in quality audience supply and tech defenses was worthwhile. The parallels to our corner of the industry are very clear.</p>
<p>It’s incumbent on all of us to invest, innovate, and do more.</p>


Using Data Science to Trust but Verify

<p><em>The insights industry is facing a global supply shortage. What can we do about it?</em></p>
<p>Hardly a day goes by without the global supply shortage being a front-page news item. On June 1, The New York Times published an article, “How the World Ran Out of Everything.” It explained in detail how the supply chain “ran amok” due to the pandemic, exacerbated by just-in-time manufacturing (first pioneered by Toyota). You may have experienced the pain of the supply shortage yourself if you bought a new car recently. Odds are you saw the low inventory of automobiles due to–of all things–a massive shortage of $1.00 chips, referred to as the “new toilet paper.” You may have even paid a premium to get the model you wanted.</p>
<p>The insights industry is experiencing its own global supply shortage. Demand for consumers’ opinions is high as marketers seek to understand the new and evolving paradigm for buying preferences and behavior. The supply of respondents is simply not keeping pace with demand for the first time in the history of online research.</p>
<p>The sample ecosystem and its buyers have been buzzing about this new dynamic. During recent conversations, a buyer representing a major CPG brand commented that even getting survey responses for a 10-minute, high incidence study was a challenge. How is this possible?</p>
<p>There are a lot of factors contributing to the changing dynamics in the supply and demand for respondents. Some issues are driven by the rush to reach consumers digitally during and post-pandemic, but many other issues have been brewing for years.</p>
<p>&nbsp;</p>
<h2>It’s always been hard to be a research respondent, now it’s worse</h2>
<p>In the early days of online research, now over 20 years ago, the issues were often masked. Taking surveys online was a novelty and there seemed to be an endless supply of respondents. While there was a lot of hype about capitalizing on digital capabilities and giving respondents an engaging, if not fun experience, it was mostly just that—hype. When offline surveys transitioned to online, preserving historical trends took priority. Questionnaires administered by phone and mail were literally transferred to an online format, onerous grids and all. The length of the interview often ran 30 minutes to an hour, particularly for syndicated studies. In my view, it was a huge, missed opportunity.</p>
<p>&nbsp;</p>

        <div class="post-related-post">
            <h3>Related</h3>
            <h4><a href="https://www.greenbook.org/insights/research-methodologies/online-surveys-are-not-enough">Online Surveys Are Not Enough</a></h4>
        </div>
    
<p>In today’s world, the respondent experience has gotten even worse. Price compression, not just during the pandemic, but over the years, has resulted in lower reward compensation that is typically well under $1.00. More often than not, legitimate people are still turned away from surveys, sometimes after investing significant time completing the screener. Many surveys are still too long and designed to achieve research objectives without considering the enjoyment, or experience of the person completing the survey.</p>
<p><strong><em> </em></strong></p>
<h2>Pandemic-spiked digital advertising costs make it more costly to recruit</h2>
<p>“Just-in-time” inventory practices affected the availability of sample across the ecosystem. As <a href="https://www.greenbook.org/market-research-companies/new-zealand">market research</a> studies were put on hold during the uncertainty of the early months of the pandemic, suppliers ratcheted back their investments to acquire new respondents. This made it difficult to catch up when demand hit new highs over the last nine months.</p>
<p>But there’s another contributing factor that’s having an even bigger impact. As consumers spent more time online than ever before, competition to reach them intensified, not just within the MR industry, but across the advertising ecosystem. Dollars from other advertising channels poured into digital, and digital advertising costs skyrocketed. From Q1 2020 to Q1 2021, prices increased across the board by 30 to 40%, and even higher in some cases.</p>
<p>We don’t see signs of this abating as we enter the era of “the new normal.” In fact, WPP’s GroupM, the world’s largest advertising buyer, just released a <a href="https://www.wsj.com/articles/global-ad-spending-expected-to-rebound-faster-than-previously-forecast-11623666600?mod=searchresults_pos1&amp;page=1">forecast</a> stating that U.S. ad spending (excluding political) would increase by 22% this year and pointed to the acceleration of digital as a major driver which they anticipate will grow by 26%.  As a result, it takes a much bigger investment and marketing prowess to acquire respondents.</p>
<p>&nbsp;</p>
<h2>Respecting and valuing respondents is the answer</h2>
<p>No one can wave a magic wand to increase the “supply” of research respondents across the sample ecosystem, but the research community can make a significant impact.  The most sustainable method is to respect and deliver great experiences. First and foremost, we must acknowledge that research respondents are not a commodity and that the pool of respondents is not infinite. If we want to preserve our most precious assets, consumers who are willing to offer their opinions to help brands make the right business decisions, we need to value them.</p>
<p>At DISQO, we built a consumer-centric platform to provide a modern, frictionless experience right from the beginning. We are focused on delivering on our promise to our first-party panel members to share information in exchange for a great experience and value. The entire MR industry, including corporate buyers, MR Agencies, and sample providers should rally around this same value.</p>
<p>Here are some critical ways to optimize the experience for our respondents, making it more user-friendly. While these are not novel, they have yet to be accomplished on a wide scale.</p>
<ul>
<li>Give respondents the right opportunities rather than screening them out of survey after survey.</li>
<li>Shorten screeners to one or two minutes. Let’s not drag respondents through a long screener only to be disqualified.</li>
<li>Improve questionnaires—make them shorter, more engaging, and less repetitious for <u>all</u> respondents, including the majority who take surveys on their mobile phones and those who use their desktops and laptops.</li>
<li>Appropriately reward respondents for their time and opinions.</li>
</ul>
<p>Finally, let’s not lose sight of the fact that engaged, attentive respondents provide higher-quality data. With millions of dollars on the line when clients make decisions using our insights, let’s make sure to treat the people fueling those insights the right way. It’s the best way to support brand growth over the long term and a win/win for all.</p>
<p>To learn more about how DISQO builds and engages our 100% first-party audience, <a href="https://go.disqo.com/productcard-marketresearch-general-greenbook?utm_source=greenbook&amp;utm_medium=paid&amp;utm_campaign=2021-MR-ProductCards-MR&amp;utm_content=S-R&amp;utm_term=greenbook-link">click here</a>.</p>
<p><em>Photo by <strong><a href="https://www.pexels.com/@jopwell?utm_content=attributionCopyText&amp;utm_medium=referral&amp;utm_source=pexels">Jopwell</a></strong> from <strong><a href="https://www.pexels.com/photo/woman-in-blue-suit-jacket-2422293/?utm_content=attributionCopyText&amp;utm_medium=referral&amp;utm_source=pexels">Pexels</a></strong></em></p>


Supply, Demand and Sustainability in Market Research

<p>Fraud in market research continues to be massively problematic. Everyone has felt it. Exacerbated by the pandemic, our industry saw a big spike in fraud in 2020. On average across the industry, 15 to 30% of market research data is fraudulent. In some studies, the figures are even higher. We&rsquo;re not alone; eCommerce, credit card, and identity fraud all went through the roof last year.</p>
<p>To preserve the integrity of our work, it&rsquo;s important that we understand how market research fraud is evolving, that we commit to innovating new deterrents, and that we band together to fight it with an industry-wide response. In this spirit, let&rsquo;s closely examine what is happening and what we can do about it.</p>
<p>&nbsp;</p>
<h2>Knowing the enemy</h2>
<p>First, let&rsquo;s define the bad actors and what they are doing. Fraudsters can be defined as unqualified people who invade market research surveys to make sums of money from the incentives. In today&rsquo;s world, they&rsquo;re increasingly difficult to identify, and staying ahead of them requires vigilance.</p>
<p>We&rsquo;ve long been plagued by bots, automated programs that drive large volumes of fake traffic to market research surveys, but now the human fraudsters are wreaking just as much havoc, probably more. We&rsquo;re faced with a professional cottage industry where people are paid by criminal enterprises to perpetrate fraud. These &ldquo;professionals&rdquo; teach others how to qualify for surveys and to pass quality control checks. They provide step-by-step instructions in videos posted on YouTube, Facebook, and other social media. And they are not in just a few countries anymore &mdash; they&rsquo;ve gone global.</p>
<p>Identity theft is also a major contributor to market fraud&rsquo;s explosive growth. Thanks to a number of massive global data breaches over the past decade, creating fake identities has never been easier. Social security numbers are being bought on the black market. And mobile phone numbers can be bought for as little as 15 cents.</p>
<p>As a result, the common techniques our industry has used to validate panelists&rsquo; identities are no longer enough. This includes steps such as matching a name and address against a third-party source or using two-step phone authentication.</p>
<p>&nbsp;</p>
<h2>Protecting the experience for good panelists</h2>
<p>With the challenges we face, multiple layers of anti-fraud defenses are necessary. But how do we do this while protecting the quality experience we need to deliver to qualified, legitimate panelists?</p>
<p>For the good people, it is essential to offer a low-friction experience and to avoid treating them with suspicion. We know that engaging panelists with a good experience is directly correlated with data quality. The good news is there are ways to deliver a great experience while also keeping the bad actors out.</p>
<p>We enable identity validation at the individual panelist level by leveraging first-party panels. We use a balance of passive and active methods in multiple layers to understand who our panelists are. With passive checks like cross-checking identity with <em>multiple</em> third-party directories, the panelist can be totally unaware of your due diligence behind the scenes. Active checks like two-step phone authentication still have a place&hellip; but need to be combined with other things. At <a href="https://www.greenbook.org/company/DISQO">DISQO</a>, we added &ldquo;old-school&rdquo; manual techniques a couple of years ago. Phone outreach by customer care teams for live verification and sending postcards with PINs to physical locations each have renewed relevance in the digital age.</p>
<p>Even with a combination of these techniques and many more, fraud can still get through. Bad actors will find ways to bypass almost any check. So we must continuously look to more sophisticated and innovative techniques.</p>
<p>&nbsp;</p>
<h2>Innovating new defenses &ndash; AI, machine learning, biometrics</h2>
<p>It is promising that there is a burgeoning marketplace of commercial anti-fraud services. They each have their own &ldquo;secret sauce&rdquo; for detecting and mitigating fraud in real-time. They also bring the important advantage of being able to see and monitor the playing field across multiple sample providers within their platforms. These services are an important tool in the arsenal.</p>
<p>Artificial intelligence and machine learning offer much-needed automation to pour over large volumes of data &mdash; looking for suspect behaviors. We&rsquo;ve enhanced our models to take into account highly accurate fraud flags that are actionable for preventative measures. We value that these are passive approaches with little to no impact on panelists&rsquo; experience.</p>
<p>This data science allows us to score the risk of an individual panelist and then apply the appropriate types of verification measures accordingly. This way, you can alleviate the need for good panelists to go through some of the more rigorous validation processes we would apply to others. Best of all, artificial intelligence models learn continuously and get smarter over time as they process more and more data.</p>
<p>The industry can also look to Fintech for powerful new solutions. After all, banks know a thing or two about identity verification. ID validation combined with facial recognition through biometrics is proving to catch fraudsters in their tracks, and it only takes three minutes for panelists to complete the process. This technique is highly effective for weeding out bad actors, viable internationally, CCPA and GDPR compliant, and can even be configured to accommodate people with disabilities.</p>
<p>&nbsp;</p>
<h2>Let&rsquo;s conquer fraud together</h2>
<p>Mitigating fraud is never-ending. Thieves will work just as hard to learn as we do, and they will always find a way &mdash; wherever there is marketing money to be siphoned online. This means the market research industry must commit to not only innovating new solutions but also to coming together to share best practices and learn from each other. We are a close industry, and I&rsquo;m proud that we collaborate on the big issues. Let&rsquo;s do more. It is essential we protect the value of market research to ensure our companies and clients can make business decisions with confidence.</p>
<p><em>Photo by <a href="https://unsplash.com/@camylla93?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Camylla Battani</a> on <a href="https://unsplash.com/">Unsplash</a> </em></p>

Market research fraud is massive. fraud spiked in 2020, as much as 30% in some studies. Understand how survey fraud is evolving, how to deter mrx fraud, and how to fight fraud.

Market Research Fraud is on the Rise — Let’s Conquer It Together

<p><em><strong>Editor&#8217;s Note: </strong><i>This post is part of our Big Ideas series, a column highlighting the innovative thinking and thought leadership at IIeX events around the world. Bonnie will be moderating a client panel at IIeX North America 2019 in Austin, TX. If you liked this article, you’ll LOVE IIeX North America. Click <a href="https://iiex-na.insightinnovation.org/">here</a> to learn more. </i></em></p>
<hr />
<p>Marketers have long relied on consumers’ memories to measure and predict shopper behavior through online surveys. Yet, it is no secret that human recall is far from perfect. Think about your own shopping experiences. Can you remember all of the products you looked at or bought when you bought them, and what price you paid?</p>
<p>DISQO commissioned an independent market research firm to put shopper recall to the test. Our key objective was to help determine when we can best apply a standalone survey-driven methodology, and when it’s preferable to leverage observed behavioral data instead. When does recall really start to break down?</p>
<h3>The Study</h3>
<p>To achieve this, we asked consumers shopping-related questions via survey, while also tracking their digital activity using proprietary technology. We quantified the “recall gap”, defined as the difference between what consumers recalled and their corresponding behavioral data. This enabled us to begin to measure where the largest gaps exist and consider why.</p>
<p>The research methodology entailed surveying approximately 700 consumers; they were surveyed in December 2018. Research participants were asked about their past 30-day shopping for Electronics and Personal Care products on Amazon and Walmart.com.</p>
<p>Panelists responded as they would to any traditional online survey. The consumers in our study were validated shoppers in specific categories of interest, eliminating the potential for inaccurate survey-based screening.</p>
<h3>The Results</h3>
<p>The findings validate that, in most cases, there is a divergence between what consumers recalled doing online and their corresponding behavioral data. However, the accuracy of consumers’ recall varies by the type of shopping behavior being measured. While recall of general online behaviors is better, recall significantly breaks down with specific behaviors.</p>
<p>Let’s start with the broad categories. Consumers accurately recalled shopping for Electronics on Amazon or Walmart.com in the past 30 days— Respondents thought they shopped less than their behavioral data demonstrates by 4 percentage points&#8211;certainly nothing to be concerned about.</p>
<p>The difference between consumers’ recall and actual behavior for the Personal Care category is 12%. Yes, that’s a larger recall gap than Electronics, but relatively small compared to more specific categories.</p>
<p>For example, within Electronics subcategories, consumers thought they shopped significantly more than their behavior suggests. Respondents over reported their shopping behavior for all three subcategories included in the research, TV/Media Players, Cell Phones/Accessories and Computers/Accessories, by at least 22 percentage points. The same pattern is seen for the three subcategories measured for Personal Care (Hair Care, Skin Care, and Oral Care).</p>
<p>Consumers also had trouble accurately remembering if they shopped on Amazon or Walmart for Electronics and Personal Care products in the last 30 days. However, Amazon fared better than Walmart in this assessment for both categories in terms of respondents’ recall.</p>
<p>A recall gap exists when looking at actual purchase activity and comparing what price consumers paid for products using behavioral vs. survey data. The difference between the price recalled and the actual price was as high as 15 percentage points.</p>
<p>The metrics evaluated in the research-on-research demonstrate the complexity of shopper research and the potential flaws of relying solely on self-reported survey data for shopper behavior.</p>
<h3>The Next Steps</h3>
<p>Where does that leave us? What steps can be taken by marketers and researchers who rely on self-reported shopper data?</p>
<p>Fortunately, today there are new solutions available to provide a bridge between human recall and actual behavior. Marketers and researchers can now utilize first party, permission-based sources of factual, observed digital behaviors as the basis for shopper research.</p>
<p>Why “screen” consumers by asking if and when they purchased a computer on Amazon, or any website when we already know the answer? Not only does this eliminate the guesswork, it positively impacts consumer engagement. The use of behavioral data reduces the time consumers spend answering screening questions that only add to what is typically a long, laborious survey experience.</p>
<p>And let’s not stop there. Why not tap into the upper and lower funnel to understand consumers’ digital behaviors before and after they made a purchase? In addition to accurate data, behavioral data provides the opportunity to discover the full consumer journey which will lead to tactical, highly actionable insights.</p>
<p>Access to digital behavior is exciting and should be leveraged. The combination of observed behavioral interactions to understand “what” people actually do and the ability to survey consumers throughout their journey to understand “why” provides a new and powerful toolkit, ultimately leading to better business decisions.</p>


Human Recall is Not Perfect: A Quantified View of Self-Reported vs. Behavioral Data

<p>How many times in the past month have “ChatGPT” and “AI” come up in conversations at work or with friends? If the answer is zero, please invite me to the rock you’ve been living under. It sounds like a great place for some peace and quiet.</p>
<p>It’s difficult to avoid conversations about the promise and peril of Artificial Intelligence. No one seems to call it by it’s full name anymore because the acronym is so ubiquitous. And to be fair, AI is nothing new; a lot of smart people have been building the AI ecosystem for years.</p>
<p>However, it’s the <a href="https://www.greenbook.org/insights/the-prompt/how-generative-ai-is-changing-the-research-industry">explosion of ChatGPT</a> in recent months that has brought the conversation to the public square. Even the consumers we’re so eager to learn from are talking about it. The conversation is now at the water cooler, on Zoom calls, and in the boardroom.</p>
<p>I’ve been in some of these conversations and have closely attended to how those in the Marketing Research industry are reacting. Much like broader responses, they run the gamut from fear and paranoia that AI may undermines the foundation of the industry to irrational optimism that it’s a panacea for the industry’s many plagues.</p>
<h2>The robots are here…and they speak like us!</h2>
<p>Unsurprisingly, early conversations have touched on the impact of AI on market research fraud. Data quality is already top of mind in the industry, and many have been burned by fraudulent participants and bad data. But what’s that? Here come the bots… and they’re smarter AI-bots? Everyone run for shelter before it’s too late!</p>

        <div class="post-related-post">
            <h3>Related</h3>
            <h4><a href="https://www.greenbook.org/insights/the-prompt/chatgpt-and-implications-for-market-research">ChatGPT and Implications for Market Research</a></h4>
        </div>
    
<p>There are very legitimate concerns around how survey participants could use ChatGPT and other tools to generate very compelling, yet fraudulent, open-ended responses in both short and long form. And with open-ended responses continually and heavily leveraged to identify what data is “bad” and should be discarded, this should be a topic of conversation. I’m not dismissing the magnitude of the challenge, but we can fight AI-empowered fraud using the very same technology.</p>
<p>When I first saw ChatGPT blow up, I flashed back to my days as a professor and was thankful that back then… I didn’t have to face the near impossible challenge of detecting an onslaught of papers written by robots posing as students. But then in a matter of weeks, an entrepreneur developed a way to detect if ChatGPT was used to create content.</p>
<p>Is that battle over? No. But it’s an example of how quickly we can leverage AI for good, rather than evil. That same thing is already happening in market research. Companies that don’t currently have AI tools at their disposal to detect and address open-ended data quality problems will very soon have them. With the barrier to entry dramatically lowered, we should relish it as an opportunity.</p>
<h2>If the robots speak like us, can they really chat like us?</h2>
<p>It hasn’t taken long for people in market research to also ponder how AI might usurp the role of a researcher. How can we use AI to do work typically done by a trained research professional?</p>
<p>Can a bot conduct qualitative interviews? On some level, yes. And again to be fair, several companies have been doing this for years – and getting continually better at it. Generative AI accelerates and democratizes access to tech, enabling a conversational interaction between humans (research participants) and robots (moderators). While online discussion boards and online qual are likely better moderated by a human, we are fully into the human-assisted moderation world already.</p>
<p>Can a bot write questionnaires? Yes, the results aren’t great, but I’ve certainly seen worse. Because of the relatively predictable structure of questionnaires, <a href="https://www.greenbook.org/insights/insights-technology/the-impact-and-ethics-of-artificial-intelligence">AI tools</a> are able to respond well to prompts, guiding creation of a questionnaire consisting of a certain number of questions and focused on a certain topic. Most of the questions are well constructed. The answer choice sets tend to be a good start, if also imperfect. However, question ordering to mitigate bias really doesn’t yet seem to be AI’s forte. This said, AI tools can very quickly craft a viable survey instrument that is mostly better than a survey someone with little to no research experience might construct.</p>
<p>While we can be rightfully critical of how out-of-the-box AI-based tools work for crafting research documents, we need to remember that the models aren’t yet trained for that. They will be. Opportunities for AI-assisted survey design and execution, qual and quant, is happening now &#8212; and quickly.</p>
<h2>The robots are coming for my job!</h2>
<p>Are you sitting back wondering if AI will replace you? If yes, and that’s all you’re doing, maybe you should be worried. But for those skilled in design, moderation, analysis, strategy, storytelling, and more , you can also look at AI as an opportunity rather than a threat.</p>
<p>Remember just a few years back, when conversations around automation and DIY furrowed brows about jobs being automated away? Did that happen? No, not really.</p>
<p>I remember not too many years ago when I spent hours upon hours coding open-ended responses manually. Now, there is no way I would do that without the assistance of automated (and perhaps AI-driven) tools. Let the robots take the work they’re good at. And let me keep the work I’m good at as a human.</p>
<p>Researchers have by and large embraced tools and platforms that speed up their work and automate tedious and manual processes. The sophistication of AI further accelerates this move to spend time where time is well spent, instead of rote activities.</p>
<p>So what does this all mean for researchers and their employers? It means you need to quickly get up to speed. If you’re not exploring generative AI, –- actually using it and not just reading about it, you’re late. But the good news is that the barrier to exploration is almost non-existent. There are free and low-cost AI-based tools aplenty. This space is moving incredibly quickly, but you can also catch up quickly.</p>
<p>Here are four incredibly easy things you can do this week to dip your toes in AI. First, start by signing up for the free version of ChatGPT through OpenAI (or choose another platform). Then…</p>
<ol>
<li>See how well it writes a survey by using the prompt, “Design a 15 question survey about [insert topic].”</li>
<li>Code open ends by pasting in a set of open-ended responses and using the prompt, “Categorize this list of open-ended data.”</li>
<li>Level up coding open-ends by using the same set of responses and the prompt, “Create a dataset that categorizes the following as positive, neutral, or negative sentiment in .csv format: Verbatim | Sentiment | Reason.”</li>
<li>Pull out themes from qual data by pasting excerpts (or full text) from an interview and using the prompt, “Identify and define themes from the following text, organize them into a bulleted list, and use direct quotes from the text to provide examples of each theme.”</li>
</ol>
<p>Playing with these simple prompts should take about a half hour, and while basic, will give you a sense of what the technology will unlock for those in the research space.</p>
<p>Resisting generative AI is truly futile. Explore how AI can improve your business by staying open minded, identifying opportunities, and getting real hands-on keyboards experience.</p>


Generative AI is Here for Good. Are you Ready for it?

<p>One could argue that market research arose from and centers around asking questions.  How can we better serve customers?  How is our target audience responding to our latest ad campaign?  In which ways can our business expand to see maximum growth?  Questions like these (and countless others) are fundamental to what we do.  That said, many of us in the industry are increasingly questioned about why we need to ask people questions at all when there is so much preexisting data at our fingertips.  The sheer volume of data and relative ease of access has improved in recent years, and will continue to change the way we do research.  We’ve come so far so fast that it’s worth taking a step back to consider the types of data available to researchers.  In the interest of making what could be a tome into something readable in between meetings, I’d like to focus on the relationship between “observational” and “engaged” data and where this data comes from.</p>
<h4>Observe or Engage?</h4>
<p>Observational data is collected without actively engaging with anyone, thus making “engaged data” (a new term I’m coining, perhaps), its foil.  And observational data takes many forms, much of it collected digitally: web and mobile activity, purchase and loyalty history, social media chatter, smart TV viewership, market research panelist profiles, CRM records; the list could go on.  One of the reasons we’ve seen these types of data become frequent topics of conversation and an increasing part of research approaches is that there is just so much of it.</p>
<p>What has really brought it to the fore is the technology used to process, analyze, and ultimately make sense of the data.  (I resisted the urge to mention automation, machine learning, and artificial intelligence here but darn, I just did it.)  Additionally, observation has become a substitute for engagement because it has become increasingly challenging to reach the right people to ask questions of.  If you still write those grid-heavy 40-minute surveys that no one, seriously no one, wants to take, have a look in the mirror and ask yourself why this might be.</p>
<p>Engaged data, that which comes from actually asking questions of people, fills something of a different but also complementary role.  Perhaps first and foremost, it helps answer the “why.”  While we might typically evoke the age-old notion of quant answering the “what” and qual answering the why, I think in this case we can look at engaged data – both quant and qual – as allowing researchers to get to the “why.”</p>
<p>For example, understanding motivations behind purchase behaviors or loyalty to a brand are difficult if not impossible to determine from simply looking at observed data.  The flip side of the coin in this specific example, however, is that some purchase behaviors are hard to ask people about.  Think of the last time you bought toothpaste three months ago.  I bet you don’t remember much, if anything, about it.  Marrying observed data from hard to recall points with engaged data: therein lies a sweet spot.  In some cases, observed data will be more accurate and in others, engaged data.  One of the keys is knowing what you are trying to understand and choosing the best approach, often a combinative one, to get there.</p>
<h4>Who’s on First? What’s on Second? “I Don’t Know” is on Third.</h4>
<p>While the approach is key, the source and associated quality of data, is even more important.  Bad data in = bad insights out.  As the market research ecosystem increasingly integrates with the broader digital ecosystem, many lose sight of – or don’t even understand to begin with – where the data they are using comes from.  I’m not going to sugar coat it; that’s a real problem.  Knowing whether your data is first, second, or third-party is crucial to understand.  So, let’s take a moment to define these terms.</p>
<p>First party data is collected directly from people, which could be customers through CRM or loyalty tracking or members of a market research panel.  Second-party data is someone else’s first party data, obtained through a relationship and perhaps most commonly garnered with a data match or append.  Still with me?  The first two are easy but third-party data is where it gets interesting/complicated/tricky.  Third party data is aggregated from various sources, some of which are observed, some engaged, some modeled, some seemingly conjured from unknown depths, and all of which is typically acquired from a data exchange.  I was slightly snide there to make a point.  To be clear, the point is not that third-party data is inherently bad.  It is not.  Some of it may be, however.  The point is when you are dealing with third party data, especially for research purposes, you should ask questions.  Ask questions about sourcing.  Ask questions about accuracy.</p>
<p>&nbsp;</p>
<p><img decoding="async" class="size-full wp-image-20867 aligncenter" src="https://wp.greenbook.org/mr/wp-content/uploads/2018/07/Greenbook-blog-post_Research-Now-SSI_-Roddy-Knowles-Post_embedded-image_20180705.png" alt="" width="526" height="217"   /></p>
<p>&nbsp;</p>
<p>Much of the third party data available on exchanges is used for marketing and activation – for serving ads and marketing to people who fit within target audiences or segments.  Using data for these purposes has, most would argue, significantly improved ad targeting and effectiveness, though it is still an imperfect science.</p>
<p>If I told you that I didn’t need to ask age or gender in a survey and I could guess (read: infer) based on other things I think I know about survey participants and be right about 70% or even 80% of the time, would you be impressed?  What if I could reasonably get income level brackets right about half the time, would that be cool with you?  I bet you wouldn’t be impressed.  The demographic example is a simple one, but could be extended to many other types of modeled or inferred third-party data.  In the demographic example, I’d bet those of us who engage with market research panelists are more accurate than the inferential magicians.  In other cases and with other data points, this may not be so.</p>
<p>What is important is to recognize and grapple with differences when we find them, such as those between observed and engaged data as well as between first and third-party data.  How can first-party data help to calibrate and improve third-party data, including for marketing and advertising purposes?  Which types of third party data are fit for which research purposes?  Which questions does it make sense to ask – and which to observe?  How can observed and engaged data be brought together effectively to best address our clients’ objectives?  I leave you, quite intentionally, with open questions, because asking questions is one of the things we do best.</p>


With So Much Observed Data, Why Bother Asking Questions?

<p class="p1">The greatest need for every marketer is to deeply understand the customer experience. Without that, all other decisions, from product development to marketing strategy, are just guesswork and not centered on the engine that drives brand success &#8211; the customer. Traditional market research has sought to understand the customer by asking them about their actions and beliefs, and while that’s better than developing strategies without the customer in mind, altogether, it is an incomplete approach.</p>
<p>Marketers now have an opportunity to gain a competitive edge by leveraging an underrated yet highly effective technique in their toolbox: behavioral targeting. This novel capability empowers researchers to transcend the confines of traditional survey targeting attributes like demographics and geographics to query people who have relevant observed behavioral traits.</p>
<p class="p3">So what does this mean for insights professionals looking to put consumer experience at the center of their strategy? The largest benefit for researchers is the ability to optimize the screening process by qualifying respondents based on real, validated behaviors.</p>
<h2 class="p3">Meet the new and improved screener</h2>
<p class="p3">Rather than leveraging traditional screeners that rely solely on human recall and honesty, qualifying respondents based on measured behaviors is a more efficient way to refine the right participants for a study. With behaviorally targeted studies, researchers can mitigate other variables that could negatively impact screening accuracy, such as recall bias and ineffective screening question design.</p>
<p class="p3">Actual behaviors offer the truest picture of consumer intent and can serve as an indication of a consumer’s stage in their buying process. For example, one can easily differentiate between consumers who are actively in the market for a new car and those who merely say that they are by studying their online behaviors. Looking for certain high-intent activities, like browsing used cars online, can help confirm if a consumer actually lies within a brand’s target market. This enhanced understanding can unlock research on valuable audiences with verified behaviors tailored to specific needs, whether that’s frequent travelers, loyal investors, or avid game enthusiasts.</p>
<p class="p3">More accurate and efficient screening subsequently results in more productive research studies. On one hand, less time is needed to filter out unqualified responses and research projects can be carried out faster. On the other hand, researchers can maximize the value generated from a study, because the insights are coming from an audience that’s already been validated by observed behaviors. This level of targeting precision unlocks powerful insights, which in turn helps drive even more business decisions.</p>
<h2 class="p3">Improve consumer experience</h2>
<p class="p3">Most important of all, behaviorally targeted surveys make for a better consumer experience. Respondents can qualify for a survey without having to answer a lengthy series of screening questions. Because the number of screeners is reduced or even removed altogether, the total length of interview is shortened, which helps boost engagement and satisfaction for the participants. In addition, shorter surveys often deliver more accurate responses by combating response fatigue. For example, researchers who want rich responses to open-ended questions can make sure their audiences have more time to answer them instead of filling out screening questions they could have already known the answer to.</p>

        <div class="post-related-post">
            <h3>Related</h3>
            <h4><a href="https://www.greenbook.org/insights/fixing-data-quality-at-the-source-a-researcher-who-became-a-respondent">A Researcher Who Became a Respondent (Part One): Fixing Data Quality at the Source</a></h4>
        </div>
    
<p class="p3">Suboptimal survey targeting is one of the most common issues plaguing the industry today. However, by choosing to target survey participants based on their real behaviors, one can ensure the validity of their audience and maximize the accuracy and actionability of their research findings.</p>
<p class="p3">An optimized screening process also ensures that consumers are served surveys that are most relevant to them. Providing the best experience for research participants has been and will continue to be a priority for everyone in the insights industry, especially given the limited supply of respondents today. Qualifying respondents by their observed behaviors and making the screening process faster for them is just one of the many different ways to improve their experience.</p>
<p class="p3">Better research studies are never the end goal, and instead are an important interim outcome contributing to larger and often critical business objectives. Accurate studies drive better product design, brand strategy, and customer experience that grow revenue. Making a small but impactful change by incorporating behavioral data into the survey process is a powerful weapon in the fight for market share.</p>
<p class="p3">Are you making full use of the behavioral capabilities available in research today?</p>


Behaviorally Targeted Surveys: The Marketer’s Secret Weapon

DISQO delivers unprecedented data and analytics, connecting what people say and
do via its high-quality, first-party audience. Glendale, California, United States of America.

DISQO

About