Social Media Analytics: Promises, Challenges and the Future

 

By Kevin Gray and Koen Pauwels

Can Social Media Analytics replace traditional MR?

Compared to traditional marketing research methods, social media analytics is fast and inexpensive.  It is also superior to survey research because it’s less prone to social desirability bias and mistaken recall – consumers are speaking in their own voices, frankly and spontaneously, about brands and the things that matter most to them.

Some Questions

These, at least, are some of the hopes many have had for social media for quite a few years now.  Promise and potential are not reality, however.  What are the realities?¹  Below are some important questions and concerns many marketing researchers have regarding social media.

• Many surveys are now turned around very inexpensively in days, or even hours, so the claim that survey research is slow and expensive seems outdated. Also, in the case of qualitative, hasn’t the popularity of online qual eroded much of the advantage social media once held in terms of cost and speed?
• How much social media demographic data are actually imputed or fictitious, i.e., made up by users to protect their personal privacy? How many posters are really fakes we are unable to detect?
• Are social media users really more representative of general consumers or of particular groups, such as young urban males, than online panelists are?
• How many people only listen to social media instead of posting their opinions? Why do people post on social media? How do posters and lurkers differ?
• Do we only observe the extremes – people who are highly satisfied and those who are highly dissatisfied? Aren’t extreme opinions over-represented in social media conversations?
• Do posters’ comments reflect their true viewpoints? How many are just amusing themselves or jousting with other posters?
• Related to this, to what extent and in what ways are posters influenced by comments of other posters?
• How much do social media data reflect real experience and how much is just opinion? Though this may vary by product category and social media outlet, to what degree do posters’ opinions actually affect buying behavior of those reading their posts?
• Is posters’ factual recall really more accurate than that of survey respondents? Why?
• How much influence do “influencers” actually have on the buying process?
• How much does the abovementioned vary by social media outlet, topic and country?
• Most importantly, how well do social media metrics match market reality? How stable are these metrics over time?  Do they correlate well with other data, such sales/share figures, cross-sectionally and over time?  In our experience, survey data usually match “hard” statistics reasonably closely, as shown in publications ranging from Journal of Advertising Research², the Marketing Science Institute³ , Marketing Science⁴ and the Journal of Marketing Research⁵.

Analysis

These questions pertain only to data.  What about analysis?  Social media analytics is a form of content analysis, and content analysis is not easy.⁶  How about computers – can’t Artificial Intelligence replace human analysts? Both computational linguistics and natural language processing promise to automate classifying the content and sentiment of human communication, but they need to be trained by humans. Superficial training leads to Garbage In – Garbage Out (GIGO), while thorough training can be at least as time-consuming and expensive as traditional marketing research.⁷  Context and disambiguation remain significant challenges, for example.  Indeed, there will always be gaps because computers cannot be programmed to feel emotions. They are not us. They do not laugh at our jokes.  They have never scored the game-winning goal nor had their hearts broken.  They have never been consumers.

The role of the human analyst has not vanished.

Clarifications

Let’s be clear that we do not deny that social media has given us a wealth of new data or that it has had a significant impact on marketing and marketing research.  Though skeptical of many claims we, nonetheless, count ourselves among the faithful.⁸  The marketing world has changed dramatically in the past decade and there is no turning back the clock even if one wished to.  Online reviews, for instance, are shaking up the way marketing is done in many product and service categories.⁹   Though progress has perhaps not been as rapid as some may have predicted, marketing researchers are becoming increasingly adept at mining social media for useful insights.  To some degree it is replacing traditional research.

A Few More Questions

As researchers and consultants, however, we have an obligation to our clients and to our profession to be realistic about what social media analytics can actually deliver.  Here are a few more questions we feel remain largely unanswered.

• Do social media analytics success stories point to the rule or are they mostly exceptions?
• Is listening alone really sufficient? Are discussion moderators therefore unnecessary and, by implication, have been all along?
• Can brand and ad awareness, both spontaneous and prompted, be supplanted by social media metrics?
• How stable over time are brand image metrics derived from social media? Are variations over time mostly a signal of meaningful changes in brand fortunes, or are they mostly noise?
• Can we obtain the accurate and fine-grained breakdowns on individual posters needed for detailed cross tabulations and multivariate analysis? Clients frequently require very specific information and opinions from consumers, and quantitative analysts must be able to tie all these variables together for each person.  Can social media give us the detail for individual posters required to develop the rich profiles of consumers and in-depth analysis many clients have come to expect?  The most actionable research is usually research designed to address particular marketing issues and the essential data are rarely just “out there” waiting to be collected.
• What kind of analytics can it replace? Consumer segmentation?  Key driver analysis with multivariate analysis or machine learners?  Can it replace choice modeling, data mining and predictive analytics?  Marketing mix modeling?  If so, where are the examples?
• Aren’t many of the shortcomings of traditional marketing research actually reflections of poor skills and inexperience? How does the best of traditional marketing research compare to the best of social media analytics?

The Future

Social media is still quite new, and the media themselves and the analytic tools for exploiting them are still evolving.  Let’s be honest with ourselves – how many true social media experts can there really be?  What would be the risks of suddenly discarding methods that have served us well for so long in favor of an alternative that has not yet stood the test of time?  Why not concentrate instead on using social media qualitatively to assist in questionnaire development, or as one component in marketing mix modeling, or to put a human face on data mining and predictive analytics?  Why not focus on utilizing it in tandem with other qualitative methods?  Social media analytics has already proven itself in these roles.

Is social media an asteroid streaking towards traditional marketing research or is it a valuable complement rather than a complete substitute?  We lean towards the second conclusion and feel social media in the main adds to but will never fully replace traditional marketing research.  We see it as an important new and increasingly indispensible source of insights, but not the catastrophe some have feared nor the research nirvana others have sought.

 

____________________________________________________________________

Notes

¹ Social Media Intelligence (Moe and Schweidel) provides a good overview of social media that addresses some of the concerns we raise in the article.  Social Media is fairly new and still evolving, however, and much research remains to be done. Furthermore, what might apply in one country may not apply in another due to cultural differences and because Social Media is not uniform across the globe.

² Lautman, M. & K. Pauwels. “What is important? Identifying metrics that matter.” Journal of Advertising Research 49.3 (2009): 339-359.

³ Pauwels, K. and B. van Ewijk. “Do Online Behavior Tracking or Attitude Survey Metrics Drive Brand Sales? An Integrative Model of Attitudes and Actions on the Consumer Boulevard.” Marketing Science Institute (2014): 13-118.

⁴ Hanssens, D. et al. “Consumer attitude metrics for guiding marketing mix decisions.” Marketing Science 33.4 (2014): 534-550.

⁵ Srinivasan, S. et al. “Mind-set metrics in market response models: An integrative approach.” Journal of Marketing Research 47.4 (2010): 672-684.

⁶ Content Analysis: An Introduction to Its Methodology (Krippendorff) is a comprehensive (and dense) textbook on this subject.

⁷ See, for example, Artificial Intelligence (Russell and Norvig), Foundations of Computational Linguistics (Hausser), The Handbook of Computational Linguistics (Clark et al.) and Introduction to Information Retrieval (Manning et al.).

⁸ Vocal advocates of social media (and some other new technologies) typically react to questions such as ours by ignoring them, or by intimating that those posing them are behind the times and set in their ways, or by acknowledging that they are legitimate questions but that, because of recent advances, they are no longer pertinent.  The last response is most convincing when supported by studies that have been replicated by independent researchers with no commercial stakes in the methodology.

⁹ Absolute Value (Simonson and Rosen) gives many examples of how online ratings are disrupting marketing.

____________________________________________________________________

Kevin Gray is a marketing scientist who has been in marketing research for more than 25 years.  His background covers dozens of product and service categories and over 50 countries.  Kevin began his marketing research career on the client side in New York, and he has broad experience with the A-Z of marketing research.  This includes advanced analytics and new product development for Nielsen Customized (CR) and Research International.  He founded his consultancy, Cannon Gray, in 2008 and works with clients, marketing research agencies, consultants and ad agencies located in many regions of the world.  His chief focus is on providing marketing science and analytic support to enhance decision making.  He’s a strong believer in taking advantage of new research tools and data to their fullest…but without letting the tools and data become the ends rather than the means.

Koen Pauwels is Professor of Marketing at Ozyegin University, Istanbul and Honorary Professor at the University of Groningen.  He received his Ph.D. from UCLA, where he was chosen “Top 100 Inspirational Alumnus” out of 37,000 UCLA graduates. Next he joined the Tuck School of Business at Dartmouth, where he became Associate Professor in 4 years and received tenure in 6. Prof Pauwels is Associate Editor at the International Journal of Research in Marketing and has received the most prestigious awards for more than 30 top publications. He consulted large and small companies across 3 continents, including Amazon, Credit Europe, Inofec, Heinz, Kayak, Knewton, Kraft, Marks & Spencer, Nissan, Sony, Tetrapak and Unilever.

 

The authors would like to thank Raoul Kübler, Professor of Business Administration and Marketing at Ozyegin University for his helpful comments.