Categories
Research Methodologies
July 14, 2011
There appear to be three approaches towards social media measurement: Word Counters, Influence Enumerators, and The Linguists.
Editor’s note: I am thrilled to welcome our newest guest author on the GreenBook Blog: Michael Wolfe of BBDO. Michael is a renowned expert on marketing mix modeling, social media metrics, and advanced analytics and is a sought after speaker and contributor on those topics. He is doing some ground breaking work on developing new, ROI-driven models for social media analysis and it is a great honor to have him share some of his insights with us. Michael is getting to the meat of what drives investment in research in this post and poses a provocative challenge to market research at the end. He will also be sharing more details on his work with all attendees of the upcoming Market Research in the Mobile World conference next week in Atlanta. I think all of our readers are going to really enjoy this one!
Everyone with any sense of what is going on in the field of marketing right now knows that there is almost a mad frenzy about social media, the likes of which we have seldom seen. In fact, this frenzy has become so strong that it has spawned almost a gazillion self-proclaimed experts and gurus. It almost feels like the American Wild West, with the itinerant snake-oil salesman coming to town to sell us a bottle of his miracle cures.
As I have reviewed the literature on this subject, there appears to be three camps or approaches towards social media measurement. These are as follows:
Presently, there is a lot of confusion regarding which approach is most valid. The only research I can cite is from Bernardo A. Huberman of Hewlett Packard. Bernardo found that social media can predict such things as movie success. He also found that the quality of a message can pierce through the noise and influence an audience. He likewise found that the number of friends and followers on social media does not add much to the influence of the message.
The impetus for developing social media metrics is obviously targeted at developing a capability to measure the impact of social media conversations on people’s attitudes and behaviors. Significant here is the conversations on brands and how these conversations affect brand performance and ROI.
My own journey here tends to favor the linguistic approach. As a player in BBDO’s marketing-mix modeling consultancy, clients challenged me to come up with an approach to measure the impact of social media on their brands’ performance. My approach to this is fairly simple. That was to find a metric which correlates best with brand sales of any of our 8 blue-chip clients. In this journey, I tested and evaluated many of the “word counting” and “sentiment” metrics from major social media data vendors. Sorry to say, none of them were found to have greater net correlations to brand sales of about 0.15. Some of the sentiment indicators, in addition, even had counter-intuitive relationships with client brand sales.
Our journey finally led us to a metric called the Semantic Engagement Index or SEI as developed by a company called Linguistics Insights. This metric was developed by Dr. Peyton Mason and Dr. Boyd Davis. Davis is Professor of Applied Linguistics at the University of North Carolina. Together, Mason and Davis developed an approach which first parses large quantities of social media conversations into positive and negative sentiment groups. From there, each group is quantitatively scored using Linguistics science and rules. The scoring algorithm used defines consumer engagement by scoring conversations on two dimensions. Those dimensions are the degree of personalization and emotional effect contained in the conversational language.’
With our first client, we generated a time-based SEI metric. We plotted this metric against brand sales for one of our clients. As shown here, we plotted the ratio of positive to negative SEI. It’s correlation to brand sales was an amazingly high +87%. We have never found any single social media metric which tracks so closely and has such a strong connection to brand sales and performance!
We have since found similar and strong correlations of the SEI metric with brand sales of other clients. We obviously think we have made a very significant discovery here. Certainly, our efforts have enabled us to include the SEI metric in our marketing-mix models and directly measure the impact and ROI of social media on our respective client brands. Overall, the evidence so far seems to support the linguistics approach to social media measurement; but obviously we need more supportive evidence across many more brands in order to draw definitive conclusions.
In sum, our research has found at least three different approaches to developing social media metrics, all with the purpose of measuring its impact on brands and consumer attitudes. While I believe there is some evidence supporting the linguistics approach, I must say that the evidence to-date of which approach is superior is significantly lacking. The bottom-line here is that this really represents an opportunity for Marketing Research to step to the center and demonstrate leadership. Certainly, many researchers have access both to the metrics and brand sales data to do so. Whether they have sufficient moxie to do so remains to be seen.
Get Inspired. Stay Informed. Sign up for our free email newsletter.
Comments
Comments are moderated to ensure respect towards the author and to prevent spam or self-promotion. Your comment may be edited, rejected, or approved based on these criteria. By commenting, you accept these terms and take responsibility for your contributions.
Disclaimer
The views, opinions, data, and methodologies expressed above are those of the contributor(s) and do not necessarily reflect or represent the official policies, positions, or beliefs of Greenbook.
More from Michael Wolfe
Super Bowl Ads more effective than ever.
New software can potentially facilitate campaign analytics and driver analysis of panel data, cutting time and effort dramatically.
Are Millennials as unresponsive to advertising as commonly believed? Michael Wolfe challenges this assumption.
While Super Bowl ads do well on “likeability” and generating “buzz”, they didn’t fare as well on critical measures like “purchase intent”.
Sign Up for
Updates
Get content that matters, written by top insights industry experts, delivered right to your inbox.
67k+ subscribers