Social Data’s Predictive Power Unlocked

Editor’s Note: Every step in the innovation journey has always been a great challenge. Finding a “big idea” for something new is among the most difficult.  We’ve all seen products that have seemed to come out of nowhere and become big sellers. Wouldn’t it be great to see big opportunities before others can? In a very interesting piece, Jeni Chapman takes us through a new approach to early detection of innovation opportunities using analysis of social media data.  I think this could be something powerful.

---

In today’s fast-paced world every industry is being disrupted in some way and cultural trends are crossing continents with a speed never seen before.  Innovation teams are being tasked to find new ways of keeping a pulse on consumer trends, in order to place the most informed innovation bets for their businesses. 

Unilever on Trend Predictions

Tony Cece, Senior Data Science Manager at Unilever talked about this with us in a panel we recently hosted with our partner, Converseon.AI, on trend predictions. Unilever sees a need to develop innovative ways to unlock the power of trend prediction using social media data in a way that is scalable and predictive. 

“…As a big corporation, we are well aware of the rising trends towards micro-segmentation and personalization… people now have more choices and know much more about products….before we needed to meet the average needs of the average consumer while today that has all changed and we need to innovate fast.” 

Tony continued to share that while social media data is certainly extensive and “real-time” – using it in the right way and extracting the information in the right way, was not an easy task.  He emphasized however that if you want to be a leader rather than just jump on the bandwagon of an existing trend, boldness was key. 

The Power of Social Data

What is going to help companies make these bold decisions? Here is what Tony thought were the benefits of finding a way to unlock the power of social data: 

“Mining social media data with a level of confidence that comes from the combination of AI-enabled machine learning filters and statistical rigor –  that allows a company to make the right decisions on what early-stage trends to take advantage of so you have the opportunity to strategically position yourself in the marketplace.”

This was certainly not an easy challenge that we took on. Our journey over the last couple of years has been focused on first, getting the right data and second, proving the concept with clients. We achieved step 1 by partnering with Converseon.AI, who has invested significantly in AI-enabled machine learning classifiers and could deliver to us research-grade data. We were then able to take these data and apply our expertise in Bayesian-based time series modeling and analytics to develop trend predictions based on statistically rigorous, time series-based modeling platforms. 

Trend Signals

Mark Garratt, the co-founder of in4mation insights, has led the development of our signal spotting prediction engine tool. He shares several critical key steps to unlocking trend signals in social data :

• One – it is critical to establish and understand the seasonality trend of an ingredient. This is something you need to control for in a model but at the same time, it is a key insight beacon to guide product launch planning. In addition, unlocking this index allows you to understand if a trend is over-indexing or under-indexing in momentum within key seasonality peaks and valleys.

In the bottom chart, you observe the seasonality trend of pumpkin which peaks in the Nov/Dec, as you would expect. 

However, our adjusted signal trend chart above, reveals how in 2017, the actual trend of pumpkin for that year dipped.

• We have also discovered how to identify what we call “ingredient/flavor trend migrations” – a critical component for anyone managing the next sensory innovation opportunity.  For example, imagine you are managing innovation in the spirit category; wouldn’t you want to know what botanical trends from personal care or sparkling beverages will be migrating to alcohol in the next 6 months? Or vice versa?
• And the concept of ingredient/flavor trend flocking or said another way – “trends of a feather flock together.” Yes, we are discovering how ingredients and flavors group together, and this is leading to a whole new range of exciting insights like – to name just one example – what kind of ingredients are bound for a decline versus an upsurge? For example, we have seen that certain types of fruits are showing a downward trend and seem to part of a longer downward trend in the fruit smoothie category. 

This new approach has been part of a multi-year development journey to power a new way to build confidence around innovation bets and trend spotting.  The future is here and we are unlocking the predictive power of social media data!