Is Your Marketing LLM Fit for Purpose?

Decades ago, my then employer took delivery of one of the world’s first mass spectrometer.  Mass spectrometers provide the precise gram-per-litre composition of a tested material. What do you suppose was the first item we tested? Not any of our company’s products but instead, Coca-Cola. The Coke formula was considered then and remains today, a closely guarded company secret.

The spectrometer identified ingredients such as sodium and phosphoric acid however, other ingredients were unidentifiable. The mass spectrometer could not identify what it had not been trained to recognize. This turns out to be an important corollary for the large language models (LLM) that are the engine of generative artificial intelligence (GenAI).

GenAI goes a step beyond traditional AI. AI machine learning based approaches applies classification and prediction to data. GenAI moves into creating original content such as text, images, audio, and video. The most common variant of GenAI is ChatGPT, which has an estimated 180 million users generating 1.6 billion visits per month. ChatGPT took just two months to arrive at 100 million users.

Efficacy Over Efficiency

In the first half of 2024, Forethought developed and tested GenAI animatics that performed in the top 10 of all ads tested by Forethought.

We have also tested GenAI produced animatics that failed on every count. Our conclusion has been that the quality of the prompt engineering and fine tuning of the LLM used to produce the animatics, is almost entirely responsible for the test outcome.

This is a critical point that will distinguish efficiency from efficacy. GenAI functions at lightening speed and produces output; however, the efficacy of the marketing communication is dependent on what domain specific knowledge the LLM has ingested.

LLM’s are architecturally designed to provide a probabilistic guesstimate. The fluency and plausibility of GenAI output will lead some to believe GenAI is factual. However, the quality of the LLM to address a specific marketing requirement such as the production of animatics or story boards is dependent on the quality of the data ingested by the LLM. If you understand the “wisdom of the crowds” assertion that in decision-making and problem-solving, crowds are collectively smarter than individuals, then you grasp the underpinning of LLM’s. At its core, GenAI is designed for prompt completion, conceptually not dissimilar to your mobile phone predicting your next word.

To get an optimal GenAI model there needs to be training of the LLM. Neural networks do relatively poorly in situations that are outside the training data. The more task specific knowledge the LLM ingests, the better the GenAI algorithm performs at the generative task. The foundation LLM is trained for general tasks. For knowledge-intensive tasks in particular, tasks that are outside its initial training data, the base LLM needs to be augmented. The following exhibit shows the usual ladder for augmented the LLM.

Figure One – Training your Generative AI

Prompt engineering is the process of guiding GenAI to produce the optimum output.  For example, you want the animatic to elicit love. Through prompt engineering, the context for eliciting love is provided to the GenAI. Continuing the example, a mother and father taking a new mortgage to enable a separate bedroom for each of the three children.

On the other hand, fine tuning is taking a general LLM and further training it, in this case to produce early phases of creative development such as story boards and animatics as briefing tools for creatives or production.

Tread Carefully From Here

The first thing you might be thinking about is how can the organization outsource some of the GenAI activity to accelerate its mastery?  n a previous article,¹ we spoke about immersing yourself in GenAI as the accelerating application of marketing LLM’s will create a revised set of winners and losers. The first principle of outsourcing is the organization should never outsource its core capabilities. LLM’s are bound to become a significant organizational core capability. Indeed, you should expect proprietary LLM’s to begin appearing in listed companies’ balance sheets alongside patents and other intellectual property.

If you wish to contain the use of your company secrets, that material should never be ingested by a non-proprietary LLM including those offered by third party suppliers. As an analogy, think of the blood–brain barrier. The barrier restricts any transfer between the circulatory system and the central nervous system therefore, protecting the brain from harmful substances in the blood.  In our context, the ‘harmful substance’ is the consequence of allowing your category knowledge to fall into the public domain.

In these early days of GenAI, the focus will be to have the best functioning generative algorithm which requires the LLM to be applied and trained using the best available corpus of knowledge.  The quality of the GenAI synthesized knowledge is dependent on the size and quality of the data that the GenAI algorithm learns from. The full potential is only realised when GenAI applications draw from specialized enterprise knowledge. In the marketing communications context, what then is the enterprise data GenAI needs to learn from?

The learning arising from past creative testing is often captured and held by third party research companies. For example, ‘the world’s largest ad testing database contains over 250,000 ads from over 100 markets.’ Organizations will now compete on having the best LLM and prompt engineers to drive the GenAI algorithm, and data such as the efficacy of past advertising, should be accessed by your GenAI and nobody else.

Training GenAI to Produce Great Marketing Communications

When it comes to GenAI, there is a giant chasm between science and science fiction. The ingestion of knowledge be it in the prompt or the fine tuning, must be science based. The greatest opportunity for a marketing LLM and GenAI is to ensure that science and not science fiction is building your capability.

Science fiction would be a continuation of the big idea generated purely through ‘intuition’. In contrast, the science are the prompts and fine tuning comprising the scientifically established category drivers of choice. GenAI hallucinations occur when the LLM is trained by inaccurate or incomplete data, or misunderstanding of the context.

All in all

Now and perhaps, forevermore, amongst enterprises’ most valuable assets will become its large language model driving its proprietary, generative artificial intelligence algorithm. The only relevant intellectual property protection for your organization’s data is company secrets.

GenAI should not be thought of as an invention but rather, a creative inventor. The inevitability of GenAI replacing elements of marketing creative is undeniable and it is, here and now.  It might start off as a collaboration but, in the end, the dramatic efficiency, rapid fire on-brief creativity and compliance of GenAI will see it eventually roll over the advertising industry the way Henry Ford rolled over the wagon wheel and buggy whip industry.

References 

1. https://www.greenbook.org/insights/customer-acquisition-retention/the-gen-ai-drizzle-becomes-torrential
2. https://www.kantar.com/inspiration/agile-market-research/how-link-elevates-the-world-of-ad-testing