Register with us
Oops! Something went wrong while submitting the form.
Welcome
Hi human, you are not logged in.
Oops! Something went wrong while submitting the form.
The Behavioral Barometer
The latest and greatest from your human experience guides at Element Human
A study of 11,000+ face videos reveals that automated facial coding predicts sales effectiveness better than purchase intent surveys—and captures the emotional dynamics that traditional methods miss.
This beautiful research PDF is brought to us by our friends at MIT, Affectiva, and Northeastern University. The research strongly suggests that emotions in advertising, particularly as measured by facial coding, do drive sales. It's not just about whether an ad evokes emotion, but how and which emotions are evoked, and how those emotions change over time. Using tools like facial coding can provide valuable insights that go beyond traditional surveys to better predict ad effectiveness.
For decades, marketers and advertisers have instinctively believed in the power of emotion to connect with consumers and drive purchasing decisions. It is widely accepted that human emotions play a significant role in influencing our choices, from the content we consume to the products and services we buy.
Our past and current emotional experiences subconsciously bias our decision-making, making emotions a crucial influencer. Advertisers aim to create ads that surprise, entertain, or even evoke tears to help consumers remember products, build positive brand associations, and ultimately drive sales.
But the critical question remains: Are ads that are successful in evoking emotions also successful in driving sales? This question was at the heart of a research partnership between MARS Marketing and Affectiva.
Traditionally, assessing advertising effectiveness and predicting sales has relied heavily on surveys. Respondents watch an ad and then answer questions about brand awareness, message persuasion, and purchase intent. These cognitive assessments are then correlated with sales data. However, this approach often misses people's visceral emotional responses, which play an equally important role in influencing memory and purchase decisions.
While manual facial coding has shown promise in correlating with sales effectiveness, it is time-consuming, expensive, and difficult to scale. Other methods that focus on salient ad features (like package shots or celebrity appearances) fail to factor in viewers' cognitive or emotional responses6. Until recently, measuring emotional responses to advertising in a passive and scalable way remained a significant challenge, limiting large-scale studies on the relationship between emotions and sales success.
The need for an objective, unobtrusive, and scalable measure of emotion led to the development and application of automated facial coding technology. Facial expressions have long been considered the universal language of emotion, recognized across cultures, ages, and genders9. Building on systems like the Facial Action Coding System (FACS), automated facial coding offers a breakthrough.
Affectiva's automated facial coding technology provides an objective, cost-effective, and scalable way to measure emotions. Unlike self-report methods, it enables the remote measurement of subtle and nuanced expressions on a moment-by-moment basis8. The technology captures facial action units and identifies main dimensions of emotion, such as valence (pleasantness) and arousal (activation), as well as discrete emotional states like enjoyment, surprise, disgust/dislike, confusion, and skepticism.
How it Works: Automated facial coding processes each frame of a face video to locate main facial features (e.g., mouth, eyebrows). It then uses the movement, shape, and texture composition of these regions to identify specific facial action units, such as an eyebrow raise or a smirk. Machine learning classifiers map these facial textures and movements to emotional states. The system is trained on a vast repository of real-world facial data from diverse age ranges, ethnicities, and cultures, ensuring robustness to challenging conditions and achieving high accuracy (over 90% generally, with a 97% accurate smile detector). The outputs are moment-by-moment measures, averaged across viewers for an ad.
To systematically explore how emotions in advertising drive sales, a large-scale study was designed to collect facial emotion responses to ads for which sales data was also available. The primary goals were to:
The study involved over 11,000 face videos from 1,153 panelists who watched 115 ads from 70 unique brands. These ads were aired between 2001 and 2012 in the United States, United Kingdom, France, and Germany, representing four product categories: pet care, instant foods, chocolate, and chewing gum15....
Sales performance indices were obtained from single-source data, reflecting the sales lift directly associated with a TV ad, after accounting for other influencing factors. Ads were categorized as having good, average, or bad sales effectiveness, providing a standardized measure for analysis.
Participants watched ads from their homes using a browser with Flash support and a webcam, making data collection scalable and allowing for a broader, global demographic sample. The study design minimized memory bias by splitting the survey into two parts with a minimum three-day delay between initial brand questions and the main ad viewing session. Importantly, 87% of viewers reported feeling "very comfortable" to "neutral" during the study, and 68% reported no difference in their behavior compared to typical viewing.
The research aimed to determine if emotional responses, especially their dynamics over time, could predict short-term sales performance. The study focused on comparing the predictive performance of:
The study found that the smile/enjoyment and dislike/disgust responses were most correlated to sales success. To analyze these, the per-respondent smile and disgust profiles were aggregated for each ad, partitioned into 10 segments, and then used to calculate positive and negative emotion values.
The results demonstrated compelling evidence:
These findings validate two key hypotheses:
The positive and negative emotion measurements, specifically the trend (gradient) of the smile response and the level of disgust expressed, are predictors of sales effectiveness. Increasing amusement throughout an ad was a predictor of success, while the presence of negative emotions was inversely correlated to sales. This high temporal resolution is difficult to obtain from self-reported measures.
The improvement in classification accuracy between models using self-report versus facial expression metrics shows that automatically coded spontaneous facial actions were more telling of short-term sales effectiveness than self-reported purchase intent or brand likability. While combining both sources of information led to a marginal increase, its statistical significance needs further testing.
The research illuminated the importance of facial emotion dynamics through several case studies.
The study analyzed 74 videos across 37 brands with 17,544 participants to understand what makes ads successful on Twitch. It examined memory, emotion, and self-reported responses using methods like facial coding, eye-tracking, and implicit association tests.
These case studies underscore the benefit of incorporating facial coding, particularly its high temporal resolution, to accurately capture how expressed emotional responses evolve over time.
This pioneering large-scale study provides strong initial evidence that automated facial coding offers significant discriminative value for ad-driven sales effectiveness. This value is directly linked to logical, intuitive, real-time, moment-by-moment emotional response pattern.The implications for advertisers are profound:
Facial coding can enable live prediction of ad performance based on current emotional responses, informing decisions on when or if to air an ad, as well as media spend.
This technology can be powerfully employed throughout the advertising creation and editing process, improving the likelihood of ad success even at early mock-up stages.
In conclusion, emotions in advertising, when captured with scalable facial coding technology, are indeed indicative of ad success. Automated facial coding offers an objective, passive, and highly insightful approach to understanding consumer responses, providing actionable insights that complement and often surpass traditional self-report methods.
As research continues to analyze ad structure and generalize findings across diverse genres and markets, automated facial coding is set to become a routine and invaluable part of copy testing for brands aiming to optimize their advertising impact and drive sales.
"You guys are excellent partners, and we appreciate the innovativeness."
"We are proud to partner with Element Human to delve even deeper into the emotional impact of creator content on audiences and offer actionable insights, empowering brands to maximise the impact of their influencer marketing campaigns."
"Element Human has been an invaluable partner in showing the business impact creators can have on brand performance."
"They’re responsive, collaborative, and genuinely invested in our success, a rare combination that has made them a trusted partner of mine across multiple companies."
"Creator Economy PSA... Vanity metrics surpassed long ago. Effectiveness, impact and ROI are all measurable with partners like Nielsen, Element Human and Circana."
"Element Human was not just key for the BBC’s project but also was engaging and a great learning experience for me personally!"
.png)
