New Machine Learning Model for CEOs Facial Expressions

Nowadays, it is a standard that fillings such as 10-Ks and 10-Qs are analyzed with machine learning models. ML models can extract sentiment, similarity metrics and many more. However, words are not everything, and we humans also communicate in other forms. For example, we show our emotions through facial expressions, but the research on this topic in finance is scarce. Novel research by Banker et al. (2021) fills the gap and examines the CEOs facial expressions during CNBC’s video interviews about corporate earnings.

The authors utilize a conventional neural network for face detection and facial expression recognition to measure the dynamic hemifacial asymmetry of expressions. The idea is based on neuropsychology, which states that facial asymmetry induces distrust. Therefore, the crucial task is to find whether the CEO´s faces asymmetry influence stocks. The results support the theory from neuropsychology. Dynamic hemifacial asymmetry is negatively connected with the three-day cumulative abnormal return after the interview. There is also a relation with bid-ask spreads suggesting that investors opinions are dispersed following the event and tend to be larger. Moreover, the distrust is even stronger when the company has a weaker information environment (high volatility and forecasts dispersion). Overall, several takeaways from the paper could be utilized with already established approaches for textual analysis. 

Authors: Rajiv D. Banker, Hui Ding, Rong Huang and Xiaorong Li

Title: Market Reaction to CEOs’ Dynamic Hemifacial Asymmetry of Expressions — A Machine-Learning Approach



Neuropsychological studies propose that listeners unconsciously assess speakers’ trustworthiness via their facial expressions. Building on this theory, we investigate how investors respond to CEOs’ dynamic hemifacial asymmetry of expressions (HFAsy) shown on CNBC’s video interviews about corporate earnings. We employ a machine-learning approach of face-detection and facial-expression-recognition based on conventional neural network to measure CEOs’ dynamic HFAsy. Consistent with the neuropsychological prediction that facial asymmetry induces distrust, we document that the stock market reacts negatively to the CEO’s HFAsy shown on the interview video. We also find that the abnormal bid-ask spread around the interview date is positively associated with the CEO’s HFAsy. We further show that these effects are more pronounced for firms with weaker information environments. Finally, we document that analyst forecast revisions are negatively associated with CEOs’ HFAsy. Overall, our study provides evidence that investor trust and trading behavior are affected by the dynamic hemifacial asymmetry of expressions appeared on CEOs’ faces.

As always we present several interesting figures and tables:

Notable quotations from the academic research paper:

“Hemifacial asymmetry of expressions, defined as the expression intensity or muscular involvement on one side of human face (“hemiface”) relative to the other side, is a welldocumented feature in the neuropsychological literature (Darwin, 1965; Borod et al., 1983; Borod et al. 1997). A high hemifacial asymmetry is neuroanatomically linked to an individual’s deliberate facial expression (Kahn, 1964; Tschiassny, 1953), because these expressions are contralaterally innervated by cortical structures through the monosynaptic connections within the pyramidal system, while spontaneous ones are presumed to be bilaterally innervated by subcortical structures through the multisynaptic extrapyramidal system (Borod and Koff, 1984). Due to its neuroanatomical association with expression fabrication, hemifacial asymmetry of emotion has been shown to trigger observers’ neural pathways of distrust (Buisin et al., 2018; Okubo et al., 2017; Carr et al., 2014; Okubo et al., 2013).

Drawing on prior evidence from neuropsychological studies, we investigate whether CEOs’ dynamic HFAsy induces investors’ distrust and affects their trading behavior. We first examine the stock market reaction to CEOs’ dynamic HFAsy shown on interview videos. Since CEOs’ high HFAsy induces investors’ distrust, investors may perceive these CEOs as selfserving managers who tend to sacrifice shareholders’ interest for their personal benefit.

We hypothesize that investors respond negatively to CEOs’ high HFAsy appeared on face-to-face interviews about corporate earnings. Next, we investigate whether investor disagreement on earnings news is more severe for companies with higher HFAsy CEOs. Since investors are skeptical about high HFAsy CEOs’ integrity, they may place less weight on public information disclosed by these CEOs and focus more on private information instead. As relying more on private information leads to higher disagreement among investors, we hypothesize that investors’ divergence of opinions on earnings news is larger for firms whose CEOs exhibit higher HFAsy on video interviews.

To capture CEOs’ HFAsy, we obtain a sample of 205 CEO-video-interviews on corporate earnings news between 2017 and 2019 from the CNBC website. We use the machine learning algorithm of convolutional neural network to detect CEOs’ faces and code the emotions on their left and right hemifaces as follows: First, we use python to scrape the interview videos and extract images from these videos. Second, we use the convolutional neural network algorithms to detect CEOs’ faces from these extracted images. Third, we segment and mirror CEOs’ left and right hemifaces and code seven emotions on CEOs’ mirrored left and right faces. These seven emotions are the basic human facial expressions proposed by Ekman and Friesen (1971)–angry, disgust, fear, happy, sad, surprise, and neutral. Finally, we aggregate CEOs’ hemifacial asymmetry of these seven emotions on each image in each video to obtain the measure of CEOs’ dynamic HFAsy.

We use cumulative abnormal returns (CARs) and abnormal bid-ask spreads measured over a three-day window around the video interview date to capture market reaction and investor disagreement, respectively. Stock returns reflect the average investor’s belief (Ball and Brown, 1968; Kotharis and Wasley, 2019) while abnormal bid-ask spreads capture the divergence of investor opinions about the interview (Blankespoor et al., 2014; Coller and Yohn, 2006). Consistent with our prediction, we document a significantly negative relation between the short-window CARs and HFAsy after controlling for various firm-, CEO-, and videospecific characteristics. We also document a significantly positive association between the short-window abnormal bid-ask spreads and CEOs’ HFAsy. Overall, the evidence suggests that the average market reaction is negative while the divergence of investor opinions is high for firms whose CEOs exhibit strong HFAsy on video interviews about corporate earnings, indicating that investors are skeptical about the integrity of CEOs with strong HFAsy.

Next, we examine whether investors’ assessment of CEOs’ HFAsy is affected by firms’ information environments. Neuropsychological literature suggests that prior experience affects an individual’s perception of trust (Chang et al., 2010; King-Casas et al., 2005; Axelrod and Hamilton, 1981). Consistent with this notion, we expect that investors’ distrust towards CEOs’ HFAsy is stronger for companies with a history of weaker information environments. Using prior quarterly return volatility and analyst earnings forecast dispersion to measure firms’ past information environments (Billings et al., 2015; Bushee and Christopher, 2000; Lehavy et al., 2011; Barron et al., 1998), we show that the impact of HFAsy on CARs and abnormal bid-ask spreads is more pronounced for firms with weaker information environments than for those with stronger information environments.”

Are you looking for more strategies to read about? Visit our Blog or Screener.

Do you want to learn more about Quantpedia Premium service? Check how Quantpedia works, our mission and Premium pricing offer.

Do you want to learn more about Quantpedia Pro service? Check its description, watch videos, review reporting capabilities and visit our pricing offer.

Or follow us on:

Facebook Group, Facebook Page, Twitter, Linkedin, Medium or Youtube

Share onRefer to a friend

Subscribe for Newsletter

Be first to know, when we publish new content

    The Encyclopedia of Quantitative Trading Strategies

    Log in

    1. bi-weekly algo & quant academic research insights
    2. tips on new trading strategies & Quantpedia features
    3. notifications about exclusive Quantpedia offers & promos
    and be the first who get:
    - bi-weekly newsletter with academic research insights
    - special report about top strategies each month
    - exclusive discount offers & coupons