When Recognizing Emotions, Is “What You See What You Get?”
Every day, most humans look at other people’s facial expressions and infer what they are feeling. With one look they feel certain, without thinking about it, that the other person is, say, sad. These judgments are usually correct, or close enough to be getting on with. The whole process happens almost instantly.
It’s also automatic: people often can’t explain how they know. “How do you know what the other person is feeling?” might even sound like, “How do you know the sky is blue?” It feels like there’s nothing to explain; you just “see” it.
Because emotion recognition feels so easy and automatic, many people assume there is one set of emotions that all people recognize and express in the same way.
That assumption matters, because it influences the way we teach autistic people to recognize other people’s emotions. It underlies charts like these:
These charts vary in realism and complexity, but they all assume that anyone who feels, say, “hopeful” in real life will look just like the “hopeful” person in the chart. And, therefore, all people who feel hopeful will look like each other.
But, are emotions really that simple?
Is what we see really what we get?
The Search for Basic Emotions
The extreme version of “what you see is what you get” thinking is the theory of “basic emotions.” The idea is that evolution has left humans with a few emotions that we are all wired to express in the same way, regardless of culture. These “basic emotions” can be identified by measuring the movement of the face muscles. There is convincing evidence for basic emotions: the expressions of a person in a remote stone age Papua New Guinea culture can be recognized by Americans, and vice versa.
Some people even believe that these “basic emotions” are like primary colors that are put together to create all the nuanced emotions people feel.
The Complexity of Basic Emotions
If basic emotions are so simple and universal, it should be easy to tell which ones exist, and how many there are. Yet researchers disagree on how many, and which, emotions are “basic.” For example, disgust, surprise, trust, and anticipation are considered basic in some theories, but not others.
Even emotions that have been seen as basic, for good reasons, are recognized by some cultures but not others. For example, fear, surprise, anger, and disgust have all been considered basic emotions. Yet different cultures may not distinguish between facial expressions of fear and surprise, which share signals such as wide open eyes. They also may not distinguish between anger and disgust, where the nose wrinkles.
This example has been interpreted as a sign that at least some of the emotions involved aren’t actually basic. Some conclude that there are only four emotions that all humans recognize and express in the same way, regardless of culture. Yet, that hasn’t become the consensus. A pioneer in emotional expression research, Paul Ekman, still posits seven basic emotions. Meanwhile, the five-emotion model that Pixar used in the movie Inside Out is still popular.
For our purposes, what matters is that there are very few basic emotions, and even these should be taken with a grain of salt.
There may also be variations on how the basic emotions are expressed.
Paul Ekman identifies seven basic emotions (anger, contempt, disgust, enjoyment, fear, sadness, and surprise). However, he views these as families of expressions that share certain characteristics. For example, the disgust family involves scrunching up the nose. These shared characteristics are universal. However, each family includes many variations that differ depending on the culture and the situation that triggered the emotion. For example, Ekman identified 60 expressions for anger, including resentment, indignation, and desire for vengeance.
Imagine an emotion chart showing all 60 versions of anger. To convey all the feelings one might see someone express in life, we’re going to need a pretty big chart.
Endless Emotions
The basic emotions are only a tiny sample of the immense palette that humans feel. Many emotions have a social or cultural aspect, such as guilt, embarrassment, pride, and homesickness.
Cultures vary in what non-basic emotions they choose to distinguish and name — much as they vary in which colors they categorize and name. For example, Spanish has the word “duende,” the state of being moved by art, while English has no word for this emotion.
Humans also feel multiple emotions at the same time, in different proportions, which adds complexity. Some researchers, like Paul Ekman, account for this. However, the typical emotion recognition chart does not.
The Meaning of Expressions Varies Across Cultures…and Neurotypes
Evidence suggests we cannot assume that each expression has a one-to-one match with an emotional meaning, or vice versa. The same expression can correspond to different emotions, depending on culture …and most likely, neurotype.
Consider smiling. At least one version, the Duchenne smile that involves the eyes, is often considered synonymous with happiness. In fact, the meaning of a smile varies by culture.
Americans see smiling people as friendly and approachable, and may even object to people not smiling “enough” (an infamous example: some men demand that women smile). By contrast, people in some Eastern European countries say that only fools and Americans smile, and a proverb loosely translates, “laughing for no reason is a sign of stupidity.”
Smiling also varies by neurotype. Autistic people, as a group, are more like Eastern Europeans: they may have a resting neutral face and not smile when happy. In American culture, they may be misinterpreted as unfriendly or angry. They may even be told to smile more.
In short, even if we assume that there is a set of universal basic emotions, the rest vary too much to assume every human expresses the same set of emotions in the identical way.
I suspect that most emotions are complex enough that even neurotypical people don’t necessarily always recognize them correctly.
Display Rules: Feeling One Emotion While Showing Another
Cultures differ in their norms for who can express which emotions, how intensely, in what situations. These are called “display rules.” Because of these cultural norms, depending on factors like whether an authority figure is present, the same situation might evoke the same feelings, but different facial expressions.
For example, when Japanese students watched films designed to evoke fear and disgust with a scientist in the room, they smiled. However, when they watched these films in private, their faces showed their fear and disgust.
In these situations, what we see is definitely not what we get.
Neurotypicals can take the display rules into account. They can, for example, tell a polite smile from true happiness.
That means they’re not “just seeing” — they are interpreting what they see.
And when there is interpretation, there’s a possibility of getting it wrong.
We need to open ourselves to the possibility that what we see in another person’s face isn’t necessarily what they actually feel.
The way people express emotions can’t be reduced to a simple chart.
What’s Next
On this blog and in my upcoming posts for Kind Theory, I will share more evidence that people with different neurotypes express emotions differently. I’ll also explain why knowing this, and being open-minded, is so important.
In the meantime, you can test your own accuracy at recognizing emotions. Every so often, check in with people you trust, asking what they feel. How close are your guesses?
