What the emotion detector measures
After finding a face, the tool passes it to a small expression network from face-api.js. The network was trained on photos labelled with seven basic expressions, and for each face it returns a score for every one of them. The scores add up to the whole, so you can see whether a face is clearly one expression or a blend of two or three.
The features that drive those scores are the visible ones: how far the mouth corners lift or drop, whether the lips part, how wide the eyes open, whether the brows rise, lower or pinch together, and whether the nose wrinkles. That is why a smile with the eyes narrowed scores differently from a smile with wide eyes, and why a raised-brow look can tip towards surprised.
Expression is not emotion
This is the most important limit to understand. A face shows an expression; a person feels an emotion. People smile when they are nervous, look neutral when they are delighted, and frown when they are concentrating. Cultural norms also shape how and when people show expressions. The detector can tell you what a face looks like in one frame. It cannot tell you what anyone is feeling, and it should never be used to judge mood, honesty, attention or intent.
Getting a clean reading
Neutral is the most common result on everyday photos, because most faces at rest are close to it. If you are testing the detector, exaggerated expressions show the clearest differences between the seven scores.
- Face the camera; side views hide half the mouth and one eye.
- Use even light so the brow and mouth corners are visible.
- Keep hands, hair and masks away from the mouth and eyes.
- On webcam, hold an expression for a moment so the scores settle.
- Small faces in group photos give rougher scores; crop closer if you can.
Fair uses
Practising expressions for acting or presentations, checking which of several photos looks friendliest, teaching how expression classifiers work and seeing where they fail are all good uses. Monitoring people, scoring job candidates or students, or making decisions about anyone from their expression are not, and in several jurisdictions emotion recognition in workplaces and schools is restricted by law.