The many faces of research on face perception

Anthony C. Little, Benedict C. Jones, Lisa M. DeBruine

Abstract

Face perception is fundamental to human social interaction. Many different types of important information are visible in faces and the processes and mechanisms involved in extracting this information are complex and can be highly specialized. The importance of faces has long been recognized by a wide range of scientists. Importantly, the range of perspectives and techniques that this breadth has brought to face perception research has, in recent years, led to many important advances in our understanding of face processing. The articles in this issue on face perception each review a particular arena of interest in face perception, variously focusing on (i) the social aspects of face perception (attraction, recognition and emotion), (ii) the neural mechanisms underlying face perception (using brain scanning, patient data, direct stimulation of the brain, visual adaptation and single-cell recording), and (iii) comparative aspects of face perception (comparing adult human abilities with those of chimpanzees and children). Here, we introduce the central themes of the issue and present an overview of the articles.

1. Introduction

Faces have long been a source of interest to scientists in a wide range of disciplines. In recent years, this breadth of interests, approaches and expertise has led directly to rapid advances in our understanding of many different aspects of how we perceive and process faces. Of course, it is not just scientists who are interested in faces. Television and movie cameras focus our attention on faces, and faces cover our magazines and dominate both ancient and modern works of art. Our sensitivity to face-like patterns is highlighted when we see faces in many everyday shapes, such as in clouds and chance arrangements of objects (figure 1).

Figure 1.

Examples of how the human mind readily perceives faces in chance arrangements of objects. Images from Flickr users eworm, Aquario, mallol, Listener42 and vectr (provided under Creative Commons licences).

Faces come in a remarkable range of shapes and sizes and are covered with an incredible number of muscles, adding to facial complexity [1]. Moreover, the importance of faces in human life is highlighted by a great deal of empirical research. Human infants only minutes old attend particularly to face-like stimuli relative to equally complicated non-face stimuli [2,3]. We rely on faces to recognize the myriad of individuals we encounter in our lives [4] and, consequently, thieves, bank robbers and super heroes wear masks to conceal their identities. Our faces also display our feelings about past, current and future events through emotional expressions [5,6].

The face is usually the first type of visual information available to a perceiver and is visible continually through almost all types of interaction. Consequently, a fundamental question in social perception, and thus in understanding the social world of humans, is exactly what information a human face conveys. Although other information may be more meaningful than that acquired from faces (such as whether a person is aggressive, which can be indicated by how violently they behave towards others in a number of different situations), it takes more time to acquire (e.g. repeated observation over time). Consequently, humans readily draw a number of conclusions about the personality attributes, appearance, emotional states and preferences of complete strangers solely on the basis of facial cues.

This theme issue focuses on bringing together recent advances in our understanding of face processing by encompassing nine topics across three aspects of face processing: (i) facial cues as social signals, (ii) neuropsychological approaches to face perception, and (iii) developmental and comparative approaches to face perception. While each article can be read individually as a primer on specific topics, we hope readers will take time to read the papers that may not be in their specific interests.

2. Facial cues as social signals

In the first paper, Little et al. [7] address the issue of facial attractiveness. What appears to be a simple judgement has generated a wide range of research and the answer to what makes a certain face particularly attractive is a complex one. People are quick to make attractiveness judgements, but it is difficult to say just what such judgements are based on. The authors start by presenting several well-researched facial cues that are known to be associated with attractiveness in humans, such as symmetry, averageness and perceived personality. The framework is an evolutionary one, and, while such approaches have been criticized in the past for predicting inflexible, invariant behaviours and preferences, their paper outlines how evolutionary thinking actually leads to compelling predictions about individual variation. They then document several important sources of such individual differences in face preferences, including factors such as hormone levels and fertility, own attractiveness and personality, social and environmental context and visual experience and social learning. Their review highlights flexible, sophisticated systems that appear to function to maximize evolutionarily relevant benefits of both our choice of mates and decisions about the attractiveness of other types of social partners.

While Little et al. [7] focus on perceptions of attractiveness, Said et al. [8] discuss the role of emotion perception in social judgments. Early models of face processing proposed that functionally distinct aspects of faces were processed by different cognitive routes and tended to assume that facial expressions of emotion were processed somewhat independently of other facial cues (e.g. [4]). They challenge this view and, perhaps more importantly, question the very notion that emotional expressions are functionally distinct from other aspects of face processing. In particular, they present behavioural and neurobiological evidence that generalized responses to emotional expressions influence evaluations of neutral faces and that evaluations of neutral faces and facial expressions are, at least partly, influenced by shared mechanisms. This work highlights the importance of integrating research on expression perception and evaluations of neutral faces and, to that end, outline a common framework that would support this integration. By combining two aspects of face research that have traditionally been studied somewhat independently (evaluations of neutral faces and processing of emotional expressions), they identify a relatively new direction for face perception research that could provide important new insights into the neurobiology of face evaluations and person perception.

Complementing both Said et al.'s [8] emphasis on the role of emotion perception in social judgments and Little et al.'s [7] emphasis on individual differences in face preferences, Calder et al. [9] examine individual differences in responses to facial expressions of emotion. They first review research on individual differences in neural and behavioural responses to facial expressions of emotion, focusing on findings suggesting that anxious and non-anxious individuals differ in their responses to threat-related facial expressions, such as anger and fear. Next, they discuss research linking individual differences in reward drive, a factor that may play a central role in obtaining or maintaining important resources such as food, reproductive partners, territory and social status, to variation in neural responses to angry facial expressions. Finally they discuss the importance of examining more complex aspects of the patterns of activation in response to facial expressions, such as the correlations among responses in different brain regions, in order to gain a better understanding of the interplay between personality and neural responses to facial expressions. Calder et al. emphasize that, though not without its critics, this approach does appear to have a lot to offer to research on complex aspects of face perception, provided it is used correctly.

In contrast to the focus on how people perceive and respond to unfamiliar individuals' faces [79], Jenkins & Burton [10] focus their attention on how people indentify specific individuals from facial cues (i.e. face recognition). They discuss various psychological theories and models that might explain how we are able to recognize so many different individuals under such a diverse range of adverse conditions while finding the process effectively effortless and then go on to describe how even relatively slight changes in appearance can cause severe problems for computer-based face recognition systems. Their elegant and engaging solution to this difficulty is to apply techniques that are used as research tools in other areas of face research, such as computer graphic methods for manufacturing face prototypes, to make computer-based face recognition more ‘human-like’. They discuss evidence from recent studies that highlights the utility of this approach, and their paper is noteworthy as an excellent example of an aspect of face research that uses the science of face perception to address real-world problems, such as the need for fast and effective face identification systems.

3. Neuropsychological approaches to face perception

Webster & MacLeod [11] present a timely review of the phenomenon of visual adaptation to faces (i.e. after-effects). In essence, how we perceive faces can be strongly affected by our recent experience with other faces. The effect of exposure is simple and powerful: if you see a face distorted and are asked to judge another face immediately afterwards, the second face is perceived to be distorted in the opposite direction. Such after-effects are observed for other visual stimuli, such as colour, whereby exposure to red, for example, causes the brief perception of the opposite colour, green. They explore the complex higher level aspects of face adaptation, factors such as relatively sex- or race-specific adaptation, and how these may inform the neural coding schemes underlying the visual representation of faces. By comparing adaptation effects for faces and colour, they suggest that even though face after-effects may in part reflect response changes at high and possibly face-specific levels of visual processing, there are many parallels with colour after-effects, and that while faces might be ‘special’ in terms of their dedicated neural architecture and representation, the underlying mechanisms of representation might be similar to the principles used to process other types of lower level visual stimuli.

In their review, Atkinson & Adolphs [12] address various approaches to studying face processing in the brain, highlighting that brain regions involved in face processing are widely distributed and distinguishing between higher level perceptions (judging identity, emotion from faces) and lower level perceptions (discriminating faces from objects). That higher level perceptions involve the occipital face area (OFA) and lower level perceptions often do not, suggests a somewhat hierarchical model of face perception in which the OFA is central to higher level perception. Atkinson & Adolphs question this view by reviewing lesion, transcranial magnetic stimulation and fMRI findings that instead suggest that higher level face perception reflects a complex interplay between different brain regions. They highlight that because the demands of the task and environmental and social context affect face perception; face perception cannot result from a simple extraction of cues by different regions. Instead they propose a more interactive model in which higher level face perception abilities depend on the interplay between several different neural regions and in which the specific interactions may vary according to task or on context.

Bridging the neuroscientific and comparative approaches, Barraclough & Perrett [13] present a review of the cellular coding of faces in non-human primates. A large amount of research on monkey neurophysiology underpins models of face perception, explanations of perceptual after-effects from viewing particular types of faces and interpretations of human neuroimaging. They focus on cells that preferentially respond to faces, especially for the social domain, such as identity perception and by comparing results from the limited amount of cell recording done in humans are able to conclude that there are both similarities with equivalent cells recorded in non-human primates and also that there are potential differences. They note that one topic for future research lies in the time courses of adaptation and the duration of after-effects, suggesting that there are multiple cellular mechanisms at play. They also highlight that cells responsive to faces are sensitive to multiple modalities, leaving the potential for interactions. For example, interaction between faces and body perception or faces and voice perception will probably prove an important avenue for future research on face perception.

4. Developmental and comparative approaches to face perception

In their paper on face processing in infants, Heron-Delaney et al. [14] emphasize the importance of being able to distinguish between members of your own and other species. First, they review previous research demonstrating that the ability to distinguish human from non-human faces emerges early in infancy. They then describe new empirical research using preferential looking times that demonstrates neonates, 3-month-old infants and 6-month-old infants can distinguish between human and non-human (monkey and gorilla) faces. Although the 3-month-old and 6-month-old human infants were able to distinguish between human and non-human bodies, neonates were not. These findings present novel converging evidence that species discrimination from facial cues emerges very early in infancy and, perhaps more importantly, suggests that species discrimination from body cues emerges considerably more slowly. They discuss the implications of these findings for models of face processing in which templates of evolutionarily relevant stimuli supporting core aspects of social perception are thought to develop during infancy and in which different levels of experience with the faces and bodies of conspecifics drive different developmental trajectories for face and body processing. Fundamentally, their paper emphasizes that core aspects of face processing are evident in children at a very young age.

Whereas Heron-Delaney et al. [14] compare and contrast face- and body-processing abilities in individuals of different ages (e.g. neonates and older infants), Parr [15] compares face-processing abilities in different primate species, discussing research demonstrating that, like human infants, infant macaques and gibbons prefer to look at faces relative to non-face objects and that experience is important for discriminating between faces from different categories (e.g. faces of different species) in many primate species. While these findings emphasize striking similarities in face-processing abilities across species, she also discusses other research that highlights possible differences in how different species of primates process faces. For example, inversion effects on face processing, whereby inverting a face image dramatically impairs many fundamental aspects of face processing (e.g. [16]; figure 2), have been widely reported in humans and chimpanzees, but appear to be considerably less robust in monkeys. Parr then goes on to discuss evidence for other differences and similarities in primates' ability to individuate faces from the same category, including new research suggesting that the sex of the faces used as stimuli, a factor that is frequently not even reported in face recognition work with non-human primates, may be critical for individuation in monkeys. Finally, she discusses the engaging possibility that these sometimes striking and sometimes subtle differences in face-processing abilities across primate species may be systematically related to variation in their social structures.

Figure 2.

Which face is more attractive? Now turn the faces upside down. The well-known Thatcher illusion highlights how human face processing is specialized for upright faces and how inversion disrupts many fundamental aspects of face processing.

5. Integration

Overall, this collection of papers places face processing in evolutionary, functionalist and developmental frameworks, highlighting both the important social cues contained in faces and the neural architecture supporting perceptions of these cues. Bringing together these topics now is important, since the potential for integration among these three topics is great but, to date, concrete examples of actual integration are far less common than one might hope and/or expect. While face perception research is carried out by scientists working in diverse disciplines, researchers in different disciplines can frequently be largely unaware of work of potentially great relevance to their own that is being carried out by researchers in other areas, using other methods or working within different frameworks. This theme issue showcases the diversity, and ultimately unity, of face perception research, which we sincerely hope will lead to new advances in the field and give researchers a flavour of the range of face research being carried out by others.

The collection of papers presented here brings together research from a variety of different fields, most notably biology, psychology, primatology and neuroscience. The collection also presents a wider variety of techniques and approaches than are commonly used in a single field. The contributors use a variety of paradigms, including experimental approaches, clinical data, non-human animal data, computer analysis, electrophysiology and brain-imaging methods. The different aspects of face perception covered in these articles are not often addressed, even in the same types of journal. For example, social perception, neuroscience, comparative and developmental research each tend to be published in their own, relatively specialized, journals. We hope that bringing together these topics will thus serve to expose individuals to a variety of ideas, issues and techniques they may not otherwise have considered. We suggest that it is the synergy of the different approaches that will be most useful in advancing the field of face perception and that the utility of reading this collection of papers is considerably greater than the utility of reading each individually.

Acknowledgements

We would like to thank all of the contributors to this special issue and the anonymous referees who read the papers. We also thank Joanna Bolesworth and Claire Rawlinson at the Philosophical Transactions office for their patience and help in putting this issue together. Anthony Little is supported by a Royal Society University Research Fellowship.

Footnotes

References

View Abstract