In my previous post, I suggested that it is more helpful to look at story as a simulation rather than in terms of plot and character when it comes to thinking about sound cinematically, at least in the early stages. So today I’d like to explore the concept of simulation more in detail.
A simulation is a model of some aspect of reality that allows us to safely carry out experiments, learn new things, and practice skills that we can then apply to real-life situations. To that end, it needs to have some kind of interface that gives us the means to interact with the virtual world it represents. A flight simulator, for example, has a real cockpit that can tilt in any direction, that makes real sounds, and that has real controls linked to a computer system that interprets the pilot’s actions and moves the cockpit accordingly as the pilot reacts to faithfully recreated settings and situations such as airport, mountains, dangerous weather conditions, and emergency landings. All these elements allow the pilot to become immersed in the situation at hand and experience it ‘for real’.
Stories, too, are powerful simulations that allow us to explore and learn about the social world we inhabit. As Jonathan Gottschall puts in his book Storytelling Animals, they are a place “where people go to practice the key skills of human social life”. And the interface that allows us to interact with this simulation is identification with fictional characters.
It makes sense. All species are hardwired to learn specifically about that which is essential to surviving their environment. To us humans, one of these essentials is being able to figure out other people’s needs and intentions so that we can adjust our actions accordingly. Not an easy task, since we have to cohabit and cooperate with large numbers of strangers we know nothing about.
If stories are simulations of the social world, then it makes sense that we interact with it by ‘stepping into the shoes’ of a fictional character. Through identification with them we get to feel their longings, frustrations, virtues and flaws. We experience first-hand their struggles, the moral dilemmas they face, and the consequences of the choices they make. We walk their walk and that’s how we learn.
But how exactly does identification work as an interface? It is easy to see how we can use a cockpit to interact with a simulation, but identification? Stepping into the character’s shoes sounds great as a metaphor but if you’re entrusted with the task of performing the feat, it can leave you feeling a bit baffled.
Luckily, there’s a way of bringing down to earth this concept of identification with fictional characters by looking at it from a more biological point of view.
I’ll start with transportation. This is another term that goes hand-in-hand with identification. Rule number one is that, in order to step into the character’s shoes, the audience need to be transported to the story world first. Another metaphor, but this time it is actually closer to the real thing since films transport us to the story world almost literally – though it is not our legs what take us there, but the physiological responses and emotions we feel in the body as a result of being exposed to the story events.
Physiological responses ‘transport’ us because, if we’re feeling anything, it means our brains think we’re there. If our brains thinks we’re there, we might as well be there. And why do our brains think we’re there?
Films work as an illusion because they exploit loopholes in the perceptual and cognitive processes that we evolved to help us navigate the environment. One of them is the communication time lapse that exists between the unconscious and the conscious brain.
In very simple terms, the brain works like this: it interprets the data that the senses have picked up in the environment, it establishes the context, it evaluates the information according to this context, it determines its importance, and it decides on the best course of action. Is it a threat? Run. An opportunity maybe to reproduce? Strike a sexy pose. A significant change? Get closer and find out more.
The first thing we feel as a result of this process is the physiological response, which tends to be quite basic and which main function is to prepare the muscles to move. Then comes a fully-fledged emotion, which contains more detailed information about the required response. A racing heart in itself does not tell us much. If it comes with a rush of fear or of lust, then we get a much more precise idea of what it’s all about.
On the whole, we’re not aware of these processes. This is because the brain operates in two basic modes, unconscious and conscious. The unconscious mode is much faster at processing things than the conscious. It can organise the neurochemistry and behaviours of our system within 80 milliseconds, whereas it takes the conscious mode about 250 milliseconds to catch up with things.
It is thanks to this gap that we experience films the way we do. When we’re watching a movie, all our brains know for the first 250 milliseconds is that the senses are sending information that is organised in patterns of light, colour, sound, movement and behaviour that feel just like the real world. So the brain does with this information what it evolved to do. It processes and evaluates it, and it prepares the body for the right action. Luckily, though, just before we actually stampede out of the theatre at the sight of a dinosaur, the conscious brain realises that it’s just a movie.
In short, by the time the conscious brain has figured out that we’re only watching a movie, the unconscious brain has already made a full cognitive evaluation of a situation it deemed to be real and it has triggered all sorts of physiological and emotional responses in our bodies. These bodily sensations are what anchor us in the story world. As far as the unconscious brain is concerned, we are there.
All this works very well for transportation into the story world, but that’s only the first stage. Other things still need to happen for identification with a character to take place.
One good way of understanding identification is by looking at what it is not – empathy. These two terms are often used interchangeably, but they are not the same. We can, for example, empathise with humans and animals alike, but we can only identify with humans. Making this distinction is crucial.
Empathy is when we recognise and share the emotions and feelings of another being. We see their situation from their perspective and as a result we get to feel what they feel. This is possible thanks to a mirroring system that we evolved to deal with our social world. It works by replicating in our biology the neural processes that happen in the brain for coordinating and carrying out actions.
Let’s say that a man decides to open the door. A set of neurons will fire away in his brain and will activate the regions responsible for coordinating the actions involved. The motor system will then receive instructions and perform the task. Thanks to this mirroring system, if I watch this man carry out this action, the same neurons will fire in my brain and activate the same regions involved in the operation, only the motor system will not perform the action.
This mechanism works just the same with emotions. By picking up very subtle sensory cues from things such as facial expressions, body language, and tone of voice in another person, the brain is able to replicate in our bodies all the neural processes involved.
It is easy to see from an adaptive point of view the benefits of having such a system. It gives us first-hand information about the intentions and motivations of others and about their mental and emotional states, and it helps us adjust our responses accordingly. If you’ve ever seen someone in distress and immediately understood that all they needed was a hug or a few friendly words, that was your mirror neurons giving you a tip. Or if you’ve ever had a feeling that someone was lying to you or was up to something, that was your mirror neurons too.
As for film, you can imagine how useful these mirror neurons are when it comes to getting the audience to connect both biologically and emotionally with the characters on the screen. Sadly, though, this is not enough. For identification to happen, it is not enough to simply get to experience the same emotions and thoughts as the character. The audience must reach those mental and emotional states through the same cognitive processes that took the character there.
With empathy, things happen there and then, mostly thanks to our mirroring system. Identification is much wider in scope. It requires the audience aligning with not only the character’s feelings and emotions but also with his or her cognitive processes – how he or she reasons, evaluates things, solves problems, sets goals, formulates plans, and so on. That is why we can empathise with an animal such as a dog but not identify with it. Dogs feel emotions similar to ours because they are mammals, but they use different cognitive processes to solve their problems and as a result we cannot ‘step’ into them.
There is one assumption in NLP (Neuro-Linguistic Programming) that makes it easier to understand this concept of identification with fictional characters: “The map is not the territory”.
The territory refers to reality, the physical world that exists independently of our experience of it. The map refers to our minds and to the model of the world we have built through our perceptions, personal experiences, culture, and what we have learnt from the significant others that have been present since early in our lives. It contains, among other things, our beliefs and values, which play a key part in determining the decisions we make and the actions we take.
We only ever get to know our own version of reality. Because it is so unique and personal, we can’t really step into someone else’s mind and experience the world through the lens of their model, not unless he or she is a fictional character, that is.
And that brings us back to story as simulation of the social world and identification as its interface.
If our mental model of the world determines the cognitive processes by which our brains perceive, interpret, and react to events in the environment…
And if identification is achieved by getting the audience to feel and think what the character does through the cognitive processes that took the character there, i.e. through the same mental model…
And if simulations allows us to create models of some aspect of the world that we can use in its stead…
… then we can achieve identification with a character by creating a customised mental model of the character’s world that contains the values, beliefs, and experiences that will lead to the emotions and behaviours we want to explore in the simulation-story. We can then ‘install’ such model in the audience’s minds. This way, their brains will be operating within that specific mental model, and as a result, they will walk the character’s walk, and arrive at the same thoughts and emotions by means of the same filters that took the character to do, think, and feel what he or she did.
Equally important is that all this process generates a lot of neural activity in the brain. The resulting synapses are what get stored in the nervous system as memories. When this happens, the film has served its evolutionary purpose, and the audience get a reward in the form of a shot of feel-good chemicals such as dopamine. That’s why films that get identification right tend to do well at the box office.
One of the main tools for achieving identification with characters is narrative structure. The set-up, for example, is all about ‘installing’ in the audience’s brains the specific parameters in the character’s model of the world that will cause him or her to take the actions and make the decisions that will lead to his or her success or demise. From that point on, even if the character is not present in the scene, the audience will be evaluating the events from within their position of identification. “Are these events good or bad?” And “Will they facilitate achievement of the goal or hamper it?”
This tool, narrative structure, is the staple of all forms of fiction in general. But there are different forms of storytelling – novels, theatre, film – and each, as anyone who’s had a go at adaptation will know, offers dramatic possibilities unique to their own format. Each allows you to get into the character’s mind in ways that the other formats can’t.
What is unique about film is that stories are told through the arbitrary combination of images and sounds which are arranged according to an established cinematic code.
It is easy to see the advantages such feature would bring to a simulation. First, the presence of the two primary senses adds realism and acts as a form of veridicality. The brain is prone to perceptual illusions due to loopholes and it knows it. Seeing is believing, so is hearing, although there’s plenty of room for mistakes. But when you hear and see, there is no doubt in the brain’s mind that it must have been an accurate perception. This makes film even more of an effective illusion.
Also, sound alone does an incredible job of transporting and immersing us in the story world. Being a mechanical wave, it literally touches us and can, through sympathetic resonance, influence our biorhythms in the mental and emotional domains.
Then there’s point-of-view visuals and point-of-hearing sound, both of which can take identification to deeper levels. Although, be warned, in and of itself this technique is not enough to get the audience to identify with a character. In horror films, for instance, a point-of-view shot of the victim does not necessarily lead to identification with the killer (identification, remember, is a process).
But where the real possibilities lie is in the actual relationship between image and sound – in how they are combined meaningfully for the purpose of creating a specific mental or emotional effect.
In reality, the brain uses emotions and perception to guide our actions. After carrying out its evaluation, it activates the right behavioural programs, i.e. emotions, and it determines which bits of information on each sensory channel are most useful to guide our actions. If most of the important information comes in the form of, say, sound waves, it will reduce the presence of other sensory data. The result will be a streamlined percept that includes only what is essential to perform the task at hand efficiently, and that will have excluded any distracting sensory stimuli.
In filmmaking, the director takes charge of this process. He or she creates a percept, or movie, by manipulating and combining images and sounds so as to fool the brain into interpreting things in a way that will lead it to trigger the physiological and emotional responses required by the narrative. Filmmaking is the art of hijacking the brain and tricking it into thinking that the film is a meaningful percept that it, the brain, created by itself.
So what we have on one side is that identification requires that the audience get to feel and think what the character does by the same perceptual and cognitive means.
On the other side, we have that filmmakers can capture image and sound separately and then recombine the two arbitrarily to manipulate the perceptual and cognitive processes that guide the audience through the narrative.
In summary, we can use the relationship between image and sound to deepen identification. Or to give it some slack if that’s what the narrative requires, like for example when the character is a dubious figure and you need to break identification in order for the audience to step back and have a chance to reflect on what has transpired and thus learn the moral lesson of the film. Identification, after all, moves along a continuum. The manipulation of audiovisual information can help things move along this continuum.
In the last few paragraphs I’ve started mentioning a few times the phrase ’the relationship between image and sound’. This is what film sound is all about. Not the sounds. Not even the act of combining the two. The relationship itself, as an entity in its own right, is the actual cinematic device, the dynamic that breathes life into film sound.
So, in my next post, I’ll be talking about the guiding principle that can make this partnership go from ‘mere combination of image and sound’ to ‘meaningful relationship between image and sound’.