Half the Picture

With Congress’ proposal of CALMer airwaves via the proposed Commercial Advertisement Loudness Mitigation Act, television viewers may be able to expect some aural relief as TV spots are required to be softer in volume than the shows they accompany. I just want to split a few hairs and talk about some of the technical implications here.

IT’S THE DYNAMICS, SILLY
This is one of those areas of mixing sound for picture that overlaps with music mixing. Every pop music mixing engineer has fought in the Loudness Wars–the dilemma of how can I get my track to sound louder and hotter than the one before it on the radio. Essentially TV spots are doing the same thing vs. the shows they sponsor. Let’s talk about how dynamic range, essential to audio enjoyment, suffers in the loudness wars.

Most TV spots usually do not require a very wide dynamic range since their narrative develops quickly over 10, 30, or 60 seconds, among other reasons. Dynamic range is sacrificed for the cause of instead being able to push up the whole mix. The tighter the dynamic range of any audio mix, the louder it can theoretically be played back/broadcast, since it is less likely to clip (overload) while raising the softest levels as well. The tradeoff is reduced impact of loud signals since everything is already loud.

A TV show on the other hand develops its narrative over at least 30 minutes and perhaps it wants to make use of more dynamic range to tell its story. When that car crash comes, it should make an impact vs. the dialogue. Loud enough to make you feel it. Unfortunately when TV dialogue is mixed as loud as it often is, impacts and explosions don’t open up the listener’s ears as much as they would in most films. When everything is loud, ear fatigue naturally starts to desensitize us…this is the last thing a mix engineer may want but loud mixes are very powerful to the listener for a short time (until she turns down the volume on her set).

LET’S GET RESENSITIZED
So let’s recap. TV spots are mixed (or broadcast) louder than TV shows. Movies are mixed with even more dynamic range than shows, since the viewer always has to turn up the volume when putting in a dvd. And there is a tradeoff of strong impact vs dynamic sensitivity (which generates interest). It would quickly get very tiring if Scarface was mixed as pungently loud as South Park. How would the filmmakers build drama & tension without quieter segments?

Doesn’t it make sense that if a TV spot is dialogue-driven then the viewer should expect more or less consistent dialogue levels from TV show to TV spot? And wouldn’t that make everyone in the house CALMer in this stressful time? Commercials may still sound louder, perhaps harsher, using more compression, or pushing a heavier assault of sound effects. But I can see no reason for the broadly indiscriminate desensitization that overly loud commercial mixes subject the viewer to today, pre-CALM. I hope this legislation helps foster a more respectful airspace in this time of war…volume war.

SPOILER ALERT! THIS POST GIVES CRUCIAL INFORMATION ABOUT THE STORY OF FOX TV’S PRISON BREAK THAT YOU DON’T LEARN ABOUT UNTIL THE 4TH OR 5TH EPISODE OF THE FIRST SERIES.

That said welcome to Half the Picture. let’s expose how advanced uses of our medium, sound & picture, can completely shatter audiences’ assumptions about a story. We begin by breaking into the filmmaker’s mindset, looking for clues into an artist who knows their medium well enough to hide its weakness and exploit its unique capabilities.

LET’S DECONSTRUCT

What is good filmmaking? What is achievement in any art? We know that it’s like magic when it comes together, when it has some kind of desired impact on an audience. In film, this immediate impact is, ironically, created by splicing together discontinuous, previously-recorded pieces of sound & video. In other words, in the words of Orson Welles, it’s fake. One big fakery.

Fake seems a little harsh. Let’s use artifice, from artificial instead. Good artists exploit the artifice of their art, deceiving audiences by using aspects of the medium to close the gap between what is invented and what is real.

A BREAK WITH CONVENTION

Good filmmakers are virtual gods, bending time & space in a virtual world. They break through what we conventionally think is possible by applying a deep understanding of the artificial medium of film. It’s not complicated; we already defined this as “splicing together discontinuous, previously-recorded pieces of sound & video.” Good filmmakers are very aware that they are working with separate sound and video tracks, and that they are free to exploit the best of each, individually.

The makers of Prison Break took good advantage of their medium when they hinged the guilt or innocence of inmate Lincoln Burrows on a surveillance tape. They led the audience to believe, for the first several episodes, that the primary evidence of the crime that put Lincoln in jail was a true representation of what happened at the crime scene. After all, if pictures don’t lie, then sound and picture is surely admissible in court.

Of course we know this isn’t true. In episode five of the series, Lincoln’s rivals in the series do a little post-production audio & video work of their own. They add gunshot Foley and visual enhancement over Lincoln’s outstretched arm just at the point when it looked like he pulled the trigger. Exploiting the fuzziness of the surveillance tape with the startling impact of an added gunshot, the forgers lead us to believe that Lincoln did in fact fire that deadly shot. Later, Lincoln’s enterprising attorney finds the original tape with unaltered sound and picture with–you guessed it–no gunshot. He never fired the gun. Our man is innocent. We are also told to listen carefully to the lack of echo in the parking lot. Yes, the forgers forgot to add reverb! Surely no self-respecting sound engineer on a jury would miss this opportunity to show the court how video can be manipulated to push a story in a number of directions.

MASTER PUPPETRY

What is achieved by masters of an art, those who know well how to exploit its medium, comes from an understanding of its elements. Film is just a series of video frames moving just fast enough to be indeterminable from live movement, then synced with sound. But our immediate reaction to it is emotional, and we forget about the very deliberate manipulation of sound and picture. Since so much effort is required to bring them together to look & sound natural, it’s not often that filmmakers deliberately exploit their separation. However this wisdom is apparent in this very effective plot twist in Prison Break. It’s a fine example of how to exploit the artifice of film for an effect unachievable b a novel, play, or other storytelling medium.

I was watching an interesting documentary the other day that had obviously spent most of its budget on traveling around the US and Europe in pursuit of interviews. They captured a fascinating depth and breadth about the subject by literally going the extra mile. While there wouldn’t be much of a film without this footage, the dialogue tracks sounded ‘choppy’ and low-budget. As a sound engineer it struck me how easy it is to overlook simple, relatively cheap post-production sound techniques that can dress up hard-won interviews into dialogue tracks that sound natural and continuous.

When the filmmakers got back to the editing room with these interviewee’s stories, naturally to avoid jump cuts they go to B-roll whenever they want to interrupt an interviewee to cut the story shorter. This helps cut out the fluff, shorten the interview, and hopefully help it flow. The astute viewer will see this happening during cuts to fidgeting hands or a shot of them walking around earlier in the day, or a sudden shot of the surroundings. The trick is to keep these cuts smooth so as to make the soundtrack appear like it is one continuous, natural speaking event. The trouble, however, is finding a natural place to cut someone off and then blend unrelated sentences together that could be minutes apart – and much different in tone quality.

Enter dialogue editorial. Once the director & editor have decided on how the interview should be cut, sewing up the disparate speaking levels and abstract sentence rhythms into something that flows is the job of the dialogue editor. Most will probably tell you that their primary tools are level adjustments and room tone.

Techniques like levels adjustments and audio crossfades are self-explanatory and par for the course: the idea is to get the perceived volume of the beginning and endings of phrases to flow smoothly in a natural way. If the interviewee got closer to the mic during a sentence, the sound engineer rides the level to keep it consistent. Et cetera, ad infinitum.

Proper use of room tone, however, seems like an obscure art when building a smooth interview track, but it is actually not hard to learn. When all that editing and cutting to B-roll results in gaps between phrases, the dialogue editor inserts a length of clean, natural background ambience between phrases and crossfades it into the two existing clips before and after it. Linearly, on a timeline, this looks like: |PHRASE 1|–x–|ROOM TONE SEGMENT|–x–|PHRASE 2| where the –x–’s are crossfades between clips. With good clean room tone (which should always be recorded on location) the audience will never know that a natural-sounding 4-minute interview was actually born from 6 hours of footage.

ADR is shorthand for automatic dialogue replacement – though hardly automatic and nothing more than a necessary evil to many directors, actors and post-production audio engineers. While it can be rather routine to coordinate and carry out these recording sessions, ADR should really be thought of as another creative tool in the sonic arsenal. Not just for flubbed lines anymore, ADR can marry dialogue with picture in some interesting ways that would otherwise be clumsy or inconceivable to shoot live.

ADR usually has an actor re-reading her lines to rough picture in the studio, and speaking them pretty much as performed on set. Reasons for doing so include: noise & interference (e.g. airplane passing overhead), equipment malfunction (such as the actor’s throat or mouth), or the director may want a different inflection, emphasis, or tone quality out of the lines. It’s basic cut-and-paste, but let’s first talk about a variation on the theme before we get into the more creative ways it can be used.

When an actor flubs a line but carries on in character, there are several ways to save the scene. One way is to interrupt the shot with a crosscut to a reaction shot of the listening character. Actor #1′s dialogue continues underneath the cutaway. Since the audience does not see the actor’s mouth move during the crosscut, it’s just a matter of dubbing in the correct line in post over the visual of the listening character (or other cutaway, say of nervous hands, tapping feet, or wind in the trees). It’s possible that this method might be problematic if the cutaway takes the viewer out of an intense emotional moment or if it interrupts comedic timing or editing rhythm. In those cases, another solution to botched lines must be found. Otherwise, cutting away to avoid trouble spots in the dialogue can be your saviour. In fact, it’s second nature to documentary editors who generally have only one take of an interviewee’s lines with which to work anyway.
So what about using this technique on purpose to enhance the story? Say a writer/director wants to change a line of dialogue in post because they’ve come up with something more effective, or funnier, more politically (in)correct, or what have you. The same technique applies: cut away to something in the surroundings or to another actor’s reaction and ADR the line. With some skillful audio engineering no one will be the wiser and the story gets a quick and easy touch-up.

Here’s an even more creative way to use ADR: writing in expletives between adult and child actors. The mouth movements of ‘shin,’ ‘chin,’ ‘chit’ or a number of other words all look convincing enough on-camera to the common expletive we know and love. But these substituted words are easier on a child’s ears (and their parents, looking on from off-camera) and they can simply be overdubbed with the real swear words in post.

I saw this technique used in a (Comedy Central) Dave Chappelle Show episode where, in the skit, Dave’s ego is getting bruised by a little boy playing his son and, in usual hilarity, Dave is cussing out the kid for giving props to his father’s rival. Dave was able to make a scene using some uncensored, brutally funny ‘adult words’ on a kid because in reality the kid never heard those words. Looking closely at how his lips are moving we can see that Dave didn’t say the F-word or even the S-word in front of the child. He was saying words with a similar mouth movement (fudge?)  and later simply dubbed in the real words in post.

Dialogue replacement sessions may not seem like creative time, but with a little brainstorming and planning they can be another chance for a savvy director to enhance a film’s writing and acting.

image credit- Darwin Bell

Blending the layers of any soundtrack together is naturally an exercise in compromise. Many fall prey to volume wars: make THIS louder, no make THIS louder, wait now I can’t hear THIS…. until most sound elements in the mix are hitting their head against the ceiling and the subtle beauty of dynamic range has gone out the window.

Dialogue, especially, is the number one volume concern for filmmakers. Rightfully so, most of the time the dialogue is the sonic layer that is telling the story.

Most filmmakers want the magic number for where the dialogue level should sit. If it did exist, this decibel level would at the least need adjustment for whispers, shouts, and perhaps when characters are nearer to/further from the camera. Fact is, the spoken word has quite a dynamic range and the art of the engineer is knowing how to shape it so it comes through a speakerbox in pleasing ways.

Ironically, getting levels right in a mix has more to do with getting the studio sounding right, not the mix. Experienced Re-recording Mixers – they are the engineers who have to negotiate sonic space for dialogue, sound effects, ambiences, and music (in other words, they mix the picture) – speak in a zen-like tongue, frustrating those who want to mix their own soundtracks. The pros talk more about getting your ears adapted to defining what is loud and what is soft. They plug proper bass management, room calibration, and acoustical treatments. Everything except the magic number.

Aspiring engineers who want to get their mixes sounding more to broadcast standards should take a tip from the pros: get the studio sounding right, and the mixes will follow.

photo courtesy http://www.flickr.com/photos/36429573@N00/

Watching American football in recent years is a much more immersing experience than years ago. Of course the HD revolution brought us gorgeous picture detail, ability to rewind/replay/record, and closer cameras that put us right on the field. But let’s talk about some of the audio engineers’ more subtle techniques for getting us all riled up.

First off, we have 5.1 surround-sound. Mix a little crowd into those surround speakers and suddenly the viewer is immersed in the weekend insanity of 30,000 bankers, bartenders, and bookkeepers. Surround-sound adds another layer of realism and intimacy of the game.

Long-time football fans will also remember the days before the shotgun microphones were able to pick up every little grunt of the quarterback, players, and refs. And now sound designers enhance critical moments of snaps, catches, and tackles by laying stock sound effects into the mix. Ball hits player, player hits player, player hits ground – and you FEEL each one of them – thanks to impact sound effects. Whether recorded live or placed by hand, impact effects are a welcome addition to football in recent years and definitely help the hard work and pain of the players to hit home – right in the gut.

Filmmakers can use these same techniques of laying in sound effects and ambiences to enhance crowd scenes, fight scenes, and almost any drama. Stock sounds are widely available both for free online and as part of commercial sound effects libraries so why not use them? With a careful ear toward creating a more full-featured mix, filmmakers don’t have to let their films ride on just dialogue; they can create immersive worlds that help viewers feel what their characters are feeling.

photo courtesy http://www.flickr.com/photos/padrian2/