home | et39 - digital audio design for games - 2015 edition prev | next

David Javelosa


Copyright © 2015 David Javelosa unless otherwise stated.


week 06 - sampled sound effects and sound design, signal processing-DSP


Sound Design And Effects

Approaching a project from the overall perspective gives the resulting production a consistency that is present at every level of detail. This consistency must tie in the wide variety of one-shot sound effects, abstract flourishes and cues, musical passages, and even the quality of how the dialogue has been recorded and edited. This overall style and mood is the bigger picture of sound design. For an entire project, the first thing that needs to be determined is the number of actual assets to be produced, and an how many of each kind. This is what is known as a script breakdown in film, but the term very well applies to software projects and specifically games. This is the slow steady work that chops up what seems like an infinite amount of work into bite-sized tasks that can be taken care of one at a time.

For the dialogue in a project, the script must be prepared into a "shooting" script, or a document that will be ready to take into the voice-over studio with your talent. This is where each of the dialogue segments should also be given a file name for data management. Besides being aware of any large segments of the script that may have to do with descriptions or first person game experiences, the characters must be identified and listed, as well as any off-screen voices or narration. Once all the lines have been determined, the talent must be cast. In the case of most budget productions, very versatile actors are the best bang for the buck because they can cover more than one character for a production. In the high profile titles, this is where you might be bringing in a celebrity voice talent, perhaps a well known actor or actress, to do the lead character or maybe just a guest appearance. This is all determined by the budget.

The budget tends to drive everything, but particularly at this stage, if the recording process is not well planned it can cost a lot more than anyone could imagine. After meetings with the producer, the writer or game designer and maybe a casting director, the final lines are determined and the available voice talent is cast into each of the characters. At this point the talent must be scheduled to take up the best use of their time. By have all the lines of a particular actor, in spite of multiple characters being covered, recorded back-to-back in one session, you are only having to pay for one day of the talent's time. Obviously the big name talent will be more expensive, so they would be given priority as far as convenience in the schedule. The patchwork of actor's schedule, versus characters to be covered, versus actual studio time booked, becomes your recording schedule or "day out of days". By having the lines of the script as stable as possible and allowing the optimum amount of studio time booked for the material and any improvisation to be used, a voice-over session can be a fun, simple and not-too-expensive of a proposition. Of course, there is always the catering...

For the sound effects in a project, the script must be prepared as a separate breakdown, into a production cue list. Unlike the recording script, this is a very straight forward list of the generated file name, the section or scene, and a concise description of the sound as the user is to hear it. This list should be created and used along side the general script to maintain consistency and context of the individual sounds. Once this list is created, the task of fulfilling the production of all these cues can be divided up among several sound designers for maximum use of time and given resources.

Now that the work is dished out to you or your team, you can dig into the task of collecting, editing and processing the sounds. These can start from a number different places. The first place to turn to is from the big collection. What collection you say? If you have been working in sound design, special effects or foley, you would probably have tapes and tapes of interesting sounds that you have collected over the years. Maybe some of these have been backed up to CD-ROM, DAT or other mass storage devices. If you are not a seasoned professional (or just hoping to be one someday), the collection starts now. There are of course the stories of wacky sound guys carrying around their DAT player everywhere they go, recording this sound or another because it sounds interesting. This is how the really unique sounds get collected and the more interesting and "signature" the sound, the better your reputation gets.

For those who are starting out, there is the benefit of the hard work of those who have come before. Either commercially available CD libraries, DAT collections of the senior designers in your studio, or even old vinyl sound effects LPs can at least be a start. The concept of filling a prototype project with "place holder" sounds is an easy application of these venerable collections. Just remember, there are a growing amount of engineers and sound people out there with exactly the same collections, so using what sounds to be typical or what you would expect will also be exactly what the other people will be using. Also, care should be taken in how much time and effort is put into a prototype regarding collecting sound effects. If you are going to do the job once, you might as well do it right the first time.

Then there is the craft of making up your own. Either going out and doing your field recording with a portable DAT player or sitting in a foley studio rattling and banging things in front of a microphone. These are always a guaranteed way to come up with an original sound and avoid any licensing or purchase expenses, but the time and effort of going out in the field for every single sound can be expensive. Foley studios can be a bit much to maintain on your own and hiring a professional studio can really ring up the dollars. One developer I know recently hired a very high profile audio production team to do sound effects and what they charged them in the foley studio would have been enough to hire their own in-house person for an entire year! The third option is creating effects with electronic instruments. Samplers and synthesizers already are adept at recreating interesting sounds and if the job is calling for the abstract, this is the often solution. Of course all of these techniques are necessary, and sometimes all of them are used to create a complex sound scape. The art of "compositing" comes from the visual artist's practice of collage. For the sound artist, being able to mix and blend sounds from different sources in the digital world has become the most important skill in sound design. Let's look at some of these approaches with greater detail.

CD Libraries

The foremost company in CD sound effects libraries is the Sound Ideas collection. These have been culled from thousands of pre-existing libraries and archives from all over the entertainment industry, and also include a great many original sounds to fill out their extensive selection. There is one basic set and there is a larger "complete" set made up of several "volumes", each volume containing ten to twenty CDs worth of sounds in the Redbook CD audio format. The index for the entire collection is published in one entire book, in spite of which volume or volumes you have purchased. This can be a little frustrating when you are looking up just the right sound and your realize it is listed in a volume that you don't happen to have. Sound Ideas has also licensed sound effects collections from some of the better know film studios such as Lucas Films and Hanna-Barbara, making available their classic sounds as well as lesser known effects lurking in their vaults.

Even having the largest sound effect library in the world still calls for sounds that are just not quite what you find in the CD collections. As mentioned earlier CD libraries should be were you start from, but often they are exactly what everyone ends up using. As a result, there have bee several new companies specializing in repackaging older collections or just going out there and creating new and exciting sets of sounds that everyone will want to buy. A popular leader is the Hollywood Edge collection. These are CDs of brand new material created and stylized specifically for action and adventure applications. Cashing in on the special effects made for cinema, the collection provides larger than life versions of sounds that would be used in every day, common place situations. The emphasis on their collection is the way the sounds stand out and retain their individuality by way of audio processing. Another feature is that Hollywood Edge comes as CD-ROM as opposed to CD audio. This means that the collection is ready to read from a computer based editing system. Sound files can just be copied right into an application or brought into an editor for compositing or further processing.

In dealing with large collections on CD-ROM, AudioShop by Opcode becomes another very handy utility. By selecting large numbers of files into AudioShop playlists, any number of sound files can be auditioned in any order from the CD. By recording this playlist out to a DAT player via digital output, an entire sequence of sounds from one CD can be digitally transferred in the order that they will be used. This can also be done with CD audio redbook tracks but currently there is no way to send out a re-sequenced output of tracks digitally from an audio CD running in a Macintosh CD-ROM drive. For this reason many high-end studios use dedicated CD audio players with the coaxial S/PDIF (Sony/Phillips Digital Interface) jack connected directly to the computers audio card. A handy trick is to have the CD player hooked up digitally to the S/PDIF jack and the DAT player hooked up via the AES/EBU connection. Pulling down the hardware setup from a menu in Sound Designer, you can switch the type of input for the sound card from the software, effectively switching back and forth between digital input from a DAT and digital input from an audio CD. This can save a lot of time when collecting and compositing sound effects from several sources or collections.


Using sound synthesis for creating musical instrument sounds has become the most common use of that technology. As electronic and later MIDI instruments moved more into sample-based synthesis and sampling in general, the practice of recreating the sound of traditional musical instruments continues to flourish. However with sample technology, the definition and creation of musical sounds has broadened considerably. The incorporation of what would have been considered sound effects as musical instruments shows a perfect example of the technology helping create its "native" art. With this approach in mind, it also shows the incredible impact this technology has had on creating actual sound effects for both traditional and interactive media.

Using synthesizers for sound effects, specifically the abstract, emotional cues found in cinema today, has certainly gone far beyond science fiction movies to the type of sound design we think of as being quite mainstream. A variety of orchestral flourishes, cymbal rolls, the odd percussion riff with deep reverb, are the sort of sounds we unconsciously expect to hear in film or on television. But taking this abstract approach to sound design one step further allows the audio designer or composer to create brand new sounds from basic components, that evoke the same sort of emotional response, or perhaps even go further. This creation of sounds that have "never been heard before" is a major part of what true sound design is all about. For interactive software productions and game scenarios, this type of cinematic practice can have an incredible effect if used properly. Over-using anything in the software environment can be annoying, but the right mix, good timing and optimized use of the presentation technology can paint the perfect picture.

By basic components, I refer to creating sounds starting with simple wave shape sounds or raw samples. Within the domain of the synthesizer, be it analog or digital, basic sounds are composited, filtered, processed and combined to create new instruments for your cinematic flourish. This is all fine and good when dealing with the ability to digitize the results for playback as digital audio. It is also the practice when direct control of the synthesizer is available in the case of FM based console systems like the Sega Genesis. In the case of creating these types of cues in the General MIDI world, the challenge becomes certainly greater. One of the best historical references for creating sound design in General MIDI is the genre of the classic cartoons. These were scored in a way that the action completely dictated the flow and profile of the orchestral accompaniment. In many cases, the sounds that we associate with cartoon sound effects were actually done by orchestral instruments. This contributes to my basic philosophy of thinking of an orchestra as the ultimate synthesizer.

With the simulated orchestra as a construction model, the General MIDI module can actually take on extreme roles. Besides all of the unusual "world" instruments, built-in sound effects, and other instrument "noises", taking something as simple as a piano patch high into the upper or lower registers of its usual range can come up with unusual and surprisingly useful material. Similarly with many of the tuned percussion instruments, some very mechanical and abstract passages can be generated considering that MIDI can play with un-human speed and accuracy. Many of the string embellishments employed by Bernard Hermann in Hitchcock films have yet to be exploited in the GM orchestration of games. Whether creating abstract sound design from electronic musical synthesis or re-synthesized orchestral samples, the main point is to be as creative and original as possible, while staying within the constraints of the mood, the context, and the pace of the production. This can be applied to both traditional linear media as well as interactive digital media, but with all of the real-time capabilities in entertainment software, it only makes the job of sound designer that much more dynamic and exciting. Having both the abilities to use MIDI and digital audio multiply your options and provide more solutions for delivering your sound design.

Types of Signal Processing
- DSP Plug-ins

1.Frequency - changing the pitch of a sound.
Pitch shifting/following
Doppler effects

2.Dynamic - changing the amplitude.

3.Spectral - changes to the frequency spectrum.

4.Time based  -

5. Special FX -
noise (surface, ambience, lo-fi)

Compositing sound and creating layers in Pro Tools

Reading Assignment

Read: Pro Tools Help Files

  • Mixing Basics
  • Building Tracks with Loops

Copyright © 2015 David Javelosa