First of all, when a composer is assigned to the task of "doing sound" for a software production (be it "multimedia", a computer or video game), this can mean a number of different things, depending on who you are talking to. If you are talking to the actual developer team, this definitely means the entire sound design. In their view, software production is all inclusive and likewise "audio" should include music, instrument sounds and sound effects. If you are talking to the producer, you could be doing any one of three main jobs, all of which falls into the producer's mind as "sound design". If the project is to be CD based and contain video with voice over, this would also include studio or location sound recording and dialogue editing. For the typical musician, this can be a very daunting, overwhelming task and a cause for distress to the uninitiated producer or lead programmer.

In these days where larger productions have become the norm, the "garage band" development team of programmers are no longer solely responsible for securing these diverse music and audio resources. Producers that have been around the block or perhaps have migrated from film, TV and other linear media, are familiar with the different specialists for each of these tasks. They are: the composer for writing, orchestrating and sometimes conducting the score; the "Foley" artist for recording, designing or otherwise "coming up" with the sound effects; the location sound engineer for capturing the audio with the video; the audio editor for making sure that the dialogue is consistent and in synch with the video; and finally, a mix engineer for making sure all of these things stay in balance and don't stomp on each other.

Sound like a big Hollywood production? You bet. And this would be the ideal approach to the task if software production had the tradition of burning money they way film production does. But for the smaller multimedia company or even mid-size game company, the value of wearing as many hats as possible comes to mind. Much was learned from the days of the one person "sound" job. For the multimedia composer, having as many of these skills as possible is a good thing. When running a medium size audio group at Sega, we had a studio engineer, a sound effects person and a music programmer, each with the ability to write music. This allowed us to cover the many hats of game sound and give us some variety in music styles. It is still possible for these jobs to be handled by one person in a small company, providing they know when to contract out the work they are least able to handle well.

The traditional role for the composer in the film/video biz has always been a hurry-up-and-wait role, or part of the post-production process of editing and sweetening. Actually, to be more precise, a hurry-up-and-wait-and-really-hurry-up-or-die. The point being, the producer of a linear production as such, is concern with getting all the other elements finished in order. This includes dialogue and sound effects. And while there have been countless meetings as to what the style that music should be at the beginning of the production cycle, and even place-holder music produced for rough cuts, the producer will not really want to hear a note until well into the assembling and editing phase. This has meant that the composer has the least amount of time to get his material together for the production.

This is changed in the new media process for two reasons. One, in digital media production, everything is happening at the same time. Scenes and characters can be thrown together for demos at the drop of a hat, and then brought to brain-storming meetings for instant feedback. Likewise music, sound effects and even scored bits of video will need to be available even during the design process, even if the material is just "placeholder" or being sought for clearance. This can be a warning sign in the early demo stages because it gets the executives and marketing people nervous. The second reason is that the "audio" person is in fact wearing many hats. The final music may be delivered at the end of the production process for the final edit, but dialogue and sound effects will be needed earlier on. All of these little hats fall under the big hat of what is now being known as "sound design", an entirely new art form for the late 20th Century.

Conceptual sound design is something that should be started at the beginning of the production process and managed throughout the entire job. Ironically, just as the video game technician of old was the sole control factor for the sound, so should the role of "sound designer" be in the larger interactive productions. The importance of consistency of sound effects, score, dialogue level, transition music, etc. becomes critical in a non-linear environment where the user's actions are determining the presentation flow. The concept and quality of sound design in commercial production makes good study because of the short length of the end experience. Everything is under tight scrutiny for such productions of just a few seconds long. As a result, all elements: music, dialogue and sound effects, are considered part of the "sound design".

The best games and even educational products will have clean, fluid audio support from beginning to end. The more complex the interactive experience, the more impressive a consistent sound design comes off. Once again, new technology creates a hybrid art form. For all of the duties and functions that need to be executed by the sound design team, they generally can be broken down in to a handful of specific components. If the team is quite small (like a team of one!) then the divisions, of course, tend to blur and the strengths of the individual team members must be capitalized while relying on the group to share the rough spots. Here is a look at each of the potentially overlapping jobs in interactive sound and music.

Composer

Some say the role of the composer (and JUST the composer) is the luckiest of them all. This is the person that writes the song the whole world sings. Sometimes it's the person that has to write and rewrite the tune so the title producer finally thinks it's just right. Sometimes the composer is able to completely express their inner feelings, utilize their academic training and theory, or just plain rock out. Other times (mostly) the task is one of finely crafting the mood and emotions to just the perfect accompaniment to a scene with out getting to compromised, mediocre or trite. There are several composer's guilty of the that last list of faults, encouraged by their own knack for selling themselves to the egos of the producers and executive above them, their music contributes to the background hum of noise that most people aren't enthusiastic about but put up with because it is what they have gotten used to expecting.

Certainly several chapters can be written discussing compositional theory and philosophy, even in writing commercial "industrial" sound tracks. The best composers are constantly composing themselves. In other words, following a style or method true to a personal focus will always translate to the quality of the work. A composer can be talented at being as versatile as needed to survive in the commercial music scene, while maintaining an identifiable style that is signature and unique. No matter how bad a composer needs the job, if that style is not the right match for the project, it does nobody any favors by using it in the product. The composer dilutes his or her creativity by compromising to much, the resultant product becomes vague and lifeless, and eventually the producer is out of a job because the title didn't sell. Identifying the right talent for the right job is again a finely tuned skill in itself, whether it is yourself or not.

The criteria for selecting a composer for an interactive project also goes beyond the appropriateness of the style. Technical savvy is a major consideration when thinking about MIDI studio productions or scored orchestral sessions. Producing for MIDI playback goes beyond being able to hook up a computer to a General MIDI module, especially if it requires configuring voices for optimum playablility, RAM space, compression ad other processing, and just sounding good on a technologically-challenged platform. This is another demand for a balance of talent, technology and experience. Of course, personality and marketing skills don't hurt much either.

Sound Designer

The sound designer, in the traditional media sense of the word is an artist unto him/herself. Recently, most sound designers have strong backgrounds in composition because of how technology (tape and sampling) has become such a predominant resource for the modern composer. The talent and ear for interpreting a visual event or environment with a combination of sounds so abstract and yet so appropriate is what many consider to be the high-point in 20th century composition. Beyond matching literal sounds with their on-screen counterparts, being able to work and compliment combinations of score and dialogue or utter silence takes the balance of a fine aesthetic, and a sensitivity for the total audio/visual experience, not to mention its interactive appropriateness.

The maintenance of a vast library of sounds and recordings is the foundation of the sound designers tools. Also the continual collection of new sounds, in the everyday environment and everywhere, contributes to the reputations of the consummate designers for film and television. Tales tell of legendary sound artists that go nowhere without a portable DAT recorder around their shoulder. Being able to capture that exact moment in sound, for either a desired cue or just to archive into the collection, likens the sound designer to the still photographer in innovation, creativity and spontaneity.

Now with digital tools, the craft is expanded into taking these collected gems and blending them into composites of split-second texture. The advent of the multi-track digital audio editing environment has combined this profession with the experimental "musique concret" composers from the age of cutting and splicing tape. As with dialogue editing and overall production, sound design may not be the cup of tea for must musicians and less adventurous composers. As one of the many hats for the one person team to wear, it does contribute greatly to the enrichment of the open-minded composer, not to mention paying a few bills. For a team that can afford it, the dedicated sound designer is an invaluable asset.

Audio Engineer

Probably the most traditional role in the interactive audio production team, the audio engineer is at home in the production studio with microphones, signal processing, and as big a mixing board as can be had. Since few new media studios can afford the luxury of a fully equipped recording facility, the position of dedicated audio engineer is a lucky spot to be. The requirements include experience in voice recording, working with film, video and animation, recording and mixing music, and acoustic knowledge of everything from hardware to microphone placement.

In recent years the audio engineer's skill sets have overlapped heavily with that of the sound designer. Multi-track tape recorders have given way to the hard-drive based digital editing systems, such as Pro-Tools, Deck, Session, and other higher-end hybrid systems. It is also hard to find a full-time employed studio engineer who is not versed in computer controlled mixing boards, digital signal processing and a variety of compression issues. Most engineers of this caliber have already broadened their client roster with interactive software projects, since most of the work by development houses is farmed out to professional studios. Still, there is a lot to be said for having one's own facility at the beck and call of a project. As smaller production houses merge into larger ones as means for survival, the acquisition of an audio engineer and studio become an enticing proposition.

Location Recording/Video Production

Similar to the audio engineer, the jobs of location recording and engineering sound for video eventually cross-over. The less common need for location recording also puts this role into the hired-out category, as many times a product design has no need for location dialogue. Ambient audio recording however has become a popular practice and one of the trade secrets of the sound designer. Besides capturing specific, individual events, the sound designer looks for longer, ambient recordings of subtle backgrounds and environmental textures. Sending a location sound engineer out with a photography or video team for a project that is only using images for compositing, can turn into a very fortunate decision when having to sound design something unique for those visuals.

Location recording for film and video is also a very traditional occupation and one that is, even in the film industry, a freelance position. This engineer is usually attached to an entire location team for capturing dialogue from the shooting sessions. In many instances, this is a union gig and requires years of training and the acquiring of specific high-end gear. The standard for this trade is the high-speed Nagra analog tape recorder but is quickly being replaced by a digital version, if not by the DAT format altogether. This is another job that may seem out of the ordinary for the typical musician or composer, but as an interactive sound designer, just another experience and excuse to get outside of the studio.

---

Sound Design And Effects

Approaching a project from the overall perspective gives the resulting production a consistency that is present at every level of detail. This consistency must tie in the wide variety of one-shot sound effects, abstract flourishes and cues, musical passages, and even the quality of how the dialogue has been recorded and edited. This overall style and mood is the bigger picture of sound design. For an entire project, the first thing that needs to be determined is the number of actual assets to be produced, and an how many of each kind. This is what is known as a script breakdown in film, but the term very well applies to software projects and specifically games. This is the slow steady work that chops up what seems like an infinite amount of work into bite-sized tasks that can be taken care of one at a time.

For the dialogue in a project, the script must be prepared into a "shooting" script, or a document that will be ready to take into the voice-over studio with your talent. This is where each of the dialogue segments should also be given a file name for data management. Besides being aware of any large segments of the script that may have to do with descriptions or first person game experiences, the characters must be identified and listed, as well as any off-screen voices or narration. Once all the lines have been determined, the talent must be cast. In the case of most budget productions, very versatile actors are the best bang for the buck because they can cover more than one character for a production. In the high profile titles, this is where you might be bringing in a celebrity voice talent, perhaps a well known actor or actress, to do the lead character or maybe just a guest appearance. This is all determined by the budget.

The budget tends to drive everything, but particularly at this stage, if the recording process is not well planned it can cost a lot more than anyone could imagine. After meetings with the producer, the writer or game designer and maybe a casting director, the final lines are determined and the available voice talent is cast into each of the characters. At this point the talent must be scheduled to take up the best use of their time. By have all the lines of a particular actor, in spite of multiple characters being covered, recorded back-to-back in one session, you are only having to pay for one day of the talent's time. Obviously the big name talent will be more expensive, so they would be given priority as far as convenience in the schedule. The patchwork of actor's schedule, versus characters to be covered, versus actual studio time booked, becomes your recording schedule or "day out of days". By having the lines of the script as stable as possible and allowing the optimum amount of studio time booked for the material and any improvisation to be used, a voice-over session can be a fun, simple and not-too-expensive of a proposition. Of course, there is always the catering...

For the sound effects in a project, the script must be prepared as a separate breakdown, into a production cue list. Unlike the recording script, this is a very straight forward list of the generated file name, the section or scene, and a concise description of the sound as the user is to hear it. This list should be created and used along side the general script to maintain consistency and context of the individual sounds. Once this list is created, the task of fulfilling the production of all these cues can be divided up among several sound designers for maximum use of time and given resources.

Now that the work is dished out to you or your team, you can dig into the task of collecting, editing and processing the sounds. These can start from a number different places. The first place to turn to is from the big collection. What collection you say? If you have been working in sound design, special effects or foley, you would probably have tapes and tapes of interesting sounds that you have collected over the years. Maybe some of these have been backed up to CD-ROM, DAT or other mass storage devices. If you are not a seasoned professional (or just hoping to be one someday), the collection starts now. There are of course the stories of wacky sound guys carrying around their DAT player everywhere they go, recording this sound or another because it sounds interesting. This is how the really unique sounds get collected and the more interesting and "signature" the sound, the better your reputation gets.

For those who are starting out, there is the benefit of the hard work of those who have come before. Either commercially available CD libraries, DAT collections of the senior designers in your studio, or even old vinyl sound effects LPs can at least be a start. The concept of filling a prototype project with "place holder" sounds is an easy application of these venerable collections. Just remember, there are a growing amount of engineers and sound people out there with exactly the same collections, so using what sounds to be typical or what you would expect will also be exactly what the other people will be using. Also, care should be taken in how much time and effort is put into a prototype regarding collecting sound effects. If you are going to do the job once, you might as well do it right the first time.

Then there is the craft of making up your own. Either going out and doing your field recording with a portable DAT player or sitting in a foley studio rattling and banging things in front of a microphone. These are always a guaranteed way to come up with an original sound and avoid any licensing or purchase expenses, but the time and effort of going out in the field for every single sound can be expensive. Foley studios can be a bit much to maintain on your own and hiring a professional studio can really ring up the dollars. One developer I know recently hired a very high profile audio production team to do sound effects and what they charged them in the foley studio would have been enough to hire their own in-house person for an entire year! The third option is creating effects with electronic instruments. Samplers and synthesizers already are adept at recreating interesting sounds and if the job is calling for the abstract, this is the often solution. Of course all of these techniques are necessary, and sometimes all of them are used to create a complex sound scape. The art of "compositing" comes from the visual artist's practice of collage. For the sound artist, being able to mix and blend sounds from different sources in the digital world has become the most important skill in sound design. Let's look at some of these approaches with greater detail.

CD Libraries

The foremost company in CD sound effects libraries is the Sound Ideas collection. These have been culled from thousands of pre-existing libraries and archives from all over the entertainment industry, and also include a great many original sounds to fill out their extensive selection. There is one basic set and there is a larger "complete" set made up of several "volumes", each volume containing ten to twenty CDs worth of sounds in the Redbook CD audio format. The index for the entire collection is published in one entire book, in spite of which volume or volumes you have purchased. This can be a little frustrating when you are looking up just the right sound and your realize it is listed in a volume that you don't happen to have. Sound Ideas has also licensed sound effects collections from some of the better know film studios such as Lucas Films and Hanna-Barbara, making available their classic sounds as well as lesser known effects lurking in their vaults.

Even having the largest sound effect library in the world still calls for sounds that are just not quite what you find in the CD collections. As mentioned earlier CD libraries should be were you start from, but often they are exactly what everyone ends up using. As a result, there have bee several new companies specializing in repackaging older collections or just going out there and creating new and exciting sets of sounds that everyone will want to buy. A popular leader is the Hollywood Edge collection. These are CDs of brand new material created and stylized specifically for action and adventure applications. Cashing in on the special effects made for cinema, the collection provides larger than life versions of sounds that would be used in every day, common place situations. The emphasis on their collection is the way the sounds stand out and retain their individuality by way of audio processing. Another feature is that Hollywood Edge comes as CD-ROM as opposed to CD audio. This means that the collection is ready to read from a computer based editing system. Sound files can just be copied right into an application or brought into an editor for compositing or further processing.

In dealing with large collections on CD-ROM, AudioShop by Opcode becomes another very handy utility. By selecting large numbers of files into AudioShop playlists, any number of sound files can be auditioned in any order from the CD. By recording this playlist out to a DAT player via digital output, an entire sequence of sounds from one CD can be digitally transferred in the order that they will be used. This can also be done with CD audio redbook tracks but currently there is no way to send out a re-sequenced output of tracks digitally from an audio CD running in a Macintosh CD-ROM drive. For this reason many high-end studios use dedicated CD audio players with the coaxial S/PDIF (Sony/Phillips Digital Interface) jack connected directly to the computers audio card. A handy trick is to have the CD player hooked up digitally to the S/PDIF jack and the DAT player hooked up via the AES/EBU connection. Pulling down the hardware setup from a menu in Sound Designer, you can switch the type of input for the sound card from the software, effectively switching back and forth between digital input from a DAT and digital input from an audio CD. This can save a lot of time when collecting and compositing sound effects from several sources or collections.

Synthesis/Sampling

Using sound synthesis for creating musical instrument sounds has become the most common use of that technology. As electronic and later MIDI instruments moved more into sample-based synthesis and sampling in general, the practice of recreating the sound of traditional musical instruments continues to flourish. However with sample technology, the definition and creation of musical sounds has broadened considerably. The incorporation of what would have been considered sound effects as musical instruments shows a perfect example of the technology helping create its "native" art. With this approach in mind, it also shows the incredible impact this technology has had on creating actual sound effects for both traditional and interactive media.

Using synthesizers for sound effects, specifically the abstract, emotional cues found in cinema today, has certainly gone far beyond science fiction movies to the type of sound design we think of as being quite mainstream. A variety of orchestral flourishes, cymbal rolls, the odd percussion riff with deep reverb, are the sort of sounds we unconsciously expect to hear in film or on television. But taking this abstract approach to sound design one step further allows the audio designer or composer to create brand new sounds from basic components, that evoke the same sort of emotional response, or perhaps even go further. This creation of sounds that have "never been heard before" is a major part of what true sound design is all about. For interactive software productions and game scenarios, this type of cinematic practice can have an incredible effect if used properly. Over-using anything in the software environment can be annoying, but the right mix, good timing and optimized use of the presentation technology can paint the perfect picture.

By basic components, I refer to creating sounds starting with simple wave shape sounds or raw samples. Within the domain of the synthesizer, be it analog or digital, basic sounds are composited, filtered, processed and combined to create new instruments for your cinematic flourish. This is all fine and good when dealing with the ability to digitize the results for playback as digital audio. It is also the practice when direct control of the synthesizer is available in the case of FM based console systems like the Sega Genesis. In the case of creating these types of cues in the General MIDI world, the challenge becomes certainly greater. One of the best historical references for creating sound design in General MIDI is the genre of the classic cartoons. These were scored in a way that the action completely dictated the flow and profile of the orchestral accompaniment. In many cases, the sounds that we associate with cartoon sound effects were actually done by orchestral instruments. This contributes to my basic philosophy of thinking of an orchestra as the ultimate synthesizer.

With the simulated orchestra as a construction model, the General MIDI module can actually take on extreme roles. Besides all of the unusual "world" instruments, built-in sound effects, and other instrument "noises", taking something as simple as a piano patch high into the upper or lower registers of its usual range can come up with unusual and surprisingly useful material. Similarly with many of the tuned percussion instruments, some very mechanical and abstract passages can be generated considering that MIDI can play with un-human speed and accuracy. Many of the string embellishments employed by Bernard Hermann in Hitchcock films have yet to be exploited in the GM orchestration of games. Whether creating abstract sound design from electronic musical synthesis or re-synthesized orchestral samples, the main point is to be as creative and original as possible, while staying within the constraints of the mood, the context, and the pace of the production. This can be applied to both traditional linear media as well as interactive digital media, but with all of the real-time capabilities in entertainment software, it only makes the job of sound designer that much more dynamic and exciting. Having both the abilities to use MIDI and digital audio multiply your options and provide more solutions for delivering your sound design.

Additional possibilities include:

---

Reading Assignment

Read: Pro Tools 8 for Macintosh & Windows OR Complete Pro Tools Handbook

• Mixing Basics
  Notes by Walter Murch