toc

Team LiB
Next Section

Beginning Game Audio Programming
Beginning Game Audio Programming
by Mason McCuskey ISBN:1592000290
Premier Press © 2003 (352 pages)

This guide covers the challenges that you will face as you create sound effects and music for your games and gives you the information you need to face these challenges head-on. You’ll also learn how to compose dynamic music, program 3D sound, and more.

Has CD CD Content
Table of Contents
 | Beginning Game Audio Programming
 | Introduction
 | Chapter 1 | - | The Lay of the Land
 | Part One - Audio Engine Basics
 | Chapter 2 | - | Introduction to DirectAudio
 | Chapter 3 | - | Wave Audio Playback
 | Chapter 4 | - | Loading WAV Files
 | Chapter 5 | - | Control Freak
 | Chapter 6 | - | MIDI Playback
 | Chapter 7 | - | MP3 and WMA Playback
 | Chapter 8 | - | Ogg Vorbis Playback
 | Chapter 9 | - | CD Audio Playback
 | Chapter 10 | - | Tracked Music Playback
 | Part Two - Advanced Audio Engine Functionality
 | Chapter 11 | - | Dynamic Music
 | Chapter 12 | - | Scripts and Effects
 | Chapter 13 | - | 3D Sound Using DirectX Audio
 | Chapter 14 | - | 3D Sound Using OpenAL
 | Chapter 15 | - | Advanced Topics in 3D Sound
 | Chapter 16 | - | DirectPlay Voice
 | Chapter 17 | - | Audio Visualization
 | Conclusion
 | Appendix A | - | Creating Dynamic Music
 | Appendix B | - | Tips and Tricks for Creating Game Audio
 | Index
 | List of Figures
 | List of Tables
 | List of Sidebars
 | CD Content CD Content

Team LiB
Next Section


backcover

Team LiB

Back Cover

No matter how exciting the game and how realistic the visual effects, without catchy audio in the background it’s obvious that something is missing. New developments in technology enable you to do things with audio that weren’t possible before, but they also make it necessary to find a guide that walks you through the technical hurdles. Beginning Game Audio Programming covers the challenges that you will face as you create sound effects and music for your games and gives you the information you need to face these challenges head-on. You’ll learn how to compose dynamic music and program 3D sound. There’s even coverage of using DirectPlay Voice for real-time voice chat in your games! If you are comfortable with C++ and DirectX, then you’re ready to dive into the exciting world of audio with Beginning game Audio Programming.

About the Author

Mason McCuskey is a software engineer with 8 years of commercial programming experience. He has successfully managed his own game company since 1990. In 2000, his company, Spin Studios, was one of nine world finalists at the Game Developer’s Conference 2nd Annual Independent Games Festival. He has also written dozens of articles for gamedev.net and other Web sites. Mason is the author of Special Effects Game Programming with DirectX from Premier Press.

Team LiB


Beginning Game Audio Programming

Mason McCuskey

PREMIER PRESS

GAME DEVELOPMENT

Copyright © 2003 Premier Press, a division of Course Technology.

All rights reserved. No part of this book may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system without written permission from Premier Press, except for the inclusion of brief quotations in a review.

The Premier Press logo and related trade dress are trademarks of Premier Press and may not be used without written permission.

Publisher: Stacy L. Hiquet
Senior Marketing Manager: Martine Edwards
Marketing Manager: Heather Hurley
Manager of Editorial Services: Heather Talbot
Series Editor: André LaMothe
Acquisitions Editor: Emi Smith
Project Editor/Copy Editor: Jenny Davidson
Technical Reviewer: Heather Holland
Interior Layout: Bill Hartman
Cover Designer: Mike Tanamachi
CD-ROM Producer: Mason McCuskey
Indexer: Katherine Stimson
Proofreader: Sara Gullion

DirectX, DirectX Voice, DirectAudio, DirectMusic, and WindowsMedia are registered trademarks of Microsoft Corporation in the United States and/or other countries. OpenAL is a registered trademark of Loki Software, Inc. in the United States and/or other countries. Sonic Foundry, ACID, and Sound Forge are registered trademarks of Sonic Foundry, Inc. AmpLib is a registered trademark of AllMyPapers. GoldWave is a registered trademark of GoldWave, Inc. Cakewalk is a registered trademark of Twelve Tone Systems, Inc. Logic Audio is a registered trademark of Emagic. The BASS Sound System is a registered trademark of un4seen developments. fmod is a registered trademark of Firelight Technologies Pty Ltd. MikMod is a registered trademark of Miod Vallat. Ogg Vorbis and xiph.org are trademarks of xiph.org.

All other trademarks are the property of their respective owners.

Important: Premier Press cannot provide software support. Please contact the appropriate software manufacturer's technical support line or Web site for assistance.

Premier Press and the author have attempted throughout this book to distinguish proprietary trademarks from descriptive terms by following the capitalization style used by the manufacturer.

Information contained in this book has been obtained by Premier Press from sources believed to be reliable. However, because of the possibility of human or mechanical error by our sources, Premier Press, or others, the Publisher does not guarantee the accuracy, adequacy, or completeness of any information and is not responsible for any errors or omissions or the results obtained from use of such information. Readers should be particularly aware of the fact that the Internet is an ever-changing entity. Some facts may have changed since this book went to press.

1-59200-029-0

Library of Congress Catalog Card Number: 2002116167

03 04 05 06 07 BH 10 9 8 7 6 5 4 3 2 1

Premier Press, a division of Course Technology
25 Thomson Place
Boston, MA 02210

Another one for my wife—the kind of person songs are written about.

Acknowledgments

Several people contributed to making this book a success. To start, I'd like to thank André LaMothe, as well as Jenny Davidson, my editor, and Heather Holland, my tech editor, for making me sound more coherent than I actually am. Also, I'd like to thank Jeremy Martin for contributing some of the sample songs on the CD. And, of course, my family and friends—you know who you are!

About the Author

Mason McCuskey is a software engineer with 8 years of commercial programming experience. He has successfully managed his own game company since 1990. In 2000, his company, Spin Studios, was one of nine world finalists at the Game Developer's Conference 2nd Annual Independent Games Festival. He has also written dozens of articles for gamedev.net and other Web sites. Mason is the author of Special Effects Game Programming with DirectX by Premier Press.

Letter from the Series Editor

If you're like me, you probably appreciate music and sound effects in games, but they aren't your major concern during development. You generate a list of needed sound effects and music segments, set a spec like 16-bit stereo, 44kHz, hire someone, and be done with it. Then in-game you might use some audio panning for "3D" in a plane, or some doppler frequency shifting here and there, and change pitch in tunnels—but more or less audio is hardly something you're going to spend a lot of time on. If this describes you too, then we both are making a big mistake! Audio has matured so much in the past 5-7 years that it's incredible. The built in abilities of DirectAudio on the Windows platform, the technological advancements in full 3D sound modeling by hardware vendors, and the plethora of ultra-high compression lossy and lossless compression schemes for audio literally give audio engineers limitless abilities to create intensely engaging audio environments for games.

I think the thing we all forget is that "sound" is much more important than visuals. In fact, sound can create tension, excitement, anger, and so on. Try playing your favorite game without sound; it sucks, huh? There's no mystery there, sound has a direct and immediate affect on your brain; it's soothing or irritating at a primal level. In any case, it's something that we can't ignore anymore. Just like 3D graphics was a mystery that was figured out, it's time to figure out sound—learn, implement, and master.

Beginning Game Audio Programming is just such a book. From cover to cover you will learn just about everything you need to know to become the "expert" at sound programming wherever you work or play. This book isn't highly technical—meaning, it's not about digital signal processing, or Fourier transforms, impulse response, and subjects thereof; however, this book is a solid foundation and serious treatise on sound engineering for anyone interested in the subject.

Within this book, you will learn to use DirectAudio's features, such as digital file playback, DirectMusic, and audio scripting. In addition, you will learn how to play .MP3 files, and next-generation royalty-free formats like the Ogg Vorbis (like .MP3, but better!).

Of course, little details about how to play a CD through the system, mix sounds, timing issues, and more are all covered. Moreover, advanced concepts such as 3D sound modeling, and the use of the OpenAL system are covered as well. The book rounds out with discussions on audio visualization as well as using tools to generate sound and music.

Finally, content generation, licensing, and tactics for creating a complete ensemble of sounds effects and music are discussed. In closing, if you want to know more about audio programming as a game developer, audio engineer, or simply a hobbyist that wants to write his own .MP3 player, this book is for you. I dare you to find another book on this subject that is in plain "English" with practical examples, and excellent writing. The author, Mason McCuskey, has really taken his time and worked hard on this book to make it fun, informative, and engaging—not to mention the interesting sound effects he has made! With that in mind, don't listen to me, listen for yourself!

Sincerely,

André LaMothe

Premier Press Game Development Series Editor


Introduction

Game audio is frequently overlooked when it comes to designing and programming a good game. Often, developers spend so much time concentrating on the other facets of the game—for example, graphics and AI—that they neglect the music and audio side until it's nearly too late. That's a shame, because good game audio and game music have at least as much, if not more, power to instantly pull the player into the game world and play with his emotional state. Even the best graphics in the world will seem flat if they're not augmented with great sound.

Historically, books on game audio have been few and far between, because programming game audio wasn't that complex. Making a speaker beep or playing a simple sound in mono through a 1990s-era soundcard was fairly self-explanatory. Today, however, things like 3D sound, environmental audio, MP3 playback, and CD audio present difficult technical hurdles for even the simplest game.

This book is an introduction to some of the challenges programmers will face in implementing audio for their games. It is written to teach beginning game audio principles. The first half teaches you how to overcome the technical hurdles present in doing basic audio tasks—playing sound effects, loading and examining WAV files, and playing music in various formats (MIDI, MP3, and tracked music).

The second part of this book teaches some more advanced audio techniques. You'll learn how to compose dynamic music, and how to implement it in your game. You'll also learn about the basics of 3D sound programming, environmental audio, audio scripting, effects processing, and visualization. There's also one chapter devoted to using DirectPlay Voice for real-time voice chat in your game!

Most of this book uses DirectX and DirectX Audio, though there are a few chapters that use OpenAL Audio or generic Win32 audio APIs. Everything is in C++ and uses the Standard Template Library's low-level data structures (vectors, maps, and so on).

To get the most out of this book, you'll need to be comfortable with C++, though not necessarily an expert. However, you will need to understand innately how a Windows program works. You should be comfortable with event-driven programming and the everyday uses of the Win32 API, including writing GUI code for buttons, edit boxes, list boxes, and so on.

A passing knowledge of DirectX is also required. You should be familiar with how to obtain and release interfaces to various DirectX components, and should understand what DirectX does, why it's useful, and where you can go for help if you need it.

If you meet all of these requirements… turn the page and get going!


Chapter 1: The Lay of the Land

OVERVIEW

You've got to start somewhere, and for learning game audio, "somewhere" is here. No sample programs yet—just a heavy dose of concepts and vocabulary to ensure that you're ready for the chapters to come. If you have a background in computer music, you might know most of this already.

In this chapter, you'll learn core digital audio principles—how your computer hears and sings. You'll also be subjected to a short history of video game audio with a quiz at the end (no, not really). Finally, I'll present a couple of ways to wiggle out of writing your own game engine by licensing one, or downloading a free one. I decided to put the engine list first so that you wouldn't spend weeks with this book, get to the end, and say, "You mean I could have downloaded an engine?!?!"


COMPUTER SOUND PRINCIPLES AND VOCABULARY

For game audio, it all ends with a single byte stream. It may sound silly, but in reality, the entire focus of this book, and game audio in general, is on generating huge one-dimensional arrays of bytes. However, this simple core idea can sometimes be obfuscated by lots of technical vocabulary. To prevent total confusion, here's a look at some basic audio terms.

Sample Rate and Sample Quality

Pervasive throughout audio programming is the notion of the sample rate. To understand this, you need to learn how digital audio makes its way onto your hard drive in the first place (see Figure 1.1).

Click To expand
Figure 1.1: How your computer recotds audio.

Suppose you're making an action game and need a bone-chilling scream sound effect. You recruit your friend/sibling/co-worker and convince him to sit down in front of a microphone. You hit the record button in your audio software and cue him to scream.

As soon as you hit the record button, your audio software instructs your sound hardware to begin capturing the sound. Basically, when your system captures sound, it relies on the performance of a special chip on your sound card called an ADC, or analog-to-digital converter. Every so often, the ADC looks at the electrical current from the mic to determine how loud it is out there. It then sends this number back to the sound card's device driver, and ultimately through Windows and back to your audio program, which stores it in memory or writes it to disk. The piece of data is called a sample. (Note that often times the entire recording, or collection of samples, is also referred to as a sample, so you have to rely on context to figure out what's being said.)

The number of times per second samples are collected is called the sample rate. Sample rates are measured in Hertz, a unit for "per seconds." That is, 8,000Hz happens 8,000 times every second. Just as there are bytes and kilobytes, there are also Hertz and Kilohertz; 22KHz means that something is happening twenty-two thousand times per second. 8,000Hz is equal to 8KHz.

In a way, sample rates are to audio programming what resolution is to graphics programming. Just as more pixels mean higher-quality video, a higher sample rate means higher-quality audio. Typically, you'll record sound effects at 11025Hz, 22,050Hz, or 44,100Hz. 44,100Hz (44.1KHz) is the sample rate of CD audio; if you're listening to a track on a CD, some computer somewhere measured the loudness in the air 44,100 times every second. That's a lot of data.

AUDIO CLIP

Play Audio Clips 1.1 and 1.2 to hear the difference between a 44.1Khz sample rate, and an 8000Hz sample rate.

When the time comes to play the sound back, your computer simply reads the numbers and feeds them back out to the sound card (see Figure 1.2).

Click To expand
Figure 1.2: How your computer plays back audio.

There's another facet to the equation, though; there's also the notion of sample quality. Sample quality (also called bits per sample or sometimes just bits) refers to how accurately each sample is measured. The two common values are 8 bits and 16 bits. Obviously, 16 bits is better than 8. With 8 bits, you can only express 256 distinct levels of loudness, but with 16 bits, you get 65,536 unique levels, making you tremendously more sensitive. Taking this to imaginary extremes, if you had only 1 bit, you could only express "loud" and "quiet," and that wouldn't make for a very good sample at all. If sample rate is akin to video resolution, then bits per sample is akin to color depth—the more bits, the more precise you can be.

Start Sidebar
Why 44100 Hertz?

Why are CDs sampled at 44,100 Hertz, anyway? It seems like such an odd number. To answer this question, we have to dive back into audio history. Before CDs, digital audio was stored on video tape—a hack rivaling the best of them. The tapes were designed to store and play back 60 frames per second of video data. Each frame of video had 245 lines, and each line had three samples (for red, green, and blue). That gives us 245 × 3 × 60, or 44,100 samples.

End Sidebar
AUDIO CLIP

Play Audio Clips 1.3 and 1.4 to hear the difference between 8- and 16-bit samples. Listen closely though; these differences are not nearly as noticeable as the sample rate differences.

Stereo Sound

Up until now, this chapter has focused on one channel sound, also called monaural sound, and abbreviated mono.

To capture stereo sound, you use two mics in two different places (one to the left of the sound source, and one to the right). Most of the time, you don't capture audio like this, though; instead, you use a mixing station or mixing software to explicitly control the sound. The mixing station allows you to pull in mono samples and control which speaker they come out.

Regardless of how you do it, eventually you end up with a stereo sample, which has two channels. CD quality is synonymous with 16-bit stereo sound sampled at 44100Hz. For every second of CDquality audio, you have 44,100 samples at 16 bits (2 bytes) per sample—88,200 bytes per channel per second. Stereo sound means two channels, so you have 88,200 × 2, or 176,400 (about 176K), bytes for each second of audio. That's why something as huge as a CD, which can store 650MB of data, can only store 74 minutes of CD-quality audio.

Start Sidebar
The Story behind CDs

My father-in-law, a musician, once told me that the reason CDs hold 74 minutes of music is because the powers that be wanted to listen to Beethoven's Ninth Symphony in its entirety, without interruption. The engineers, always anxious to please, calculated the length of this symphony as 74 minutes, and came up with a physical specification for a disc that could store that much audio. I find it fascinating that a classical musician who died several hundred years ago had a very large hand in shaping one of today's most omnipresent pieces of audio technology.

End Sidebar

Sound Formats, Compression, and Codecs

Of course, the world would be a pretty depressing place if the only way you could store CD-quality sound was by burning through 176KB per second. A typical 3-minute song would burn through 31,680KB (roughly 30MB)! This would make record labels happy, because they wouldn't have to deal with people swapping songs online. Even on a fast broadband connection, 30MB is a lot of bytes to transfer for three minutes of audio.

Clever people have come up with ways around this, though. Several audio compression algorithms have come along. These algorithms represent the same data in more compact ways, allowing you to store the same information in less bytes. For example, say you have a mono WAV file with two seconds of silence in it. Ordinarily, if you were using CD quality, you'd write out 88,200 bytes with the same "zero" value. However, you could compress that by saying something like, "Okay, the next 88,200 bytes are zero," and not bother to write out each one. That's what a common compression algorithm called RLE (run-length encoding) does.

RLE is a very simple algorithm (see Figure 1.3). A more advanced one goes by the name of MPEG Layer 3. In the days of 8.3 file names, sound files compressed using the MPEG Layer 3 algorithm were given the three character extension MP3, and the rest is history. MPEG itself is an acronym for the Moving Picture Experts Group. For more history on the MP3 format and the people who made it (Fraunhofer IIS-A), check out http://www.iis.fhg.de/amm/techinf/layer3/.

Click To expand
Figure 1.3: The algorithmbehind the simplest form of RLE compression

MP3 is a "lossy" compression algorithm, which means that some information is lost when you compress a WAV into MP3. MP3 works by making sure that the information that's lost is information you can probably live without. For example, a lot of high frequency things, such as cymbal crashes and vocalizations of the hiss of an "s" sound or a crisp "k" sound, are lost. Usually this loss is imperceptible to the average listener, but with a good pair of speakers and a keen ear, you can hear the difference. Try it sometime—go to a quiet place, and listen to your favorite song on CD, then listen to the same song in MP3. If your speakers are good and you're young or have taken good care of your ears, you'll be able to hear the difference.

Other compression formats are out there. There's ADPCM, an acronym for Advanced Differential Pulse Code Modulation, Ogg Vorbis (an open source, patent-free audio compression algorithm that's quickly gaining popularity), and several lesser-known formats.

These pieces of code that implement these algorithms are called codecs, an acronym for compressor/decompressor (don't you love all these audio acronyms?). Contained in a WAV file is the name of the codec it was compressed with; Windows provides several of the most common codecs, and automatically uses the right one based on the tag in the WAV file. If you try to play a WAV file that contains a tag for a codec not installed on your system, you won't be able to until you hunt down the codec it was made with. Happily, this is not a common occurance—99 percent of the WAVs out there are PCM or ADPCM.

Mixing Sound

Of course, there's more to good game audio than just playing WAV files. Often you'll want to play more than one sound effect at once, and that's where audio mixing comes in.

The easiest way to play two sounds at the same time is simply to add their samples together (see Figure 1.4).

Click To expand
Figure 1.4: Simple additive sound mixing

So, if you have a scream sample and a growl sample, and you add them together, you'll hear both the scream and the growl at the same time. This doesn't seem like it should work, but it does.

Of course, to get good results requires more than just adding the sound together. Your next thought might be, "Well, what about averaging them together?" The problem with that is it makes each individual sound get softer. If you're playing three sound effects at once, each sound effect will have one-third the volume it has when played alone.

There are different solutions to this, some involving logarithmic lookup tables and other advanced math stuff. The good news is that unless you're an expert working on something very advanced, you never have to worry about mixing samples yourself; that's all done by the Windows OS. The closest you'll usually get is to just say, "Hey DirectX Audio (or hey Windows), mix this sample at this volume with that sample at that other volume," and then through some black magic it works.


A BRIEF HISTORY OF GAME AUDIO

Game audio has undergone many changes throughout the years. When fanny packs were high style and Pac-Man was hi-tech, it was much too difficult for games to use samples as sound effects. The most powerful gaming machines of the ‘80s had between 8KB and 128KB of memory, and a typical sound effect, even at poor quality, was between 30 and 50KB.

Early PC audio revolved around what's known as frequency modulation synthesis, or FM synth. I remember the first Sound Blaster card I bought for my 386, proudly proclaiming that it supported FM synth.

AUDIO CLIP

Play Audio Clip 1.5 to hear early FM synth music.

FM synthesis uses algorithms to re-create sound waves on the fly. Instead of storing an actual sampled wave, FM synth approximates that wave by mathematically re-creating waves of different shapes, frequencies, and volumes. For example, FM synth says something like, "Well, if I create this kind of wave with this frequency, and combine it with another wave of frequency, y, I get a sound that's pretty close to a trumpet."

Unfortunately, pretty close didn't sound too great, especially on instruments like trumpets and snare drums. Flutes and other instruments whose sound was close to a pure sine wave sounded tolerable, and synthesizer instruments sounded fine, but everything else was pretty bad—cheap electronic keyboard bad.

Fast forward a couple of years to the next revolution, which occurred when the Amiga catapulted the MOD music format into popularity. MOD music sounded vastly better than FM synth because it used real samples instead of mathematical approximations. The MOD music format originally supported four tracks, and at any given time each of those four tracks could be playing one sample at any pitch. Computers now had enough memory to store instrument samples in memory, and enough processing power to manipulate the pitch of those instruments in real time. The sound quality wasn't great—most of the samples were in the 11KHz or 22KHz sample rate, but it was vastly better than FM synth. Drums especially sounded much better because they were recordings of actual drums.

AUDIO CLIP

Play Audio Clip 1.6 to hear MOD music from the early days of tracking, and listen to Audio Clip 1.7 for a clip from the later days.

This technique was a primitive form of what's now known as Wave Table synthesis. Wave Table synth gets its name from the fact that the creation of the music relies on a table of sampled waves. Check out Figure 1.5 for a comparison between FM synth and Wave Table synth.

Click To expand
Figure 1.5: A comparison between FM synth and Wave Table synth.

It's debatable which tracked music format came first, but once the idea of tracked music caught on, dozens of trackers sprung up: StarTrekker, ProTracker, NoiseTracker, Oktalyzer, and many others. A demo group calling themselves the Future Crew created and popularized the STM (ScreamTracker) and, later, S3M (ScreamTracker 3) music formats, which allowed more channels, better effects, and higher instrument sample rates. Impulse Tracker took tracked music even further, bringing even more advanced sampling abilities to the video gaming masses. For a few years in the early to mid nineties, tracked music was all the rage.

Tip

Descendants of the MOD music format are still in use today. Frequently, games distributed online use some form of tracked music because it's much smaller than sampled songs. A typical tracked piece of music is in the 200–300KB range, whereas a similar MP3 is usually measured in megabytes.

Tracked music has a very unique sound to it. Often, game developers use it for techno songs. I've included a small spattering of tracked music, including some tracked songs I created back in the day, on the CD. Tracking is still somewhat popular, so I've also thrown in links to a couple of tracking Web sites.

MIDI

Parallel to all of this FM synth and tracked music development, there was another revolution occurring—this time in the land of MIDI. MIDI is an acronym for Musical Instrument Digital Interface. MIDI version 1.0 went gold in 1983, and was quickly adopted by professional digital musicians. As the price of MIDI synthesizers fell, amateurs and the public in general started using MIDI.

MIDI is very similar to tracked music, only with a slightly different philosophy behind it. Tracked music started as a closed system—the CPU did everything; it stored the instruments, stored the notes, applied the effects, and played the finished product through its own speakers.

Conversely, MIDI's philosophy is that of several machines working together to create great sound. A typical professional MIDI setup includes several devices, each with its own purpose. A MIDI buff might have a drum machine, a synthesizer, multiple instrument banks, a computer, and a few effects processors. All of these devices need to talk to each other, and MIDI was developed so they could.

At its core, MIDI is really just a special kind of networking protocol. It specifies how notes are stored, and how they're transmitted across wires to different devices. In particular, the MIDI interface specified a set of 128 different instruments—everything from flutes to violins, to special effects like gunshots and helicopters. These 128 different instruments were called the General MIDI Level 1 Sound Set, or GM sound set for short. That way, when a MIDI synthesizer got a message saying, "Play a fifth octave B-flat at this volume using instrument #47," it would know what instrument had code 47, and could re-create the appropriate timbre. The GM sound set was sort of like the ASCII character set for MIDI. It's important to note, however, that although the MIDI specification dictated which number went with which instrument, it didn't say a thing about how the instruments actually sounded.(Imagine your MIDI data as sheet music for a player piano. The song data—the sheet music—is the same sequence of note on/note off commands, but how good it sounds is directly related to the quality of the player piano that plays it.) The sound of the instruments was left up to the MIDI hardware manufacturer. Most professional hardware manufacturers sampled real instruments and stored the samples in ROM. Most consumer-level sound hardware used FM synth to re-create the GM sound set, because storing real samples in ROM would have made the cards too expensive for your average consumer (remember, this was back when a couple of megabytes of memory cost what half a gig does now). It was much more efficient to store the mathematical recipes for making "close enough" instrument approximations than to store the sampled instrument waves themselves.

The GM sound set was broken down into the categories shown in Table 1.1.

Table 1.1: General MIDI Sound Set Categories.
Numeric Range | Family
1–8 | Pianos
9–16 | Chromatic Percussion
17–24 | Organs
25–32 | Guitars
33–40 | Basses
41–48 | Strings
49–56 | Ensembles
57–64 | Brass Instruments
65–72 | Reed Instruments
73–80 | Pipe Instruments
81–88 | Synth Leads
89–96 | Synth Pads
97–104 | Synth Effects
105–112 | Ethnic Instruments
113–120 | Percussion Instruments
121–128 | Sound Effects and Special Effects

DLS1

So, in the nineties, all of these different approaches to game music were swimming together. MIDI and tracked music were the two most popular, and they both had their benefits and drawbacks.

MIDI was great because MIDI files were typically very small (less than 100KB) because the instruments of a song weren't stored with the song itself.

Unfortunately, this meant that good playback of a MIDI song depended significantly on the quality of the instruments. Professionals had instrument libraries with high-quality samples of the 128 General MIDI instruments, but everyone else (game players in particular) had shoddy approximations of those instruments, frequently using FM synth instead of sampled sound. This meant that a MIDI file would sound great when played through the (professional) setup of the composer, but would sound awful when played on the typical consumer (game player) hardware.

The alternative was tracked music. Tracked music was great because the instruments were contained directly in the song. This meant that the songs sounded the same regardless of the hardware that played them. Also, because tracked music used actual song samples instead of FM synth, the quality was often better, and musicians could create their own instruments instead of being limited to the 128 MIDI ones. Some tracked music actually had lyrics created by making custom instruments for each word or phrase in the song!

The drawback was that these songs were huge (remember, 100–200KB was a lot back then— almost a quarter of a 3.5 inch disk's capacity, and that was for just one song). Also, they required more CPU power because the CPU had to calculate the appropriate sample pitches and mix all the data together itself.

Progress eventually saved the day. Once hardware got faster and storage space got bigger, the scene was set for tracked music and MIDI music to merge into one unstoppable super format. In 1997, the MMA (MIDI Manufacturer's Association) approved and ratified the specification for Downloadable Sounds Level 1 (DLS1). DLS1 gave consumers real wave-table synthesis (that is, synthesizing instruments by using a table of wave samples to represent their notes).

In fact, DLS1 took MIDI one step further by allowing each instrument to have multiple samples. The samples were broken out by region; you could have one "low note" sample for the low notes of a piano and another "high note" sample for upper octaves. In fact, you could have up to 16 different regions for each instrument. To do this same thing in tracked music required a little trickery.

DLS1 greatly advanced the popularity of MIDI as a game music format. The DLS spec included guidelines on how the instruments should sound, and all of a sudden, MIDI files played on consumer hardware began to sound very similar (and certainly close enough) to their professional hardware counterparts. Game music composers could now safely write MIDI knowing that what they heard would be very close to what the player heard. Further, musicians were no longer limited by the 128 GM instruments—they could sample whatever they wanted and include their custom instruments in the MIDI song, like the tracked music formats could.

Microsoft further solidified DLS's reign by including, as part of DirectX, a GM sound set of sampled instruments created by Roland, (a highly respected manufacturer of musical instruments, MIDI keyboards, and MIDI software). Now, as long as game developers used that same DirectX software synthesizer, what they heard was what the player heard.

DLS2

The MMA knew that when you had a hit, as they so clearly did with DLS, you made a sequel. In 1999 they ratified the DLS level 2 specification. DLS2 did a number of technical things to fix technical shortcomings in the DLS1 specification. I won't list them all here, but suffice it to say, the DLS2 is better than DLS1, and there was much rejoicing.

DirectMusic 8 was the first to support DLS level 2. Microsoft's synthesizer, included in DirectMusic, is DLS level 2 compliant and requires very little CPU usage, even on slower machines.

CD Audio (Redbook Audio)

I haven't talked about CD audio yet; I've decided to save the best for last. For commercial games (distributed on a CD), CD audio has been by far the music storage mechanism of choice.

Redbook audio allowed games published on CD to skip the whole MIDI/tracked music mess. Songs on CD sounded exactly the same no matter where they were played. The CD-ROM drive did most of the work associated with playing a track, so there was little CPU usage, and a CD held so much data (650MB) that storage space was a non-issue.

Tip

CD audio is commonly called Redbook audio because the original spec came from Sony in a red binder. The Redbook audio spec is also the exact specification audio CDs are written to. All audio CDs are made according to Sony's original Redbook specification. If you have a game stored on CD and you can play its music in your car's CD player, the game is using Redbook audio as its music format.
Start Sidebar
Redbook Audio: Squeezed Out?

The first CD prototype was developed by Philips in 1979, and the first CD player was released by Sony in 1983 (and cost $1,000). The first album you could play on it was Billy Joel's 52nd Street. In 1983 there were 30,000 players and 80,000 discs in the US. Seven years later there were 9.2 million players and 288 million CDs.

The skyrocketing popularity of the CD, combined with the seemingly limitless storage capacity it made available to PCs, earned it the reputation of being the best way to store audio content for video games. However, nowadays, Redbook audio is becoming less attractive for game development. 650MB isn't nearly as much as it once was, and why use uncompressed Redbook audio tracks when you can deliver (virtually) the same audio experience using MP3s, for a tenth of the storage space? Also, CD music has no chance of being dynamic. But MIDI synthesis, or even sampled music snippets, can form the building blocks of dynamic soundtracks that can change based on what's going on in the game.

For many games, Redbook audio remains a great choice, but it's lost a lot of ground lately to compressed and dynamic music scores. The constraints that once made Redbook audio a great idea are slowly becoming a thing of the past, and it remains to be seen whether Redbook audio will continue to be a popular format for games.

End Sidebar

So Where Does All This Leave Me?

Of the technologies I've presented, three—tracked music, MIDI/DLS/DirectMusic, and Redbook audio—are still in use today. FM synth went the way of the dodo along with MS-DOS and Windows 3.11.

One of the most important decisions you'll make regarding your game audio is which music format to choose. Here's a somewhat ad-hoc guide to choosing the right music format. Keep in mind as you read this that I'm using terms generally; for example, feel free to substitute your favorite compressed audio format whenever you see "MP3" mentioned next. When I refer to DirectMusic, I'm talking about music authored using the DirectMusic producer.

First, decide if you want your music dynamic or static.

If you want dynamic music, DirectMusic is probably your best bet. Of course, you could retrofit or soup-up an old tracked music engine and turn it into a dynamic music engine. This might be okay if you want only simple dynamic music, but be aware that to get a dynamic music engine rivaling DirectMusic's is going to take a lot of work. You could also write your own system for constructing dynamic music based on sampled segments of music; for example, you could sample a few different variations of the same song portion, and write code to select one at random, or based on the game situation. If you want static music, you can't eliminate any formats just yet. If your game is going to be sold online, I strongly suggest tracked music for its small size.

Redbook audio and compressed, sampled music are powerful only if you have the money and means to create true professional quality music tracks, or if you have some specialized music score already written that you'd like to just use as is. If you're targeting old hardware, Redbook is a slightly better choice because it uses virtually no CPU power. Otherwise, my advice would be to go with a compressed format, like MP3, WMA, Ogg Vorbis, and so on. Also, note that for an independent developer, Redbook is a hassle because to hear your music in-game, you either have to write some abstraction layer in your game and "fake" a CD track using another method (adding complexity to your system), or you have to burn a new CD-R (or, slightly better, CD-RW) each time your music changes.

Be aware that if you do use MP3, there might be licensing issues involved. Usually you can code things so that you don't have to pay a licensing fee (you piggyback on the licensed decoder already on the user's system), but to do that requires relinquishing a bit of control (see Chapter 6, "MIDI/DirectMusic Playback," for details).

In the end, for commercial games, it usually comes down to a choice between MP3 (for static soundtracks) and DirectMusic or your own system for dynamic music. For online games, it's usually a choice between MP3 and tracked music.

At any rate, I hope you've enjoyed this whirlwind tour of game music through the ages. Your CD contains additional links to Web pages that can supplement what I've talked about here—check them out!


BUYING OR BUILDING AN AUDIO ENGINE

This book's main goal is to teach you how to build an audio engine. In the coming chapters, you'll integrate basic and advanced features into an audio engine. And by the end of the book, you'll have a good idea of how an audio engine works, as well as a completely functioning audio engine you can use in your own games.

Of course, not everyone need write his own audio engine. The glory of capitalism allows you to trade your money for your time and buy an existing audio engine. Here's a list of commercial, shareware, and freeware audio engines that you can use if you don't want to write all this code yourself. To be fair, these are listed in alphabetical order—no favorites. Also, you can find demo versions of several of these engines on your CD.

Tip

Please note that this information was collected during early 2003, and little things have changed since then. Consult the vendor's Web site for the most up-to-date information.

BASS

  • Free for freeware

  • $100 for shareware

  • $995/$2750 for commercial

  • http://www.un4seen.com/music/

BASS is a very comprehensive sound engine for the Windows platform. It provides support for all of the popular sound file formats in use today (WAV, MP3, WMA, and Ogg Vorbis), requires only a DirectX 3 interface (so you can use it on NT), understands EAX and 3D sound, and supports the most popular tracked music formats (MOD, XM, IT, S3M).

Another BASS strength is that it supports a variety of languages. Of course, there's a C/C++ API, but there are also APIs for Visual Basic, Delphi, MASM, and TMT Pascal.

The BASS engine is in wide use today, mostly by independent developers.

MikMod

  • Freeware

  • http://www.mikmod.org

MikMod is a freeware, cross-platform tracked music library, originally written by Jean-Paul Mikkers (MikMak), and now maintained by Miod Vallat. Several other developers have also contributed to the project.

It supports all the popular formats, and can run under lots of different OSes—MacOS, BeOS, Java, UNIX, and of course, Windows.

FMOD

  • Free for freeware

  • $100 for shareware

  • $1250 for commercial games

  • http://www.fmod.org/

FMOD is a very comprehensive audio engine available on virtually every platform out there, including PS2, Xbox, and GameCube. It fully supports a variety of C++ compilers, as well as Visual Basic, and includes advanced features such as EAX support and occlusion/obstruction.

ModPlug

  • Free

  • http://www.modplug.com/

If all you want is a quick and easy way to play tracked music, ModPlug could be the answer. ModPlug has a very simple interface, coupled with a powerful playback engine that supports all of the tracked music formats you're likely to see in use today. It has interfaces for C++, Visual Basic, and Delphi, and there's also an ActiveX control available.

Miles Sound System

  • $4,000 for a single product license

  • $12,000 for a site license

  • $22,000 for a cross-platform site license

  • http://www.radgametools.com

The Miles Sound System, a product of RAD Game Tools, is by far the most comprehensive game audio library available. Originally released by John Miles in the summer of 1991, it has currently been used in over 3,000 (no typo, that's three thousand) games. It does everything you want it to. In the immortal words of Ferris Bueller, "If you have the means, I highly recommend picking one up."


CONCLUSION

You've now learned the vocabulary and background information you need to know before attempting to write an audio engine. As always, for more information, check out the links I've provided on your CD—they'll take you to a whole slew of fun Web pages devoted to the topics covered here.


Part One: Audio Engine Basics

Chapter 2: Introduction to DirectAudio
Chapter 3: Wave Audio Playback
Chapter 4: Loading WAV Files
Chapter 5: Control Freak
Chapter 6: MIDI Playback
Chapter 7: MP3 and WMA Playback
Chapter 8: Ogg Vorbis Playback
Chapter 9: CD Audio Playback
Chapter 10: Tracked Music Playback

In this part of the book, you begin the creation of a full-fledged audio engine. This audio engine will be written in C++, using the Standard Template Library. It will operate with Microsoft DirectSound and DirectMusic.

In each chapter of this first part, you'll learn how to implement one or two critical features in the engine. You'll begin by adding sound effects to the engine. You'll start with a low-level introduction to digital sound, and learn the low-level DirectX routines that enable you to create sound effects by pumping bytes into a sound buffer. From there, you'll learn how to read and inspect WAV files, as well as how to play and loop them.

After the sound effects are in and working, you'll turn your attention to game music, learning how to read and play back a variety of musical formats. Of course, you will learn how to play an MP3, but you'll also learn how to read and play back MIDI, Ogg Vorbis, and a variety of tracked music formats (S3M, IT, XM, and so on).

By the end of Part One, you'll have an audio engine ready to be used by your game. It won't do anything terribly complex—that's in Part Two—but if you want just basic functionality, it's certainly up to the task of playing back sound effects and music.

Okay, let's get to it!


Chapter 2: Introduction to DirectAudio

Download CD Content

OVERVIEW

If the longest journey begins with a single footstep, then the greatest audio engine begins with a single tone—specifically, a 440Hz "A" tone that this chapter's sample program will teach you how to produce.

This chapter is an introduction to DirectX Audio. If you've dealt with DirectX before, a lot of this will come as no surprise to you—the essentials of getting an audio interface, setting the cooperative mode, and so on, are very similar to their counterparts in Direct3D. The most interesting parts are toward the end, where I explain the basic math behind generating tones.


DIRECTX AUDIO BASICS

I know you're probably dying to get to some sample code, but there are a few conceptual things you need to know first. That's what this first section is all about. You'll get to the code in the next section.

DirectMusic, DirectSound, or DirectX Audio?

In the beginning, there was DirectSound. Then there was DirectMusic. And then, just when you thought it was safe, Microsoft (in DirectX 8) introduced a third name: DirectX Audio.

What's unfortunate is two out of the three terms are misnomers. You can play sound effects using DirectMusic, and you can play music using DirectSound. Funny how that works, isn't it? Your sanity will last longer if you think of these interfaces not in terms of, "Okay, this one's for sound effects, and this one's for music," rather, in terms of low-level and high-level APIs. Table 2.1 summarizes the differences between these three terms.

Table 2.1: Definitions of DirectX Audio Terms.
DirectWord | Meaning
DirectSound | The low-level audio API for Windows. This API concerns itself with the minutia of getting raw sound bytes in and out of the sound card.
DirectMusic | The high-level audio API. This API takes a step back and allows you to say things like, "Okay, I'm not sure if this is a WAV file, a MIDI file, or a SGT (dynamic music segment) file, but I want you to load it and play it."
DirectX Audio | Microsoft's name for the audio API as a whole—the house in which DirectSound and DirectMusic live.

The giant invisible Microsoft hand is pushing us all to use DirectMusic as "the" audio API. As of DirectX 9 it is still possible to talk directly to the low-level API (DirectSound); however, it is very common to use just the DirectMusic routines and never bother with the low-level DirectSound API (unless you want low-level control over your WAV files). I wouldn't be surprised if in the future Microsoft sealed off the DirectSound API and made the DirectMusic API the only interface available in DirectX Audio.

However, the rest of this chapter will focus on DirectSound, not because it's easier than DirectMusic or that it's the better way to go, but simply because DirectSound allows you to strip away a lot of the layers Microsoft built and deal directly with sending bytes to the sound card. For people new to game audio programming, this is the simplest possible example, and for people experienced with game audio, it's probably the method with which they're most familiar.

Primary and Secondary Buffers

At its core, DirectSound relies on the concepts of primary and secondary buffers. A buffer in this case is a 1D array of bytes that will eventually become audio (as explained in the last chapter).

Each application has exactly one Primary DirectSound buffer. This is easy—what's in the primary buffer is what comes out of the speakers, period.

An application needs at least one, but can have as many Secondary DirectSound buffers as it wants. Secondary buffers hold a single sound or stream of audio data.

These buffers fit together as illustrated by Figure 2.1. Essentially, you store your various sounds in secondary buffers, and then mix them together into the primary buffer. Usually, DirectSound handles the mixing, so you never actually touch the primary buffer yourself. However, you can gain access to it, but keep in mind that if you want direct access to the primary buffer (to do your own mixing), you can't create any secondary buffers. But, unless you're doing something very strange, you won't want to do this.

Click To expand
Figure 2.1: DirectX Audio mixes secondary buffers into the primary buffer, then out to your speakers.

Additionally, buffers can be created in hardware or in software. A hardware buffer lives on the sound card's RAM, whereas a software buffer lives in system memory.

Tip

Nowadays, there's no real difference between hardware buffers and software buffers. Way back in the nineties, PCs had a very slow ISA bus. So slow, in fact, that it was often better for sound cards to store commonly used sounds in their own RAM, located on the sound card itself. That way, when the audio chip needed to mix a sound, it wouldn't have to transfer all those bytes over the slow ISA bus.

Today's PCI bus is much faster, to the point where it no longer makes sense for sound cards to have their own RAM—everything is stored in system memory and transferred over the wicked-fast PCI bus when needed. The PCI bus has more than enough bandwidth for audio data (though it's still a bit too slow for graphics data, which is why you see video cards with on-board RAM).

I'm glossing over this terribly; if you want a more technical explanation, read the MSDN article, "Sound Cards,Voice Management, and Driver Models," by Brian Schmidt, available at
http://msdn.microsoft.com/library/default.asp?url=/library/enus/dnaudio/html/ntk_sc.asp.

Streaming

Okay, so you know that sound effects go in secondary buffers. But what happens when the secondary buffer is too small to hold the whole sound effect? For example, if you want to play digitized music for a 15-minute in-game cut scene, you can't just load the whole 15 minutes of audio data into memory all at once (well, maybe you could, but that'd be silly, and it would take up a lot of memory). A better solution is to load the portion of audio data you need right before it's needed—in other words, to stream it off the disk.

Streaming is a common technique that works very well (thanks mostly to the fact that a hard drive or CD-ROM drive can read bytes much faster than a speaker plays them). You'll become more familiar with streaming as this book goes on; for now, just understand the concept, as illustrated by Figure 2.2.

Click To expand
Figure 2.2: Streaming a big sound file requires loading pieces of it just as they're needed.

When you stream audio data, instead of loading all of the data into memory at once, you treat your secondary buffer as a window scrolling over your sound data and load the data as you need it. For example, in Figure 2.2, you see that ten seconds of audio data (the beginning of a 3-minute song) have been loaded into the buffer. The code then instructs the buffer to start playing, and the song starts to play. As the song is played, the computer reads new data from the disk and places it in the buffer, right behind the play cursor, overwriting the data that's just been played. Eventually, the play cursor wraps around to the start of the buffer and starts playing the new data. Things continue this way until all data has been played.

Obviously this whole setup depends on being able to read data faster than you can play it. Happily, hard drives today can read bytes much faster than the speaker can play them.


HOW TO SET UP YOUR COMPILER FOR DIRECTX AUDIO

Before you can do anything else, you must learn how to properly configure your compiler to build applications that use DirectX Audio. There are two main things you need to do: you need to tell your compiler where it can find the DirectX headers, and where it can find the DirectX libraries. This section will teach you how to do that, assuming you're using Visual C++ version 6. If you're on another compiler, or if you're in Visual Studio .NET, a.k.a. Visual Studio version 7, the concepts are the same, but the actual buttons and menus you have to click will be different.

Global versus Per-Project Configurations

DevStudio has two different configuration sets. There's the global configuration, which applies to all projects you load on your computer, and is kept inside your computer's registry, and there's a per-project configuration, which applies only to the project you have loaded (and is stored inside the DSP project file). The per-project configuration takes precedence over the global configuration (that is, DevStudio first looks for what it needs using the per-project settings, and if it can't find it, it uses the global settings).

Building the Samples in This Book

All of the sample programs in this book assume that you have the DirectX paths set up in your global configuration. (See Figure 2.3.) When you install DirectX, there's an option that lets you automatically add the appropriate paths to DevStudio's configuration. If you didn't select that option, complete the following steps to add the paths.

  1. Select Options from the Tools pull-down menu; then select the Directories tab from the dialog that appears (you may need to scroll the tabs using the scroll buttons on the right).

  2. Select Include Files in the Show Directories For… drop-down box.

  3. Click the button to the left of the red X to create a new entry, and type in the full path to the DirectX include directory. The DirectX SDK installs itself by default at C:/dxsdk, so unless you specified a different location, you should enter C:\dxsdk\include. Do the same thing to enter the path to the samples include directory, normally C:\dxsdk\samples\Multimedia\Common\include.

  4. Drag these two folders up to the top of the list, making DevStudio check these locations before any other locations. This is important, because the default installation of DevStudio has include files for an older DirectX SDK, and grabbing those by mistake will result in errors like "LPDIRECTSOUND8 doesn't exist."

  5. Select Library Files from the Show Directories For drop-down, and follow the same process to add the full path to your DirectX library location (usually C: \dxsdk\lib).

  6. Click OK, then exit DevStudio and re-launch it. DevStudio saves its global configuration when it exits, so if you get in the habit of always exiting after you change the global configuration, you'll protect yourself from having to redo what you changed if (and we all know this is a rare occurrence) DevStudio crashes on you before you exit.

Click To expand
Figure 2.3: The global include path settings.

Congratulations, you should now be all set. If you still get fatal errors (for example, "could not open include file dsound.h: no such file or directory" ), or if you're getting similar link errors, you may not have put in the right paths.

Tip

You don't need to do this to get the samples to run, but if for your own projects you want to change the perproject configuration settings, you can get to them by selecting Settings from the Project menu. The include directories can be found under the C/C++ tab in the Preprocessor category, and the library directories can be found in the Link tab under the Input category.

Making Your Own Projects

When dealing with DirectX, there are always a few steps you need to complete just to get a bare-bones application up and running. Here, I've written down the steps you need to create the Ch2p1_ToneGenerator workspace.

Tip

The Ch2p1_ToneGenerator workspace is included on your CD-ROM. I provide these steps only as a reference for when you create other projects.

  1. Select New from the File menu; select the Projects tab, and select the project you want (for example, Win32 Console Application, Win32 Application, or if you're into MFC, MFC AppWizard EXE). Also set up the project name and location.

  2. Work through the wizard that appears. There are no special settings you must have here— pick what you like. For ToneGenerator, I started with a console "Hello World" app.

  3. When the wizard finishes, you should have a program that compiles and runs without error. If you do not, you have a bigger problem—call Microsoft and tell them their stock source code doesn't compile.

  4. Select Settings from the Project menu. On the left side of the dialog, in the Settings For drop-down, select All configurations.

  5. I like to jump to the Precompiled Headers category and select either Not using precompiled headers or Automatic use of precompiled headers. For small projects like this, I think that using precompiled headers through the stdafx.h and stdafx.cpp files are nothing but a pain.

  6. Go to the Link tab and add dxerr8.lib winmm.lib dsound.lib dxguid.lib to the front of the Object/Library modules edit field. These libraries are required for most of the sample programs and most of the DirectX Audio apps you'll create.

  7. Go to the Input category and add the relative path to your DX libraries in the Additional Library Path edit field.

  8. Click OK to dismiss the Project Settings dialog.

  9. Go to your fileview and remove stdafx.cpp and stdafx.h from the project. Also, in all of your source files (you should have just one at this point), remove the line that includes stdafx.h. Because you turned off precompiled headers in step five, these files are no longer needed.

  10. Stdafx.h contained some useful lines that defined WIN32_LEAN_AND_MEAN and included stdio.h. If you want these, you should put them into your main C++ file or one of your own global header files.

  11. Be tidy by deleting stdafx.h and stdafx.cpp from your drive.

  12. Verify that everything still compiles. If not, you probably made a mistake in one of the previous steps, or something "very strange" is going on.

Tip

You can also easily create DirectX applications by selecting DirectX AppWizard from the projects list. This AppWizard will walk you through the creation of a DirectX app similar to some of the DirectGraphics samples included in your SDK. Don't use it to start making a game, but do use it if you want to quickly get a skeleton app together so you can test something.

Even though you've removed all traces of stdafx.h, you might still see it in the External Dependencies folder. This folder gets refreshed when a compile completes without error, so just build your project and stdafx.h should disappear.


TONEGENERATOR: A SIMPLE DIRECTX AUDIO EXAMPLE

Now that your compiler is set up, you're finally ready to dive into your first DirectX audio sample program! If you want, follow along by loading the Ch2p1_ToneGenerator sample into your IDE.

AUDIO CLIP

Play Audio Clip 2.1 to hear the output from this chapter's sample program.

What Is ToneGenerator?

ToneGenerator is a simple console application designed to illustrate the basics of initializing and pumping sound through DirectX Audio. ToneGenerator aims to teach you the basics of using DirectX Audio, as well as the core low-level behavior of sound. It is not terribly complex nor terribly useful, but I think it does a great job of stripping away all the abstractions and showing you, at a bare-bones level, how sound is created digitally.

ToneGenerator generates a tone of a specific frequency. It does this by calculating the bytes for a sine wave and pumping those bytes into a secondary buffer, which DirectX Audio then mixes with a primary buffer and sends out to the speakers. It's a console app—no GUI code to clutter things up. Also, it doesn't do anything fancy like sawtooth waves or square waves, though if you're into the math behind those, I encourage you to add them. See Figure 2.4 if you're not sure what a sine wave looks like, or what the terms amplitude, period, and frequency refer to.

Click To expand
Figure 2.4: A sine wave.

A few years ago, to a black-hat hacker, this program would have been incredibly useful, but I'm not going to tell you why or how. Also, lest they sic the lawyers on me, I should probably mention that I don't condone wreaking havoc on our nation's phone system, or trying to rip off the phone company even if they are the most incompetently run organization you've ever seen. Please use this humble program for good, not evil.

I'll bet you're all giddy with excitement now. A program with that sort of disclaimer on it has got to be cool, right? Here's how it works!

Initializing DirectSound

This is the first and most important step. Initialization consists of three main steps: getting the interface, setting the cooperative level, and setting the format of the primary buffer.

Tip

When you call SetCooperativeLevel, one of the required parameters is a handle to your application's main window. Because the tone generator runs as a console window, it uses some Windows black magic to obtain its HWND. Check out the GetConsoleWindowHandle function in the sample program  Ch2p1_ToneGenerator.cpp. This function queries Windows for its title (window caption), then finds the first window handle that has that caption. This is a somewhat roundabout way to get a handle, and only works if there's only one instance of the application running. However, for the purposes of the sample program, it works like a charm.

Those first two steps are easy; you can skip over the whole mess of enumerating audio cards and selecting one explicitly by passing NULL as the first parameter to DirectSoundCreate8 (or, skip ahead to Chapter 4 to learn the gory details of device enumeration). Setting the cooperative level is similarly easy—for most games, you'll want to use the DSSCL_PRIORITY, which lets you set the format of the primary buffer. If you're writing a "regular" audio application, like Winamp or ACiD, you might want to use DSSCL_NORMAL, which doesn't allow you to change the format of the primary buffer, but does facilitate faster switching between applications, and in general is "nicer" to the system than DSSCL_PRIORITY.

So that leaves the task of setting the format of the primary buffer. This is a little involved because you have to grab the primary buffer's interface, as shown in the following code:

HRESULT       hr;
LPDIRECTSOUNDBUFFER pDSBPrimary = NULL;
// Get the primary buffer
DSBUFFERDESC dsbd;
ZeroMemory( &dsbd, sizeof(DSBUFFERDESC) );
dsbd.dwSize    = sizeof(DSBUFFERDESC);
dsbd.dwFlags    = DSBCAPS_PRIMARYBUFFER;

// "create it" (or get it if it already exists, which it does)
if(FAILED(hr = g_pDS->CreateSoundBuffer(&dsbd, &pDSBPrimary, NULL))) {
 cerr << "CreateSoundBuffer failed:" << DXGetErrorString8(hr) << endl;
 return(false);
}
// set its format
WAVEFORMATEX wfx; ZeroMemory( &wfx, sizeof(WAVEFORMATEX) );
wfx.wFormatTag   = WAVE_FORMAT_PCM;
wfx.nChannels    = (WORD)2;
wfx.nSamplesPerSec = 22050;
wfx.wBitsPerSample = (WORD)16;
wfx.nBlockAlign   = (WORD)(wfx.wBitsPerSample/8*wfx.nChannels);
wfx.nAvgBytesPerSec = wfx.nSamplesPerSec * wfx.nBlockAlign;
if(FAILED(hr = pDSBPrimary->SetFormat(&wfx))) {
 cerr << "pSDBPrimary->SetFormat() failed: "
    << DXGetErrorString8(hr) << endl;
    return(false);
}
// we're done with it, so release!
SAFE_RELEASE( pDSBPrimary );

Here you can see the code performing three steps. First, it gets the primary buffer. The method it uses to do this is called CreateSoundBuffer. CreateSoundBuffer takes a structure, and depending on the flags in that structure, either creates a secondary buffer and returns its interface, or gives back the primary buffer's interface. In this case we get the primary buffer's interface and store it in pDSBPrimary.

The next step is to fill in the WAVEFORMATEX structure with the audio format properties you want. In this case, those are 2 channels (stereo), 22,050Hz (half-CD quality!), with 16 bits per sample. That means that for each second of audio, we have 22,050 samples, and each sample is 16 bits (2 bytes), so we end up burning 44,100 bytes per second of audio. 22,050Hz is an "average quality" rate; for comparison, music that's on CD has been sampled at a rate of 44,100Hz. Consult Table 2.2 for some of the most common sample rates.

Table 2.2: Common Audio Sampling Rates.
Windows Name | Sample Rate | Bits | Channels
N/A | 8,000Hz | 8 | 1 (Mono)
Telephone Quality | 11,025Hz | 8 | 1 (Mono)
Radio Quality | 22,050Hz | 8 | 1 (Mono)
CD Quality | 44,010Hz | 16 | 2 (Stereo)

Once wfx is filled in correctly, the code makes a call to the SetFormat method of the primary buffer's interface. This turns the primary buffer into what's specified in wfx.

Finally, notice that the code releases the primary buffer's interface by calling the SAFE_RELEASE macro. Note that this doesn't actually delete the primary buffer; it just frees up the interface we created by the call to CreateSoundBuffer. A DirectX object (and actually, any COM object) is only deleted when all of its interfaces have been freed, and because DirectX Audio keeps an internal interface to the primary buffer, this doesn't happen until the code un-initializes DirectSound itself.

If this code didn't release the interface, however, the primary buffer would theoretically never get deleted, and a leak would develop. In reality, DirectX Audio is smart enough to detect when this happens and recover from it, but regardless, it's a bad thing to do; always remember to release your primary buffer's interface after you're finished with it.

Creating a Secondary Buffer

With DirectSound initialized, the next step is to create a secondary buffer. Recall that a secondary buffer is where actual audio data is kept; DirectSound uses the primary buffer to mix all of the secondary buffers together on their way out to the sound card.

The good news is that because the function call is the same, you already know 90 percent of what there is to know about creating a secondary buffer. To create a secondary buffer, call the CreateSoundBuffer method of IDirectSound8, as shown by the following code:

int CreateSecondaryBuffer(LPDIRECTSOUNDBUFFER *ppBuffer, int channels,
             int secs, int samplerate,
             int bitspersample, DWORD flags)
{
 HRESULT hr;
 stringstream err;

 DSBUFFERDESC dsbd;
 WAVEFORMATEX wfx;

 ZeroMemory( &wfx, sizeof(WAVEFORMATEX) );
 wfx.wFormatTag   = WAVE_FORMAT_PCM;
 wfx.nChannels    = channels;
 wfx.nSamplesPerSec = samplerate;
 wfx.wBitsPerSample = bitspersample;
 wfx.nBlockAlign   = (WORD) (wfx.wBitsPerSample / 8 *
                     wfx.nChannels);
 wfx.nAvgBytesPerSec = wfx.nSamplesPerSec * wfx.nBlockAlign;
 ZeroMemory( &dsbd, sizeof(DSBUFFERDESC) );
 dsbd.dwSize     = sizeof(DSBUFFERDESC);
 dsbd.dwFlags     = flags;
 dsbd.dwBufferBytes  = samplerate * bitspersample / 8 *
                       channels * secs;
 dsbd.guid3DAlgorithm = GUID_NULL;
 dsbd.lpwfxFormat   = &wfx;
 if(FAILED(hr = g_pDS->CreateSoundBuffer(&dsbd, ppBuffer, NULL)))
{
  err << "CreateSecondaryBuffer: CreateSoundBuffer failed: " <<
   DXGetErrorString8(hr) << endl;
  throw(err.str());;
}
 return(dsbd.dwBufferBytes); // return size of buffer
}

Most of this code is concerned with filling in a DSBUFFERDESC structure, which specifically tells DirectSound what you want it to create. The majority of the members of the DSBUFFERDESC structure are self-explanatory, but if something's confusing you, type DSBUFFERDESC into the DirectX SDK Help index and you'll be taken to a page that explains each member in detail.

The code is creating a buffer using the WAVE_FORMAT_PCM flag. This flag tells DirectSound that the format of the data in the buffer is going to be in pulse code modulation (PCM). When you think of a "normal" sound effect, with no compression or funny codecs, you're thinking of PCM.

Creating a PCM buffer involves a little bit of math. For instance, look at the block alignment, nBlockAlign, of the wave format structure. That probably looks a little strange, but it is correct, because the documentation says that for a PCM formatted buffer, nBlockAlign must be equal to the product of nChannels and wBitsPerSample, divided by 8. This is where you shrug and say, "Well, that was easy." There's a bit of code in DirectSound, or in the sound card driver, that wants this calculation, so your best bet is to just give it what it wants and don't worry about why it wants it.

Tip

Notice the error handling built into the buffer-creation code. You should always use the FAILED macro to check the return value of DirectSound interface methods. If something goes wrong, this macro will return true, allowing you to take the appropriate action. Generally, you'll want to use the DXGetErrorString8 function to turn the error code you got back into a descriptive error message.

Moving on, check out the calculation for the dwBufferBytes member of the buffer description structure. Normally, if you're dealing with a WAV file you're loading from disk, you know how many bytes you need, but this sample program generates the sound data itself, so it has to figure it out. The variables samplerate, bitspersample, channels, and secs are all arguments to the CreateSecondaryBuffer function. The function multiplies all of them together to determine the total number of bytes the sample needs, then sticks that number in dwBufferBytes. For each second of audio, there are samplerate samples, so that's secs × samplerate. The number of channels of the sample is stored in channels, and for each of those channels there are bitspersample bits, so that's channels × bitspersample / 8 bytes for each sample.

Put the whole thing together and you end up with a long string of multiplications that explains why MP3 compression is so useful; without it, a stereo (two channel), CD-quality (16-bit samples, 44,010Hz samplerate), three-minute song (180 seconds) takes 44,010 × 16 / 8 × 2 × 180, or 31,687,200 bytes of storage. That's about 31 megabytes.

Tip

Be careful when doing math involving bits per sample. A common mistake is to forget to divide by eight to convert the bits per sample into bytes per sample. If you forget to divide by eight when calculating the size of your buffer, you risk wasting memory by making a buffer that's eight times as big as you need it to be. If you forget to divide by eight in other areas, worse things could happen.

Protect yourself from yourself by giving your variables long, descriptive names. A variable called bitspersample tells you that you need to divide by eight. A variable called bps is much more insidious, because you don't know if the "b" means bits or bytes.

Returning to the code, once the DSBUFFERDESC structure is filled in, the code calls the CreateSoundBuffer method of IDirectSound8, which creates the buffer and puts an interface for it into ppBuffer. Make note of the double indirection going on here; pBuffer is a pointer to a pointer to a IDirectSoundBuffer (remember, LPDIRECTSOUNDBUFFER is really a IDirectSoundBuffer *, so a LPDIRECTSOUNDBUFFER * is really a IDirectSoundBuffer **. The LP comes from Hungarian Notation, and stands for Long Pointer, so LPDIRECTSOUNDBUFFER is just a long pointer to a direct sound buffer).

Making Noise

Once you have created your secondary buffer, you need to fill it with data. To fill it, you must first lock the buffer (by calling the Lock method of IDirectSoundBuffer). Lock tells DirectSound that you're about to modify the buffer's contents. When you call Lock, you get back a pointer to the buffer itself. If your bitspersample is eight, you should cast the pointer DirectSound gives you to an unsigned char *. You can then go wild with filling in the bytes. Once you're done, you must unlock the buffer (via a call to Unlock).

Here's a peek at that process in code:

void FillBuffer(LPDIRECTSOUNDBUFFER pBuffer,
        float frequency, int buffersize, int samplerate)
{
 HRESULT hr;
 stringstream err;
 unsigned char *pBufferBytes;
 DWORD lockedsize;
 // Lock the buffer down
 if(FAILED(hr = pBuffer->Lock(0, buffersize, (void **)(&pBufferBytes),
      &lockedsize, NULL, NULL, 0L))) {
  err << "FillBuffer: Lock failed: " << DXGetErrorString8(hr) <<
    endl;
  throw(err.str());;
}
for (DWORD q=0; q < lockedsize; q++) {
 // Determine cycle we're in
 float pos = frequency/(float)samplerate*(float)q;
 // take remainder and convert to radians
 float r = (pos - floor(pos)) * 2 * PI;
 float value = sin(r);
 // we now have a value somewhere between -1 and 1… multiply
 // by 127, then add 127, to get unsigned value.
 // change multiplier to change amplitude of wave, aka volume
 pBufferBytes[q] = 127+(value * 127);
 }
 // Unlock the buffer
 pBuffer->Unlock( pBufferBytes, lockedsize, NULL, 0 );
}

This code begins by locking the buffer, via a call to the Lock method of IDirectSoundBuffer. It passes in the first byte it wants to modify (zero), and the total number of bytes it wants to modify (which is buffersize because it's manipulating the entire buffer).

It may surprise you to learn that when you Lock something, you get back not one, but two pointers. This is to help you with streaming. Check out Figure 2.5.

Click To expand
Figure 2.5: When you lock something, you might get two pointers back.

DirectSound allows you to specify a starting byte and a length that would normally exceed the length of your buffer. For example, it's perfectly legal to allocate a 100-byte buffer, and then request 50 bytes starting at byte 75. You'd think that this would be a major no-no because at byte 75 there are only 25 bytes left to give, but instead, DirectSound "wraps" the buffer. If you request 50 bytes starting at byte 75 of a 100-byte buffer, you get two pointers back from DirectSound. The first pointer starts at byte 75 and goes to byte 99. The second pointer starts at byte 0 and goes to byte 39.

Pretty cool, isn't it? By automatically wrapping the buffer for you, DirectSound frees you from having to make two lock/fill/unlock cycles.

If you know that what you're locking isn't going to wrap around, you can safely pass in NULL for the second pointer, and just use the first pointer DirectSound gives you. This is what the tone generator sample program does. However, when you're streaming data, you'll want to give the Lock method two buffers and two sizes, and pay attention to what it gives back to you, so that you can handle the wrapping situation elegantly.

Tip

The Lock method accepts a couple of useful flags. DSBLOCK_ENTIREBUFFER tells it to lock the entire buffer. I didn't use it in the example program because I wanted to explicitly show you how to lock bytes, but in a normal app, you'll probably want to use it instead of specifying zero for the first byte and the size of the buffer for the length.

There's also a DSBLOCK_FROMWRITECURSOR flag that tells DirectSound that you want to write starting at the place you last stopped writing. This is sort of like writing to a file, and is very useful when streaming.

Once the buffer's locked, the real fun begins. To get a sound to play, this sample program fills up the buffer with data generated via the sine function. If it's been a while since you've done any trigonometry, the sine function returns a value between –1 and 1, in a wave pattern.

The tricky part here is getting the frequency of the wave correct. There's a small algorithm to do this. This algorithm first determines the wave cycle it's in—that is, how many full waves have come before it. When pos is a whole number, it's the beginning of a new wave iteration. Take a peek at Figure 2.6 to help visualize this.

Click To expand
Figure 2.6: Use the fractional part of pos to determine where in the wave you are.

In the code, the expression pos – floor(pos) gives you the "remainder" of pos, which is how far into the current wave you are (and will always be something between zero and one). For example, if pos is 2.54 (meaning you've already completed two waves and are just over halfway done with the third), then floor(pos) is 2, and r is 0.54.

Note that regardless of how long or short the wave actually is, you always wind up with a remainder between zero and one. It has nothing to do with frequency, because you've already taken frequency into account when you determined pos, the cycle. That's the beauty of the algorithm.

From here, it's simple—plug the r value into the sine function to get back the wave height at that point. You now have a wave height between –1 and 1. The next step is to multiply this wave height by 127, making it as tall as it can be. Recall that the height of a wave is its volume, so by multiplying by 127, you're giving this tone the maximum volume. If you wanted to make it half as loud, you'd multiply by 64.

Now you have a wave height that's somewhere between –127 and 127 (or –64 and 64, if you halved the volume). The final step is to offset it so that the wave fits into our unsigned char's range of 0 to 255. To do this, you simply add 127.

Tip

The tone generator could have used cosine instead of sine when calculating the buffer. I prefer sine because it starts at zero.

Also, if you end up making a "real" tone generator instead of just a quick example program, always make your buffer size a multiple of the wave frequency you're generating. That way, you're guaranteed that the last byte of the buffer fits seamlessly with the first. In this sample, depending on the frequency chosen, the buffer might not loop seamlessly, so you might hear a very small click or pop as the buffer loops.

Whew! So the code does that for every byte of the secondary buffer. Once it has made its way through the for loop, it unlocks the buffer by calling the Unlock method, supplying the pointers and sizes it received from Lock. No flags available here—Unlock simply unlocks what you've previously locked.

Tip

The ToneGenerator sample program doesn't support task switching, but if you want your game to, remember that you should restore your buffers when your user task switches back into your game. This is accomplished via the Restore method of the IDirectSoundBuffer interface. For an example of how to restore "nicely," consult the DirectX SDK, DirectSound sample programs.

Playing the Buffer

After plowing though all the complexities in creating and filling the buffer, you'll be happy to learn that it's very easy to get a secondary buffer to start mixing with the primary buffer and playing out the speakers. All you have to do is call the Play method of the buffer:

void PlayBuffer(LPDIRECTSOUNDBUFFER pBuffer)
{
 pBuffer->Play(0, 0, DSBPLAY_LOOPING);
}

The first parameter to Play is reserved for use in future versions of DirectSound, and must be set to zero. The second parameter is a priority value, but requires a special type of buffer (created with the DSBCAPS_LOCDEFER flag). Because you didn't create the secondary buffer with this flag, you don't get to specify a priority (which is okay)—just enter zero. The third parameter is the omnipresent flags parameter. Again, if you had a DSBCAPS_LOCDEFER buffer, there'd be all sorts of nifty flags to choose from (consult the DirectX SDK help for the full list), but because you're just using a simple buffer, the only flag available is DSBPLAY_LOOPING, which tells DirectSound to loop the buffer infinitely.

To stop a sound from playing, use the Stop method. Stop takes no arguments; all it does is silence a buffer that's currently playing.

Cleaning Up

Once the user has heard the tone and pressed a key to stop it, the code begins cleaning up the things it had previously allocated:

// release secondary buffer
cout << "Releasing sound buffer..." << endl;
SAFE_RELEASE(soundbuf);
// un-init DirectSound
cout << "Un-Initializing Audio..." << endl;
SAFE_RELEASE(g_pDS);

There are only two things the sample program has allocated: the secondary buffer and the DirectSound interface itself (which also contains the primary buffer).

In case you've never seen it before, here's what SAFE_RELEASE looks like (it's declared in  dxutil.h):

#define SAFE_RELEASE(p)    { if(p) { (p)->Release(); (p)=NULL; } }

SAFE_RELEASE looks at the interface it's given, and if it's not NULL, it releases it (by calling its Release method), and then sets it to NULL (so that if anyone tries to use it again, your program will access violate and you'll be able to see what went wrong). This is standard practice for DirectX, and it behooves you to use SAFE_RELEASE instead of just calling the Release method directly.

Also, make sure you release things in the right order. You should release all secondary buffers before releasing the main DirectSound interface. If you don't, DirectSound will warn you (through debug output) that you forgot to release the secondary buffers because it assumes that any buffer not released by the time it's shutting down has been neglected and forgotten.


CONCLUSION

I hope you've enjoyed the quick tour through DirectSound that the tone generator sample program has given you.

Keep in mind that you will probably never use this tone generator sample program in a real game. As you'll learn in later chapters, DirectX gives you much higher-level controls than this— you never have to create your own tones at the correct frequencies to make music. If you were creating a game and wanted to have cool retro music and sound effects, you could use this code to create a synthesizer engine, but realize that you'd be re-inventing a lot of wheels. In the coming chapters, you'll learn what wheels already exist, and how to use them.

This sample program had two goals. The first goal was to show you the basics of initializing and using DirectSound. The second goal was to show what's happening on a very low level when you play sound. When you're locking buffers and dealing with audio in its byte form, you're very low level. I decided to put this sample program first so that you knew what was really happening under the hood. I've found that if I understand the core of something, I have a much easier time understanding the higher levels of it.

Congratulations on completing your first DirectSound program. In the chapters ahead, you'll learn how to leverage DirectX Audio's higher-level interfaces to help out with other sound and music tasks.


Chapter 3: Wave Audio Playback

Download CD Content

OVERVIEW

With this chapter, you begin writing some actual audio code. The previous two chapters have dealt with foundation topics—things that give you a firm understanding of the basics, but aren't terribly useful in an actual game. Starting with this chapter, however, you'll learn practical techniques you can immediately apply to your own games.

Also, this chapter's sample program begins the creation of a state-of-the-art C++ audio engine. The next several chapters will focus on the development of this engine, and by the end of them, you'll have a fully functional game audio engine you can drag and drop into your own projects (or use as a starting point for your own engine).

But, before you get there, you need to understand a few new concepts, including the intimidating, programmer-devouring pit, DirectMusic.


DIRECTMUSIC FOR SOUND EFFECTS

To most game programmers, DirectMusic is a pretty intimidating chunk of code. I know when I was first learning, I was very confused and often very frustrated. That was a couple of DirectX versions ago, and to their credit, Microsoft has cleaned things up a bit, but it's still complex.

It's complex because it's just so darn big. DirectMusic is one of those all-encompassing libraries. It plays wave files. It streams waves from disk. It plays MIDI. It plays dynamic music. Shoot, it even composes music. There are very few things DirectMusic can't do.

This makes it incredibly powerful but incredibly difficult at times. I have a saying: "big libraries make the difficult stuff easy, but the easy stuff difficult." For example, it's pretty easy to stream a wave file from disk. Just load it, get a segment interface, and tell the performance interface to play that segment. In DirectSound, streaming a wave file is more complicated than that, as you'll see later in this chapter. That's one example from the "difficult made easy" camp.

On the other hand, the thing I remember getting really mad about—the thing that made me want to chuck my monitor out the window—was the volume controls. I had a hard time finding DirectMusic's volume knob, and when I finally did discover it, I realized there isn't just one. There's about half a dozen different ways to control the volume, and if one volume control is turned all the way down, you hear nothing. Sometimes the easy things become difficult.

My advice to you is to take it slow and make sure you understand exactly what's going on with one thing before you move on to another. DirectMusic is a huge API, but it's not a terribly dense API—there are only a few tricky concepts, and there's not a lot of complicated math involved. This is different than, say, DirectGraphics, where if you don't know Linear Algebra, you come to the race with a tire already flat.

I'm going to ease you into DirectMusic over several chapters. In this chapter, you'll dip your toes into the water—you'll learn the core concepts you need to know in order to use DirectMusic for sound effects. In the coming chapters, you'll slowly walk deeper into the pool. In Chapter 5, "Control Freak," you'll learn about the interfaces used for music playback, and in Part Two of the book, you'll learn the interfaces for dynamic music.

For now though, begin by referring to Figure 3.1 as you read the following sections.

Click To expand
Figure 3.1: Overview of the DirectMusic interfoces used for playing sound effects.

The Loader

It all starts by reading the wave file off disk. In DirectMusic, this is done through a special Loader object. You access this Loader object through the IDirectMusicLoader8 interface.

The loader is by far the most under appreciated of all the DirectMusic interfaces. Loading a wave file is a complex task, as you'll see later in this chapter. The loader takes care of all the complexities of loading a wave file: figuring out if and how it's compressed, reading the bytes, and storing them into memory. You simply say, "Loader, please load this wave file, create an object for it, and give me an interface to that object."

The Loader is also very powerful. You can tell it where to look for support files, you can make it read from the resource attached to your EXE, and you can even write custom loading objects to teach it how to load your own sound formats.

Segments

When you tell the loader to load something, it creates a DirectMusic segment and fills that segment with whatever it gets from the drive. You access this DirectMusic segment through the IDirectMusicSegment8 interface.

A segment can contain any segment of audio. This includes wave files, plus any length of music. Each segment can contain multiple tracks (which you can play with through the IDirectMusicTrack8 interface). Think of a track as a single musical staff in a song. Open a piano songbook and you'll see two tracks—the top track is played by the right hand (usually treble clef), and the second track is played by the left. A typical song might include tracks for bass, percussion, chords, and vocal (or melody). When you load a wave file into a segment, you end up with a segment with one track. Usually you don't deal directly with tracks, but it's good to know they're there.

There are many ways to create segments; you don't have to load them. For example, if you're using dynamic music, you have DirectMusic create segments for you. You supply the chord progressions, groove, instruments, and melody, and DirectMusic will create a segment for you. Do this correctly and never bore your player with a song that just repeats over and over again.

Contrary to what you might expect, the IDirectMusicSegment8 interface doesn't have a Play method. You can't just tell segments to play themselves. To hear anything, you need to use another object: the Performance Object.

The Performance Object

The Performance object is the main object that controls all of the other objects. When I want to hear music, I pay money for a ticket, which gives me the right to hear a performance. The performance (hopefully) includes several songs, played on several different instruments, played through a variety of speakers using different effects.

Similarly, once I grab an IDirectMusicPerformance8 interface, I can use it to play segments, control what speakers those segments are played out of, and adjust the tempo, volume, and other settings that influence the segment's playback.

To get a particular segment to play, all one has to do is say, "Mr. Performance object, please play this segment." You can also ask about and set a whole slew of timing controls. For example, you can ask the performance what the current latency time is (latency time is the difference between when you say "play!" and when sound actually starts coming out the speakers).

Other Objects

As mentioned previously, DirectMusic has a whole slew of other objects and interfaces, but the three you just learned about—loader, segment, and performance—are the ones you need to know right now. The other DirectMusic objects are most useful when playing music, so you'll have to wait until Chapter 5 to learn about them.

Now that you've been introduced to the star interfaces of this chapter, you can begin learning how to wrap, tame, and use them to construct the beginnings of an audio engine. But first, you must learn how to handle errors.


AUDIO ENGINE ERROR HANDLING

One of the most important traits of any engine is its robustness, or resistance to crashing under extreme circumstances. Of course, it's always important to make sure your code doesn't crash, but when you're dealing with engines, it's especially important. Even if your engine isn't being sold for a half-million dollars and is only getting used by you, I guarantee that you'll forget the intricacies of it by the time it's finished and you start developing the game that uses it. It's no fun to have to re-learn how an API or engine works halfway into writing the actual game. So, spend the time up front, and put in some solid error handling.

One way to do that is through the use of C++ exception handling. If you haven't heard of exception handling before, you should stop now and learn what it is before you continue. I'm going to assume from this point forward that you understand C++'s try, throw, and catch statements. If these are new to you, use one of the links I've provided on the CD to brush up.

Should You Use Exception Handling?

I know a lot of programmers, and all of them have varying opinions on exception handling. Some of them insist that it's the only way to write fail-safe code. Others insist exception handling is the work of the devil, right up there with gotos and premature optimization. The supporters say that exception handling makes your code more robust, and easier to maintain. The pundits assert that it makes for sloppy, bloated code and lazy programming.

When it comes to exception handling, I fall somewhere in the middle of the two camps, leaning slightly toward the "exception handling rocks!" camp.

Rules to Live by When Using Exceptions

I think that exception handling can make your code easier to read and easier to write, provided you follow a few cardinal rules.

  1. If you write the throw, you write the catch.

  2. Throw when something unexpected happens, not when something bad (but plausible) happens.

  3. Catch errors as soon as possible, but not any sooner.

The rules are explained in the following sections.

Cardinal Rule Number One

If you write the throw, you write the catch.

In other words, whenever you throw an error, you must also decide what catches the error and what it does.

Any thrown error that your program doesn't catch is an "unhandled exception," and brings your program to a screeching halt with an error dialog similar to Figure 3.2. I've seen many good programmers errantly write code that crashes just because a relatively harmless exception "escapes" without being caught by anything.

Click To expand
Figure 3.2: What happens when you fail to catch an exception.

Also, this rule is particularly important when you're dealing with a large code base that doesn't use exceptions.

Cardinal Rule Number Two

Throw when something unexpected happens, not when something bad (but plausible) happens.

The Cheshire Cat version of this rule goes like this: Exception handling handles exceptional situations. It doesn't handle bad things that are considered normal.

The classic example of this rule is when someone is trying to log in to a server. There are two "normal" paths of execution: either the server lets them in, or the server denies them access.

These two code paths should be handled without exceptions (for example, by returning true if access is granted or false if not).

On the other hand, an exceptional situation might be that the server returned garbage: It didn't say yes; it didn't say no—it spoke some gobbledy gook that makes no sense whatsoever. In this case, throwing an exception would be appropriate, because this situation is truly bizarre—your user has problems much bigger than a mistyped password.

Other exceptional situations could involve the client being unable to talk to the server, running out of memory, or being unable to send data through the network.

Why follow this rule? For starters, exception handling is insanely slow. The people who wrote your C++ compiler assumed that you wouldn't be throwing exceptions unless something was very wrong, in which case, hey, who cares if it runs slow, at least it didn't just crash. To be honest, I don't know if they actually thought that, but for whatever reason, throwing an exception is really slow.

Another reason (just in case you needed another!) is that heavy use of exceptions makes code difficult to read.

Okay, the horse died a couple paragraphs ago, so I'll stop now and move on to the next rule.

Cardinal Rule Number Three

Catch errors as soon as possible, but not any sooner.

Obviously, you don't want to catch an error before you can do anything about it. The code that catches the error should be high enough up the chain of command so that it can decide what to do when something goes wrong.

Here's an example to illustrate the rule (just one this time, I promise!). Let's say you're writing some code to read in a wave file and store it in a CSound class. You have several functions, set up as shown in Figure 3.3.

Click To expand
Figure 3.3: Hypothetical call stack for a game, showing where an error might be thrown and caught.

At the top is your C++ program's main() function. The main() function calls LoadGameResources, a big function responsible for loading all of the art, music, and levels for the game. LoadGameResources, in turn, calls LoadSoundFile, which in turn calls LoadWaveFile for loading waves, and LoadMP3File for loading MP3s.

Now, suppose that LoadWaveFile performs an fopen command to open the wave file from disk, and fopen fails (suppose the file that it wants to open isn't actually on the drive). You are using exceptions, so you decide that if fopen fails, you should throw an error.

Cardinal Rule Number One says if you write the throw, you write the catch, so now you have to decide where the catch goes. You have several places you could catch that error:

  • In the LoadWaveFile function

  • In the LoadSoundFile function

  • In the LoadGameResources function

  • All the way up in the main function

Start by thinking about LoadWaveFile. What could this function do in its catch handler? Unfortunately, it couldn't do much. LoadWaveFile doesn't really know why the file's being loaded, so it can't really decide what to do if something goes wrong. As a counterexample, say that LoadWaveFile catches the error and bails out by calling exit(-1). For a game this might be okay, but suppose you wanted to use LoadWaveFile in a sound-editing application. In this context, it wouldn't make much sense for your application to immediately exit and lose all of the user's work just because a file it was asked to load couldn't be found. It would be much better to just pop a "File not found" message and let the user continue using the application. The point is that LoadWaveFile is just a "slave function," knowing very little about the context of the program.

So, moving up the call stack, how about LoadSoundFile? LoadSoundFile is closer, but still not good enough. It suffers from the same lack of context that LoadWaveFile does.

Next up is the LoadGameResources function. This is the first point at which you have enough context to figure out what to do. LoadGameResources knows that if it can't get those bytes off the drive and into RAM, the game is not going to work as designed. In this function, you could write code to pop a message box asking your player to re-install the game, or you could attempt to recover from the situation (say, by seeing if the game CD is in the drive, and if so, reading the rest of the resources off of it).

Finally, what about catching the error in main? You certainly could handle the error here, but I would argue that the main function is too high up the call stack. To some extent it's a judgment call; it certainly wouldn't be the end of the world if you caught the error here, but to me, main handling this is like Picard pushing Geordi out of the engine room and fixing the warp core himself. The main function shouldn't really need to think about the inner workings of resource loading. Now, maybe if LoadGameResources caught an error, tried to recover from it, but still had problems, main could catch its re-thrown error. That's the code equivalent of a chief operating officer telling the CEO of their company, "Look, we had this major problem, and I tried to deal with it, but this is too big for me, and I need your advice in order to go forward."

Knowing the best place to catch an error is one of those skills that you can only learn through experience. There are guidelines, but nothing that works in all situations. Generally, it's much worse to catch an error "too low" (too close to where it was thrown) than "too high," especially for game programming. In game programming, errors are frequently show-stoppers, so catching them high up is reasonable. Even if all you do is catch everything in main, pop a message box saying "this game crashed," and clean up what you can, you're doing better than some of the professionally-made games I've played, which frequently just crash with an access violation and leave my system in a complete mess.

Also at play is the law of diminishing returns—frequently, doing "exactly the right thing" with thrown errors requires lots of code and doesn't have a lot of benefit (unless you're making lifesupport software or shuttle guidance systems). For game programming, trust your gut, and don't worry about getting it exactly right.


THE BEGINNINGS OF AN AUDIO ENGINE

Finally, no more concepts to learn for this chapter! Now you can start learning how to put together the first version of your audio engine. Check out the Ch3p1n_WAVPlayback sample program, which uses our budding audio engine to play a WAV file.

Exceptions in the Audio Engine

I know, you've just come through my exception handling rules and are probably sick of me rambling on and on about exceptions. But, this is where it gets interesting; in this section you'll put all that theory to use and build the error-handling logic of your engine.

All of the code I'm about to explain resides in the [image] ErrorHandling.h and [image] ErrorHandling.cpp source files, if you want to follow along.

The CError Object

It all starts with a CError object. Most of the time, when programmers use exceptions, they don't throw primitive data types like strings or integers. Instead, they throw error classes, which contain not only the error, but some diagnostic information and context about why and when the error occurred. This book's audio engine stores that information in a CError object, outlined here:

class CError
{
public:
  CError(HRESULT hr, std::string err, std::string filename, int line) {
    SetFile(filename);
    SetError(err);
    SetLine(line);
    if (hr) SetReason(DXGetErrorString8(hr));
  }
  virtual ~CError() { }

  std::string GetFile() { return(m_File); }
  void SetFile(std::string f) { m_File = f; }

  std::string GetError() { return(m_Error); }
  void SetError(std::string f) { m_Error = f; }

  int GetLine() { return(m_Line); }
  void SetLine(int l) { m_Line = l; }

  std::string GetReason() { return(m_Reason); }
  void SetReason(std::string f) { m_Reason = f; }
  std::string GetMessageBoxString();

protected:
  std::string m_File;
  int m_Line;
  std::string m_Error;
  std::string m_Reason;
};

As you can see, CError is a pretty simple class. It keeps track of the filename of the source file where the error occurred (m_File), the line number within that source file (m_Line), an error string displayable to the user (m_Error), and an internal diagnostic message (m_Reason). It has accessors for all of its members, a shortcut constructor that will automatically call DXGetErrorString8 if needed, and a GetMessageBoxString member that formats the variables into a user-displayable text message.

Error Handling Defines

I've written some #define magic to make dealing with CErrors a little easier. I based the #define code you're about to see on an excellent article by Steve Rabin called "Squeezing More Out of Assert," published in Game Programming Gems 1 (ISBN 1-58450-049-2), an excellent book I recommend for any serious game programmer (and not just because I wrote a couple of articles for it!).

Inside  ErrorHandling.h are two defines called ThrowIfFailed and Throw (notice the capital T to avoid clashing with the throw keyword). I'll explain Throw first because it's easier:

#define Throw(err) { \
  if (!ThrowCError(NULL, err, __FILE__, __LINE__)) { \
  _asm { int 3 } \
  } \
} \

Throw is just shorthand for a call to the ThrowCError global function. This is how CError gets its m_File and m_Line variables—the #define automatically supplies the current filename and current line number, thanks to the built-in __FILE__ and __LINE__ macros.

The ThrowCError function is responsible for creating the CError object:

bool ThrowCError(HRESULT hr, std::string err,
                 std::string filename, int line)
{
  CError e(hr, err, filename, line);
  string displaystr = e.GetMessageBoxString();
  displaystr += "Press ABORT to end the program, RETRY to debug, IGNORE to throw the error.";
  int result = MessageBox(NULL, displaystr.c_str(),
    "ErrorHandling.cpp", MB_ABORTRETRYIGNORE | MB_ICONSTOP);
  switch(result) {
     case IDABORT: // end the program immediately
      exit(-1); // could also use abort() here
     case IDRETRY: // break into the debugger
      return(false);
     case IDIGNORE: // continue as usual (throw the error)
      throw(e);
  }
  return(true); // just to avoid compiler warnings
}

ThrowCError's job is to present a message box with Abort, Retry, and Ignore buttons (see Figure 3.4). If the user selects Abort, the function immediately exits. If he chooses Ignore, the code throws the error as if the message box was never displayed. If he selects retry, the function returns false. Look back at the Throw define—if ThrowCError returns false, we execute (in assembly) the int 3 instruction. The int 3 instruction fires off the debug interrupt, which, when the program's running under a debugger, causes execution to break immediately.

Click To expand
Figure 3.4: The message box presented by our errorhandling code.

So, why not just put the int 3 directly into ThrowCError? If it were part of ThrowCError, whenever you hit retry, you'd break into the code inside the ThrowCError function, and you'd have to move up the stack one spot to see what actually threw. With the int 3 as part of the define, when you break, you're taken immediately to the source of the error. It's a neat trick, and one for which I must give credit to Steve Rabin.

Now that you understand Throw, here's the more complicated throwiffailed:

#define ThrowIfFailed(result, err) { \
  if(FAILED(result)) { \
    if (!ThrowCError(result, err, __FILE__, __LINE__)) { \
      _asm { int 3 } \
     } \
   } \
  } \

This define does what Throw does, but only if the HRESULT you give it is an error code. If FAILED(result) returns true, the code passes the HRESULT to ThrowCError, which in turn gives it to the CError constructor. The constructor looks up the code and its corresponding error string into m_Reason.

These two defines come in very handy, so you'll see them sprinkled liberally throughout the audio engine. All together, this small error-handling system gives you a lot of robustness. Sure, you could always add more things to it, but I've found that just this simple scheme really helps when something goes wrong.

Building CAudioManager

Exception handling toolbox firmly in hand, you're now ready to start writing CAudioManager, the class that will manage all of your game's audio. The biggest engine begins with a single header file, and this one is no exception. Here's the class declaration for CAudioManager:

class CAudioManager
{
public:
  CAudioManager();
  virtual ~CAudioManager();
  void Init(HWND hwnd, bool stereo = true);
  void UnInit();
  CSoundPtr LoadSound(std::string filename);
  IDirectMusicPerformance8* GetPerformance() { return(m_Performance); }
protected:
  bool m_InitGood;
  IDirectMusicLoader8* m_Loader;
  IDirectMusicPerformance8* m_Performance;
  static const int CAudioManager::m_PerfChannels;
};

Obviously, the finished engine will have more methods than this, but this is a good start. You can see the methods for initializing and un-initializing the engine, as well as loading a sound. There's also a way to get the performance interface directly.

Internally, CAudioManager keeps track of the performance and loader interfaces, as well as an m_InitGood flag that keeps track of whether the manager has been initialized properly. The m_PerfChannels constant represents the number of performance channels, also known simplistically as the number of sounds you can play at once.

Tip

Your CD contains several source code snapshots of the audio engine as it stands for each chapter. That is, the [image] AudioManager.cpp and [image] AudioManager.h files in the Chapter 3 directory only contain the functionality discussed in Chapter 3. The Audio Manager files in the Chapter 4 directory contain this chapter's functionality, plus the Chapter 4 enhancements. This allows you to follow along as I add features to the engine, and gives you several different points along the code history at which you can diverge and start writing things your way.

If you want to take this book's audio engine and run with it as is, use the finished audio engine contained in the AudioEngine folder of your CD. It has all the features.

Init

CAudioManager uses two-phase creation. Two-phase creation is a programming term meaning that to create an object requires not only a constructor call, but also a call to an initialize method. I like to call this method Init, but I've seen other programmers call it Create, Setup, Initialize, or Reset.

Two-phase creation is a smart idea for a couple of reasons. First, it's great for consoles and lowmemory platforms like PDAs. On these devices, you want to minimize the number of times your game allocates memory. So, you can use two-phase creation to initialize every object your game will ever need, once, as your program begins. Once the actual game starts, no calls to new are allowed. Instead, the code simply calls the object's Init method, which restores the object to its pristine state. It's like getting a new object without all the fuss of memory allocation.

This book's audio engine doesn't really care about that, but it does care about the second reason to use two-phase creation: control over when an object gets created. To make the engine as flexible as possible, it allows you to control exactly when it gets initialized. So, the engine uses twophase creation.

The Init method looks like this:

void CAudioManager::Init(HWND hwnd, bool stereo, int perfchannels)
{
  HRESULT hr;
  // initialize COM
  hr = CoInitialize(NULL);
  ThrowIfFailed(hr, "CAudioManager::Init: CoInitialize failed.");
  // Create the loader
  hr = CoCreateInstance(CLSID_DirectMusicLoader, NULL, CLSCTX_INPROC,
    IID_IDirectMusicLoader8, (void**)&m_Loader);
  ThrowIfFailed(hr,
   "CAudioManager::Init: CoCreateInstance for loader failed.");
  // Create performance object
  hr = CoCreateInstance( CLSID_DirectMusicPerformance, NULL, CLSCTX_INPROC,
  IID_IDirectMusicPerformance8, (void**)&m_Performance);
ThrowIfFailed(hr,
  "CAudioManager::Init: CoCreateInstance for performance failed."); hr = m_Performance->InitAudio(NULL, NULL, hwnd, stereo ? DMUS_APATH_DYNAMIC_STEREO
: DMUS_APATH_DYNAMIC_MONO,
  perfchannels, DMUS_AUDIOF_ALL, NULL);
if (hr == DSERR_NODRIVER) {

  // output a warning message, then continue as usual
  MessageBox(hwnd,
     "The program could not locate your audio hardware.",
     MB_ICONSTOP);
   return; // notice we didn't set m_InitGood true
 }
 else ThrowIfFailed(hr,
   "CAudioManager::Init: m_Performance->InitAudio failed.");
 m_InitGood = true;
}

Of the three function arguments, two are optional—only a window handle is required. If you want, though, you can also specify whether the performance's audiopath should be stereo or mono, and the maximum number of channels it should prepare for. Channels are cheap (unless they're being played), so it's okay to pick more than you need. The default is 128.

The code starts by initializing COM, and then grabbing an interface to a DirectMusic loader. Once it's got the loader, it gets a performance object. All of these calls use CoCreateInstance, a COM API function that you can use to create any COM object. Some of the other components hide all this CoCreateInstance stuff inside functions, but with DirectMusic, there are just too many COM interfaces to make that practical, so you have to deal directly with the CoCreateInstance function. Essentially, CoCreateInstance creates an object of a certain class, and an interface used to talk to that new object. From first to last, the arguments specify a class ID (a GUID usually named starting with CLSID_), which identifies the object to make; an aggregate pointer (usually NULL); a context (usually CLSCTX_INPROC, which means the new object shares the same address space as the creator code, like a DLL); an interface ID (a GUID usually named starting with IID_), which identifies the exact interface you wish to use to talk to your newly created object; and finally, the address of a pointer that will hold the new interface. Once the code has created all its objects, it calls the InitAudio method of the IDirectMusicPerformance8 interface it just created. InitAudio initializes the performance and sets up a default audiopath. You can see the code checking the stereo parameter and specifying either DMUS_APATH_DYNAMIC_STEREO or DMUS_APATH_DYNAMIC_MONO. I should mention that there are two other options including DMUS_APATH_DYNAMIC_3D (for creating 3D sounds, which you'll learn in a few chapters) and DUS_APATH_SHARED_STEREOPLUSREVERB, most commonly used for enriching music by using reverb.

Notice that the code checks InitAudio's return value for DSERR_NODRIVER. This is a special error that indicates that DirectX Audio couldn't find any sound hardware. When it sees DSERR_NODRIVER, the code pops a message box informing the player that his audio hardware couldn't be found.

Assuming nothing fails and, therefore, no errors are thrown, the function ends by setting the m_InitGood flag to true, so that future functions know whether the manager is in good shape to play audio.

UnInit

The UnInit function of CAudioManager is much simpler:

void CAudioManager::UnInit()
{
   if(m_Performance != NULL) {
    m_Performance->Stop( NULL, NULL, 0, 0 );
    m_Performance->CloseDown();
    SAFE_RELEASE( m_Performance );
   }
   SAFE_RELEASE( m_Loader );
   m_InitGood = false;
}

The code begins by calling Stop and CloseDown on m_Performance, assuming it's non-NULL. It then releases the m_Performance and m_Loader objects and clears the m_InitGood flag.

LoadSound

So far, the most important method of CAudioManager is the one that loads a wave file and creates a sound object from it. Here's what that looks like:

CSoundPtr CAudioManager::LoadSound(std::string filename)
{
   HRESULT hr;
   CSound *snd = new CSound(this);
   // convert filename to wide-string
   WCHAR widefilename[MAX_PATH];
   DXUtil_ConvertGenericStringToWide( widefilename, filename.c_str());
   // tell loader to load this file
   hr = m_Loader->LoadObjectFromFile(
     CLSID_DirectMusicSegment,
     IID_IDirectMusicSegment8,
     widefilename,
     (void**) &snd->m_Segment);
   ThrowIfFailed(hr,
     "CAudioManager::LoadSound: LoadObjectFromFile failed.");
       return(CSoundPtr(snd));
}

This is much easier than it would be in DirectSound. That's because the DirectMusic loader groks wave files, making this code a two-step process. First, it converts the given filename into a wide character string (DirectMusic only operates with wide character strings). Next, it tells the loader to load the wave file by calling LoadObjectFromFile. It gives the loader the class ID (CLSID_DirectMusicSegment) and interface ID (IID_IDirectMusicSegment8) that it expects back, and the loader gladly obliges.

Tip

Think of a wave file as a one note segment of music, played using one instrument. You still get a segment interface, but it's really just a wave file.

Once it has the segment interface, it stores it in CSound's m_Segment variable. (CAudioManager is a friend class to CSound, so it has access to CSound's protected variables.)

You'll learn about the last line of code in the next section. Just put that return line on your brain's back burner for now.

Smart Pointers

On the list of critical engine features to implement—right behind robustness and error handling—is resource management. A good engine must make it easy for its clients to create objects, use them, and dispose of them when no longer needed.

One of several good ways to do this is to use smart pointers. A smart pointer is a class that behaves like a normal pointer, but with one difference: it's smart! It is so smart that it knows when to delete objects that are no longer being used.

Many programmers use smart pointers to help ease the burden of memory management, especially when also using exceptions. If you'd like a refresher course on what a smart pointer is, consult the links on your CD.

The smart pointers for the audio engine all derive from a common base class, which also happens to be a template. This template base class is CRefCountPtr, and is implemented in [image] RefCountPtr.h, which is based closely on code by David Harvey. On the CD, I've included a link to the article that originally accompanied the code; go there if you want to learn how it works.

CSoundPtr is a derivation for CSound pointers. I've provided a constructor and a Release method that output messages to the debug console, so you can see (in real time!) sounds getting created and destroyed.

CSound

In the audio engine, a CSound object represents a sound effect. As you now know, at the core of this object is a DirectMusic segment interface which contains the wave file itself. CSound wraps this interface.

I decided that I didn't want to follow DirectMusic exactly; instead, I wanted a play method for CSound so that I could say, "sound object, play thyself!" The alternative would have been to put the Play method inside CAudioManager, and pass a CSound as an argument. The reason I chose not to do that was mainly personal preference; it just seemed cleaner to me that way.

Playing a Sound

Putting the Play method inside CSound required me to expose the performance interface. That's why CAudioManager has a GetPerformance interface. The CSound object uses that method, then calls PlaySegment, passing its own m_Segment:

bool CSound::Play()
{
   if (NULL == m_Segment) return(false);
   if (NULL == m_Manager) return(false);
   m_Segment->Download(m_Manager->GetPerformance());
   m_Manager->GetPerformance()->PlaySegment(
     m_Segment, 0, 0,
     (IDirectMusicSegmentState **)&m_SegmentState);
   return(true);
}

The heart of this method is the PlaySegment call. To get a segment to play, you have to provide the performance interface with the segment you want to play (m_Segment), any flags you want (none in this case), and the time it should start playing (zero, for "as soon as possible"). The function gives you back a IDirectMusicSegmentState interface, which you can use to query the segment as it's playing.

Downloading Sounds

Notice, in the preceding Play code, the line right before the call to PlaySegment. Before anything can be played, it must be downloaded to the synthesizer. This is very important because if you don't do it, no sound will play. You download a segment by calling its Download method, passing the interface to the performance you will eventually use to play it.

Tip

Sick of having to remember to download your segments? You can tell DirectMusic to automatically download segments when it needs to. You do this by calling the SetGlobalParam method of the IDirectMusicPerformance8 interface and passing GUID_PerfAutoDownload. Check out the DirectX docs for the details.

Determining If a Sound Is Playing

The most interesting thing you can do with the IDirectMusicSegmentState interface you get back from PlaySegment is to determine whether or not your segment is still playing. Simply call the IsPlaying method of your performance interface, and give it the segment state interface you're interested in. IsPlaying will return S_OK if the segment is still playing, or S_FALSE if the segment isn't playing.

Be careful when calling this on a segment you've just started playing. IsPlaying is sometimes too accurate; it simply tells you whether or not sound from that segment is coming out the speakers. If you've just called PlaySegment, IsPlaying may return false because of latency—you've told the sound to play but it hasn't made it to the speakers yet.

Unloading Sounds

Once you're done with a sound, you need to unload it by calling the Unload method of the segment interface. This unloads the segment's data from the performance.


ALTERNATIVE WAYS TO PLAY SOUNDS

Up until now, you've focused mainly on how to use the DirectX APIs to play sounds. However, this isn't the only way, and sometimes it isn't the best way. Included in this section are a couple of alternative ways to play sounds.

DirectSound

One alternative to using DirectMusic is to use the low-level DirectSound interfaces. Most of the time, if you've already committed to using DirectX, it doesn't make any sense to not use the higher-level DirectMusic Loader interface. My advice is to use DirectSound only if you don't want to use DirectX 8 (for example, you want your game to run under Windows NT 4.0).

Here's how to load and play a sound using nothing but the DirectSound interfaces.

  1. Initialize DirectSound. How exactly you do this depends on which version of DirectSound you're using, but in general the process is similar to what you learned last chapter.

  2. Create a secondary buffer and load your wave file into it. This is the tricky part because it involves parsing the wave file format manually. Consider using code provided either by the DirectX sample programs or the Internet.

  3. Play your sound by calling the Play method of the secondary buffer you created.

  4. Delete the secondary buffer when you're done.

Tip

There are several DirectX sample programs that demonstrate how to play waves using the DirectSound interfaces. In the DirectX 8 SDK, there are a couple of files (dsutil.cpp and [image] dsutil.h) that provide a lot of utilities, including code for a CSound class that uses DirectSound buffers.

The Ch3p2_DirectSound sample program illustrates how to play back a sound using just the DirectSound interfaces.

Windows API Waveform Audio

If you don't want to use DirectX at all, there's another option for getting wave files out the speakers: use the Win32 API—specifically, the Waveform Audio functions. The Waveform Audio segment of the SDK is documented inside MSDN, at Platform SDK\Graphics and Multimedia Services\Multimedia Audio\Waveform Audio.

In particular, pay attention to the Win32 API function, PlaySound, which can play a sound from disk, on a resource, or in memory. The Ch3p3_WaveformAudio sample program illustrates how to play a sound from memory using this function.

Tip

If you want to use PlaySound, you must link with the winmm.lib library. Be sure to add "winmm.lib" to the list of libraries on the linker tab in project settings or you'll get link errors.

Windows MCI

If you want a little more than just a PlaySound function, but still don't want to use DirectX, you can play a sound using the Win32 API's Media Control Interface (MCI). The MCI was designed to communicate with a wide variety of multimedia devices, wave audio playback devices, MIDI sequencers, CD audio playback devices, and digital video playback devices.

MCI is a good interface that gives you more control over playing sounds. In particular, you can play a segment of a wave file and specify your in and out points in time. That is, you can say, "play this sound from 2.54 seconds to 3.81 seconds."

You'll learn how to use MCI to play CD data later in the book. There's also more information about it in MSDN, at Platform SDK\Graphics and Multimedia Services\Multimedia Audio\MCI.


CONCLUSION

In this chapter, you learned how to wrangle three DirectMusic interfaces: the loader, the segment, and the performance. You also learned some ground rules for building a good engine, as well as a couple of alternatives to using DirectAudio to play sound effects.

In the next chapter, you'll sink your teeth deeper into DirectMusic and extend the audio engine to support some manipulation of sound.


Chapter 4: Loading WAV Files

Download CD Content

In the last three chapters, you've come a long way toward total dominance over your game's sound effects. This chapter will further that dominance by showing you a few different ways to get wave files off your drive and into DirectMusic.

ANATOMY OF A WAV FILE

Wave files are not as simple as you might think, and although DirectMusic takes care of the complexities for you, you never know when you're going to be outside of Microsoft's umbrella, and doing things yourself.

So, here's a quick look at the WAV file format. For a more detailed analysis, consult one of the links I've provided on your CD, or, for a quick byte-by-byte map of the file, look at Figure 4.1.

Click To expand
Figure 4.1: This figure illustrates the file format of a typical uncompressed WAV.


RIFFs

To understand WAV, one must first understand RIFF. That's because the WAV file format is a subset of Microsoft's RIFF technology. RIFF stands for Resource Interchange File Format, a file format designed specifically to contain multimedia data. Interestingly enough, DirectMusic songs and other musical data are also stored using the RIFF format.

The RIFF file format is a beast of mind-boggling complexity. Fortunately, to load a WAV file, you don't have to understand all of it. In fact, unless you're doing something very advanced, you don't even have to understand most of it. All you need to know for a normal WAV file is where each element of data is stored. That's what this section is going to teach you. If you're morbidly curious, you can explore the higher-level RIFF philosophy by cruising around MSDN or by following some of the links I've provided on your CD.

Chunks

Nope, I didn't make that up. The different parts of a RIFF file are actually called chunks. These chunks are like pieces of fruit at the grocery store. Just like a Granny Smith apple has a 4017 tag on it, a chunk has a four-character "chunk ID" that tells you what the chunk is. This special kind of ID is called a FOURCC (four-character code).

A FOURCC is really just a 32-bit integer (it's typedef'd as a DWORD) that has four 8-bit ASCII values packed into it (see Figure 4.2).

Click To expand
Figure 4.2: The FOURCC character code for RIFF—note the character swapping.

A WAV file is made up of three chunks. First, there's the main chunk, which has a FOURCC of "RIFF". Inside the RIFF chunk is data telling you you're dealing with a wave format. Also inside the RIFF chunk are two other sub-chunks: "fmt" (notice the space in there) and "data". The fmt chunk contains the format of the wave file (8 or 16 bits, stereo or mono, sample rate, and so on). The data chunk contains the actual bytes that form the sound wave.

Every chunk has two things in common. The very first four bytes of a chunk will always contain the FOURCC of that chunk. Because the WAV file itself is a RIFF file, the first four bytes of a valid WAV file will always be RIFF.

Tip

Uppercase FOURCCs are standardized; lowercase FOURCCs usually signify a proprietary format.

Next, in every chunk, immediately following the chunk's ID, are 4 bytes that tell you the size of that chunk. Note that this size does not include the size of the chunk's ID, or the size of the size bytes. In other words, the size contained in here is 8 bytes less than the total size of the chunk because the first four bytes are the chunk's FOURCC, and the next four bytes are the size of the chunk. This is tremendously important to remember; if it helps, think of the size bytes as specifying "size from this point forward."

The following sections will examine the main, format, and data chunks in detail.

The Main Chunk

This chunk contains the other two chunks. Its FOURCC is RIFF.

This chunk contains one piece of data. After the chunk ID and the chunk size, the only thing left is a FOURCC that describes what's actually in this RIFF file. For a WAV file, these four data bytes will always contain "WAVE." For other types of RIFF files, there will be different codes here.

So really, all the main chunk says is, "I'm a RIFF file, and I contain a wave."

Note that the size of this chunk is the size of the next two chunks plus the four bytes used to contain the format data.

The Format Chunk

Now it gets interesting. The format chunk contains everything you need to properly interpret the data chunk. As you can see from Figure 4.1, this chunk tells you the audio format, number of channels, sample rate, byte rate, block alignment, and bits per sample of the data.

Audio Format

First, there's the audio format. This is a two byte piece of data that tells you whether the data chunk is compressed. For uncompressed WAVs, this attribute is 1, which means the data chunk is stored using Pulse Code Modulation (PCM). PCM is a fancy way of saying that the wave data is uncompressed.

Values other than 1 indicate that this wave file was compressed somehow (see Table 4.1).

Table 4.1: Common Wave Format IDs
Wave Format | #define in MMREG.H | Number
Unknown (this is bad) | WAVE_FORMAT_UNKNOWN | 0
Pulse Code Modulation (PCM) | WAVE_FORMAT_PCM | 1
Adaptive Differential PCM | WAVE_FORMAT_ADPCM | 2
32-bit floating point | WAVE_FORMAT_IEEE_FLOAT | 3
CCITT G.711 A-law | WAVE_FORMAT_ALAW | 6
CCITT G.711 u-law | WAVE_FORMAT_MULAW | 7
MPEG Layer 3 (MP3) | WAVE_FORMAT_MPEGLAYER3 | 55

There are many other formats besides the ones listed in the table; check out MMREG.H for the entire list.

Tip

There are a few libraries available online that allow you to read various compressed wave formats. I've included links to a few of them on your CD.

As you can see, there are a ton of formats available, and writing code for all of them would take forever. Fortunately, the vast majority of the WAVE files out there are PCM, and you can always use a program to convert a non-PCM wave into a PCM one. If you do have to (or want to) write your own WAV file parser, my advice would be to only support PCM.

Number of Channels

Immediately following the audio format are two bytes that tell you how many channels the WAV file has. If it's mono, there will be a one here; if it's stereo, there will be a two. There's the potential for other numbers—for example, a WAV could contain six-channel surround-sound data. Mono and stereo are the most common, however.

Sample Rate

This is self-explanatory. These four bytes will contain 8,000 for 8,000Hz, 44,100 for 44.1KHz, and so on.

Byte Rate

This field tells you how many bytes are used per second of audio. This number is always the sample rate multiplied by the number of channels, multiplied by the bits per sample divided by eight (remember, 16 bits per sample really means two bytes per sample).

Block Align

This tells you the number of bytes for one sample. It is always the number of channels multiplied by the bits per sample divided by eight.

Bits per Sample

An 8 here means that you're dealing with an 8-bit sample, a 16 means 16 bits, and so on. 8 and 16 bits are the most common values.

The Data Chunk

Once you've read all of the format data, you know exactly how to interpret the data in this section. That's good, because in the data chunk, after the FOURCC and size fields, is an ocean of raw bytes. How these bytes are interpreted depends on the format block.

Tip

Again, it's important to remember that you usually don't have to deal with the wave file format directly; most of the time you'll use the DirectMusic Loader.

For uncompressed WAV files, the size of this data block is always going to be the number of samples multiplied by the number of channels, multiplied by the bits per sample divided by eight.

Putting All This to Use

Now that you know what's inside a WAV file, it seems only natural to enhance our beloved audio engine with a few new methods.

CWAVFile

These methods allow direct manipulation of wave file data through a class called CWAVFile:

class CWAVFile
{
public:
 friend class CAudioManager;
 void Init();
 CWAVFile() { Init(); }
 CWAVFile(const CWAVFile &r) { Init(); Copy(r); }
 virtual ~CWAVFile();
 int m_AudioFormat;
 int m_NumberOfChannels;
 int m_SampleRate;
 int m_ByteRate;
 int m_BlockAlign;
 int m_BitsPerSample;
 unsigned int m_DataLen;

 CWAVFile &operator=(const CWAVFile &r) { Copy(r); return(*this); }
 unsigned char *GetData() { return(m_Data); }
 void SetData(const unsigned char *data, unsigned int len);
 void Load(const unsigned char *data);
 void Load(std::string filename);
 unsigned char *Save() const;
 bool Save(std::string filename) const;
 int GetTotalSize() const { return(44 + m_DataLen); }
 std::string GetInfo();
protected:
 void Copy(const CWAVFile &r); // snip
 // data chunk
 unsigned char *m_Data;
 };

This class keeps track of a WAV file's format and data chunks. It has exposed variables for all of the pieces of information found in the format block, as well as a protected m_Data pointer that stores the data block. I didn't build accessors onto this class because none of these data types are going to change, but if you feel more comfortable using Get/Set methods, go for it.

Tip

CWAVFile has an overloaded assignment operator so that it can properly copy its m_Data pointer. Failure to override operator= for a class that dynamically allocates memory is a major C++ no-no; be careful. And remember your virtual destructors, too.

Loading into CWAVFile

Loading a WAV file into CWAVFile is fairly straightforward once you understand the file format. Notice in the CWAVFile class that there are two overloads for loading—one for a file and one for in-memory data. The file overload just reads the file into memory and calls the in-memory data overload. It does all the real work:

void CWAVFile::Load(const unsigned char *data)
{
 FOURCC riff, wave, fmt, datacc;

 // check for RIFF ChunkID
 memcpy(&riff, &data[0], sizeof(FOURCC));
 if (riff != mmioFOURCC('R', 'I', 'F', 'F')) {
     Throw("invalid WAV file data");
 }
 // check for WAVE format
 memcpy(&wave, &data[8], sizeof(FOURCC));
 if (wave != mmioFOURCC('W', 'A', 'V', 'E')) {
    Throw("Invalid WAV file data (not WAVE format).");
 }
 // check for FMT ChunkID
 memcpy(&fmt, &data[12], sizeof(FOURCC));
 if (fmt != mmioFOURCC('f', 'm', 't', ' ')) {
    Throw("invalid WAV file data (FMT subchunk not found).");
  }
 // check for DATA ChunkID
 memcpy(&datacc, &data[36], sizeof(FOURCC));
 if (datacc != mmioFOURCC('d', 'a', 't', 'a')) {
    Throw("invalid WAV file data (DATA subchunk not found).");
 }
 // load in data members
 memcpy(&m_AudioFormat,      &data[20], sizeof(short int));
 memcpy(&m_BitsPerSample,    &data[34], sizeof(short int));
 memcpy(&m_BlockAlign,       &data[32], sizeof(short int));
 memcpy(&m_ByteRate,         &data[28], sizeof(int));
 memcpy(&m_NumberOfChannels, &data[22], sizeof(short int));
 memcpy(&m_SampleRate,       &data[24], sizeof(short int));
 unsigned int datasize=0;
 memcpy(&datasize, &data[40], sizeof(int));
 SetData(&data[44], datasize);
}

This code is pretty pessimistic; before it does any real work, it does everything it can to make sure that data actually points to a valid WAV file. If all the FOURCCs check out, it proceeds to pick the relative data bits out of the memory chunk, using memcpy. Finally, it grabs the wave data itself, via a call to its SetData method. Note that this class doesn't assume ownership of the data pointer; it makes a copy of it, which means the client should free this memory as soon as possible.

The mmioFOURCC function is a handy Win32 function, defined in mmsystem.h, that takes four characters and turns them into a FOURCC. A FOURCC is really a 32-bit value, so there's no compression magic going on, just some clever byte rearranging and casting.

Saving from CWAVFile

The CWAVFile Save methods are symmetrical to the Load ones. There are two Saves, one for saving to disk, and one for saving to memory. The disk overload just calls the memory overload and then saves out that memory.

I'm not going to include the Save method here, but suffice it to say that it allocates enough memory, re-creates the FOURCCs, puts the CWAVFile members into the proper offsets, and tacks on the wave data. Once that's all done, it returns a pointer to the memory it's created, again with the understanding that the caller owns the memory once Save returns.

Tip

For extra credit, try integrating into the audio engine a library that reads compressed wave file formats. It would be really cool if you could tell the audio engine to read a wave file from disk and have it automatically decompress that wave file as it put it into a CWaveFile object.

The Ch4p1_WAVFileXRay sample program shows how to read a WAV file into memory.


OTHER WAYS TO LOAD WAV FILES

Now that you have a basic understanding of the WAV file format, it's time to switch gears back into DirectMusic mode and learn some different ways to load wave files. In the last chapter, you learned how to load wave files from disk; now you'll learn how to load them from memory or from your application's resources.

Loading from Memory

It's possible to tell the DirectMusic Loader object to read a string of bytes from memory instead of from disk. This is a very powerful feature because it allows you to store your wave files however you want, as long as you can read the entire file into memory. You could store your wave files in a ZIP file, or in a custom file format, or you could even download them from the Internet.

However you do it, if you get a wave file into memory, you can rely on the DirectMusic Loader to parse it and give you back a segment.

In Chapter 3, "Wave Audio Playback," you used the LoadObjectFromFile method of IDirectMusicLoader8 to load a wave file from disk. It turns out that LoadObjectFromFile is just a shortcut you can use for loading things from disk; the real power lies in the loader's GetObject method, which can read from disk or memory.

To use GetObject, you must first fill in a DMUS_OBJECTDESC structure, which tells DirectMusic where your object is and how you want to load it. To load an object from memory, you need to fill in two important members in this structure. First, fill in the pbMemData byte pointer with the location of your "file" in memory. Next, fill in the llMemLength member, which tells DirectMusic the length of your memory chunk.

Using this knowledge, you can enhance the audio manager class by creating another LoadSound overload:

CSoundPtr CAudioManager::LoadSound(unsigned char *data, int datalen)
 {
  HRESULT hr;
  CSound *snd = new CSound(this);
  DMUS_OBJECTDESC desc;
  memset(&desc, 0, sizeof(DMUS_OBJECTDESC));
  desc.dwSize = sizeof(DMUS_OBJECTDESC);
  desc.dwValidData = DMUS_OBJ_MEMORY | DMUS_OBJ_CLASS;
  desc.guidClass = CLSID_DirectMusicSegment;
  desc.llMemLength = datalen;
  desc.pbMemData = data;
  hr = m_Loader->GetObject(&desc, IID_IDirectMusicSegment8,
      (void **)&snd->m_Segment);
  ThrowIfFailed(hr, "CAudioManager::LoadSound: GetObject failed.");
  snd->m_OriginalData = data;
  return(CSoundPtr(snd));
}

This code starts off similar to the original LoadSound method—it creates a new CSound object. From there, the code paths diverge; whereas the original LoadSound called LoadObjectFromFile, this code sets up the DMUS_OBJECTDESC structure and calls GetObject.

Not every member of the DMUS_OBJECTDESC structure needs to be filled. The dwSize variable is mandatory, as is the dwValidData flags. Whether anything else needs to be filled in depends on the dwValidData flags. By specifying DMUS_OBJ_MEMORY, you'll tell DirectX Audio that you want it to load an object from memory. Specifying this flag obligates you to fill in the llMemLength and pbMemData structures with the size of the data, and the data itself, respectively.

The other flag, DMUS_OBJ_CLASS, tells DirectX Audio that you want to load the type of object specified in the guidClass member. The code sets guidClass to CLSID_DirectMusicSegment, telling DirectX Audio that it should give us back a segment interface.

After the call to GetObject, the code then sets the m_OriginalData member of the new sound object. This is the manager's way of telling the sound, "Hey, your segment object is using memory that the program has allocated, so you need to free this memory when you are destroyed." You can't free the memory immediately after the call to GetObject because DirectX Audio uses that exact memory inside the segment. It doesn't make a copy of the memory; instead, it relies on that data staying where it's at for the life of the segment.

However, when the segment does die, it's still your responsibility to free the memory.

Ch4p1_WAVFileXRay demonstrates the use of this method.

Loading from CWAVFile

Hmm… time for something fun. The audio engine currently has CWAVFile, and it currently has CSound. It'd be sweet if you could convert a CWAVFile into a CSound. If you could do that, your game could create sound effects at runtime, using CWAVFile, and then pump those sound effects into CSound for playback inside the game.

This glorious integration is really easy. Just make CWAVFile save itself out to memory, and then give that memory to the DirectMusic Loader, which will create a segment from it. Here's the code:

CSoundPtr CAudioManager::LoadSound(const CWAVFile &wavfile)
{
  unsigned char *savedata = wavfile.Save();
  return(LoadSound(savedata, wavfile.GetTotalSize()));
}

You've gotta love that! It's two lines of code! Just remember not to free the memory that Save allocated because this memory is in use by the CSound. Actually, because the raw memory LoadSound overload sets the new CSound's m_OriginalData member, there's nothing to worry about—when the CSound is destroyed, it will free the memory.

Loading from Resources

Once you know how to load a wave file from memory, loading from a resource is a snap. Just call a few Win32 API functions to get a pointer to your custom resource (for example, wave file), then give that pointer to the Loader via the DMUS_OBJECTDESC structure described before. Here's a code snippet:

CSoundPtr CAudioManager::LoadSound(HMODULE hmod,
                                   char *type, WORD resID)
{
  CSound *snd = new CSound(this);
  HRESULT hr;
  DMUS_OBJECTDESC ObjDesc;

  HRSRC hFound = FindResource(hmod, MAKEINTRESOURCE(resID), type);
  if (NULL == hFound) { Throw("couldn't find resource!"); }
  HGLOBAL hRes = LoadResource(hmod, hFound);

  ObjDesc.dwSize = sizeof(DMUS_OBJECTDESC);
  ObjDesc.guidClass = CLSID_DirectMusicSegment;
  ObjDesc.dwValidData = DMUS_OBJ_CLASS | DMUS_OBJ_MEMORY;
  ObjDesc.pbMemData = (BYTE *) LockResource(hRes);
  ObjDesc.llMemLength = SizeofResource(hmod, hFound);

  hr = m_Loader->GetObject(&ObjDesc, IID_IDirectMusicSegment8,
    (void**) &snd->m_Segment);
  ThrowIfFailed(hr, "GetObject failed.");
  return(CSoundPtr(snd));
 }

Again, this code is very similar to the other LoadSound methods. The main difference is that before it fills in the DMUS_OBJECTDESC structure, the code has to find and load the resource. The call to FindResource gives back a HRSRC handle, which is then used in the call to LoadResource. LoadResource gives back a HGLOBAL, which is used by LockResource. LockResource takes a HGLOBAL and gives back a pointer to the resource data in memory. The code puts this pointer into the pbMemData member of the DMUS_OBJECTDESC structure, and it's good to go. Note also the call to SizeofResource to determine how big a resource is.

Ch4p2_WAVInResource demonstrates the use of this method.

But how do you get the wave file into the resource to begin with? Here's the recipe for adding a wave file to your program's resources:

If you don't already have a resource file, you need to create one and add it to your project. In DevStudio, select New from the File menu, and select Resource Script as the type of file. The convention is to name the program's main resource script <projectname>.rc, where project name is what you named your EXE, DSP, and DSW. For example, the resource script for Ch4p2_WAVInResource is [image] Ch4p2_WAVInResource.rc. Close the file immediately after DevStudio opens it up for you.

Once you've created the file, you'll see a ResourceView tab appear on your workspace (see Figure 4.3). Click this tab, then right-click on the topmost folder. Select Insert from the context menu, then click the Import button (see Figure 4.4). Select the WAV file you'd like to add from the file browser.

Figure 4.3: After inserting an RC file into your project, you get a new ResourceView tab.

Figure 4.4: Importing a WAV file into your application's resources.

Next, you need to tell DevStudio what kind of resource this is. You can pick anything you want for the resource type, but you should remember to pass that same type to the LoadSound method of the audio manager. I use "WAVE" to denote wave files.

If all goes well, you should see your custom resource listed underneath a folder with a name identical to the resource type name you specified in the last step.

Compile your program and your resource will get sucked in and included in your EXE.

Make sure you include [image] resource.h inside any CPP files that want to use resources. Resource.h uses #define to bind resource IDs (what the Win32 API wants) to the resource names you've specified in the resource editor.

Tip

By default, DevStudio doesn't store the entire custom resource in the RC file. Rather, it stores a callout, telling it where to go to get the original file. Usually this is fine, but if you would rather have DevStudio store a copy of the WAV you selected directly inside the resource script, right-click on the resource in question, select Properties, and you'll see a dialog like that in Figure 4.5.

Click To expand
Figure 4.5: Uncheck the external file checkbox to embed your WAV directly in the resource script of your program.

Uncheck the External File checkbox and your resource will get stored inside the script itself. Note that resources stored in the RC file take up significantly more space—roughly five times their normal size—due to the fact that they are "ASCII-ized" (for example, a single byte of 30 hex becomes an ASCII string of "0x30" or something).

Specifying a Search Directory

Most games like to store their data in special directories. I've seen many professional games store all of their audio data inside a separate "Sound" or "GameAudio" folder, underneath their main install location. Here's a trick for making this easier.

You can tell the Loader what directory it should look in to load objects. Then, instead of calculating and supplying the full pathname to each sound you load, you can supply the filename and the loader will look for that file in the directory you told it about earlier.

The method you want is called SetSearchDirectory. Here's a new audio engine function that tells the loader where to find the sounds:

void CAudioManager::SetSearchDirectory(std::string dirname)
 {
 // convert dirname to wide-string
 WCHAR widedirname[MAX_PATH];
 DXUtil_ConvertGenericStringToWide( widedirname, dirname.c_str());
 HRESULT hr = m_Loader->SetSearchDirectory(
   CLSID_DirectMusicSegment, widedirname, true);
 ThrowIfFailed(hr, "SetSearchDirectory() failed.");
 }

As you can see, there's not much to it. Just convert the directory name into a wide character string, and then call the SetSearchDirectory method. The first parameter tells DirectX Audio what type of object you want to set the directory for (in this case, segments). The next parameter is the directory name, and the last parameter, if true, tells DirectX Audio to "forget" the previous locations of objects it may have already loaded (usually a good idea).

The Ch4p3_SearchDir sample program demonstrates this new method.


CONCLUSION

In this chapter, you learned several different ways to get audio data into DirectMusic. As you can see, the DirectMusic Loader is a fairly versatile piece of code; it allows you complete flexibility when it comes to loading your sound files. Also, keep in mind that all of the loader techniques you've learned here will eventually apply to music as well as sound effects.


Chapter 5: Control Freak

Download CD Content

OVERVIEW

You're getting close to having the sound effects portion of the audio engine completely finished. However, there are still several important features missing. All of these features revolve around controlling sound effects in one way or another—adjusting their volume, making them play concurrently, and so on.

This chapter will teach you all of the little tricks involved in making sound effects sound good.


PLAYING SOUNDS CONCURRENTLY

Astute readers may have noticed that there's a significant problem with the audio engine as it stands now: it can't play the same sound repeatedly. In other words, if you have a wave of a bell dinging, you can't make three bells ding concurrently. The best you can do is make one bell start dinging, then stop dinging, and then start again, three times. This section will look at how to correct that problem.

Primary versus Secondary DirectMusic Segments

DirectMusic has the notion of primary and secondary segments. The difference is that there can be many secondary segments, but only one primary segment.

The primary segment is considered the "main" musical track for your game. Sound effects or other music jingles are usually secondary segments, played over the primary segment. The primary segment usually dictates the parameters of the music (for example, the tempo, chord pattern, and so on). You'll learn more about this in the coming chapters; for now, just remember that the only way to have multiple sound effects playing is by making sure they play as secondary segments.

Goodbye PlaySegment, Hello PlaySegmentEx

In the last chapter, you used the PlaySegment method of the DirectMusic performance object as the workhorse for getting sounds to play. This works well for simple one-sound playback, but to get the same sound to play many times, you need to use PlaySegmentEx. PlaySegmentEx lets you tell DirectX Audio whether a segment should be played as a primary or secondary segment.

By using PlaySegmentEx (instead of PlaySegment) inside CSound's Play method, you're able to Play multiple sounds concurrently:

m_Manager->GetPerformance()->PlaySegmentEx(m_Segment, NULL, NULL,
    DMUS_SEGF_SECONDARY, 0,
    (IDirectMusicSegmentState **)&m_SegmentState, NULL, NULL);

PlaySegmentEx takes more parameters than PlaySegment, so you should look in the docs to figure out what everything means. The really important thing is the DMUS_SEGF_SECONDARY flag, which tells DirectX Audio that it should Play this segment as a secondary segment instead of a primary.


SOUND INSTANCES

As you might have guessed, you're not quite finished dealing with concurrent sound Playback just yet. In the last section, you learned how to get DirectMusic to Play back more than one sound. This section will teach you how to change the audio engine to accommodate the (new) fact that more than one sound can be Playing.

The big problem with the audio engine right now is that it only keeps track of one segment state (IDirectMusicSegmentState8). Play a sound once and m_SegmentState is filled in with an interface to the new segment state; Play another sound and that exact same thing happens, and you lose forever the first sound's state. What's worse is that the first sound's segment state is never released, creating a resource leak.

It's time to fix all of that. What the audio engine needs is a new class—CSoundInstance. This new class will represent a currently Playing sound. The CSound::Play method will create (and give back) a new CSoundInstance every time that sound Plays. That way, the code that called Play can keep track of that particular sound instance.

This is useful when sound effects loop (which you'll learn about in the next section). Usually, most code that calls Play for a non-looping sound effect won't care about the sound once it's Playing. When a Player fires a bullet, the bullet firing code will typically adjust the sound's parameters (the volume), Play it, and then let it Play. Like Ron Popeil's rotisserie grill, the code sets it, then forgets it.

For looping sounds, the pattern is a bit different. Usually a looping sound will need to be adjusted in response to game events. Imagine a looping "hum" of a power generator in a real-time strategy game. At first the hum is barely audible, but when the Player builds a super weapons factory, the power usage skyrockets, and that hum becomes louder.

Sound instances will become more important as you get farther in the book. Right now, you can't do much with them, but when you start learning about 3D sound, they will become very important. Each sound instance will have a 3D position and several other properties you can adjust.

Follow along in the coming sections by looking at the Ch5p2_SoundInstances sample program.

Tip

The CSound and CSoundInstance relationship is a pattern common in game programming. In 2D games, you have sprites, which run around the game world, and bitmaps, which represent how a sprite looks. This lets you have multiple sprites that all use the same bitmap, and saves you from needlessly duplicating the bitmap data.

In the audio engine of this book, CSound is like a bitmap, and CSoundInstance is like a sprite.

Creating CSoundInstance

CSoundInstance is a simple class, so I'm not going to include its declaration here. Look at it in your development environment and you'll notice that it has Init and UnInit methods, a protected segment state member, a standard issue audio manager pointer (used by IsPlaying), and a flag indicating whether the object is valid.

This class, like CAudioManager, uses two-phase construction. This allows the game to pre-allocate sound instances that will be used later. For example, inside a CTank class, you could have a statically-allocated member called m_EngineSound:

class CTank
{
public:
  CTank() { } // nothing to do
  virtual ~CTank() { } // nothing to do
protected:
  CSoundInstance m_EngineSound; };

If you designed CSoundInstance like CSound (one-phase construction), this wouldn't be possible because CSoundInstance would need constructor parameters specifying the sound it came from.

So, the Init and UnInit methods, as well as the m_Valid flag, are all things you need for two-phase construction. That leaves just a few other things—the IsPlaying method and an m_SegmentState member and associated accessor function, GetSegmentState.

The IsPlaying method looks very similar to its counterpart in CSound:

bool CSoundInstance::IsPlaying()
{
   assert(IsValid()); if (!IsValid()) { return(false); }
   return(GetAudioManager()->GetPerformance()->IsPlaying(
     NULL, m_SegmentState) == S_OK);
}

The performance's IsPlaying method is a two-headed beast. It can check to see if a segment is Playing, or if a segment state is Playing. To differentiate, you put a segment in the first parameter and a segment state in the second. Because CSoundInstance has a segment state, it passes NULL as the first parameter. Nothing to it!

Tip

In the IsPlaying method, notice the IsValid check. One of the drawbacks of using two-phase construction is that in each method, before you try to do anything, you need to first make sure that you've got an object that's fully initialized. That way, if someone is being stupid and calls IsPlaying before the instance has been initialized, the program doesn't crash.

Adjustments to CSound Methods

Now that CSoundInstance is coded, you need to make a few changes to CSound. CSound no longer requires a m_SegmentState member because this is now handled by CSoundInstance. Also, the CSound::IsPlaying() method needs to change to use the segment instead of the segment state. Here's the new code:

bool CSound::IsPlaying()
{
   if (NULL == m_Segment) return(false);
   if (NULL == m_Manager) return(false);
   return(
     m_Manager->GetPerformance()->IsPlaying(m_Segment, NULL) == S_OK
   );
}

Here, I've changed the call to IsPlaying, passing in the segment as the first parameter and NULL as the second. This tells DirectX Audio to return S_OK if any instance of the sound is Playing.

By far however, the biggest change is in the Play method. Here's the new code:

bool CSound::Play(CSoundInstance *newinst)
{
  if (NULL == m_Segment || NULL == m_Manager) {
    if (newinst) newinst->UnInit();
    return(false);
  }

  static CSoundInstance dummy;
  if (NULL == newinst) newinst = &dummy;
  m_Segment->Download(m_Manager->GetPerformance());

  // initialize the sound instance
  // this will also set the sound Playing
  newinst->Init(this);
  return(true);
}

For starters, notice that we have a parameter, newinst, which is a pointer to the instance the client would like to use to keep track of the sound that's about to be Played. The newinst parameter is defaulted to NULL in CSound's class declaration, so code that doesn't care about retaining a sound instance can just call Play() as it has always done.

If newinst is NULL, the code uses a dummy static CSoundInstance to initialize the sound and Play it. Otherwise, it uses the newinst passed in.

Note that the CSoundInstance::Init() method wants a CSound pointer. Right now the Init method doesn't use the CSound pointer for anything, but in later sections it will.

Ch4p4_ConcurrentPlayback illustrates how all this new code works together.


LOOPING SOUNDS

It's often very handy to be able to loop a sound. Before DirectMusic, the traditional way to loop a sound was to start it Playing and then check it every so often to see if it was still Playing. If it wasn't, you started it again.

The biggest problem with that method is that to loop without a delay, you have to check up on the sound very often, on the order of 10 times per second. With one sound that may not seem like a big deal but with several, you can end up devouring a lot of CPU power.

There's an easier way—just call the SetRepeats method of the segment interface. You can pass the DMUS_SEG_REPEAT_INFINITE flag to SetRepeats to tell DirectMusic to loop a segment infinitely. Or, you can pass in an integer specifying the number of times to loop.

If you need finer control over looping, you can use the SetLoopPoints method of the segment. This allows you to specify where inside the segment the looping should begin and end.

However, this doesn't operate as you might expect. Check out Figure 5.1. Even if you use SetLoopPoints to specify a starting and ending position, the entire segment will still Play. DirectMusic will Play from the start of the segment to the end loop point you specified, loop back to the start point you specified, loop for the number of times set by SetRepeats, and finally Play from the end of the loop point you set to the end of the segment. In other words, SetLoopPoints only applies during the loop; the segment still Plays itself into and out of the loop.

Click To expand
Figure 5.1: DirectMusic uses several different parameters to control looping of a segment.

If you want to control the actual start and end points, you need to use the segment interface's SetStartPoint and SetLength methods. By default, the loader sets the start point at the beginning of the segment, and the length to the length of the entire segment.

Tip

Of course, there are also Get methods corresponding to all of the methods that set looping parameters.

Ch5p3_Looping illustrates how to loop a sound.


CONTROLLING VOLUME USING DIRECTX AUDIO

Frequently, games need to be able to change the volume of a sound based on game events. This section will teach you how to do that using DirectX Audio.

Audio Paths

In DirectMusic, volume is tied to an object called an audio path. An audio path contains all of the information that influences how a segment is Played, but doesn't necessarily belong as part of the segment itself. The classic example of this is (you guessed it) volume.

It wouldn't make sense for the volume knob to reside in a segment. If it did, and you wanted the same segment to be Played at different volumes, you'd have to duplicate the segment object—a horrible waste of resources. So instead, DirectMusic puts most of the "per-instance" controls inside an object called an audio path.

Tip

The position of a 3D sound is also stored as part of the audio path object, but you'll learn about that later.

Currently the audio engine has been using the "default audio path" for all the sounds it Plays. The default audio path is the one you created when you called InitAudio. This is fine for simple sound Playback, but if you want to change the volume of a particular sound, you'll run into trouble. All of the sounds use the same audio path, so when one sound changes the volume of that audio path, it indirectly changes the volume of all the other sounds (see Figure 5.2).

Click To expand
Figure 5.2: Sounds that share audio paths also share volume knobs.

There are a couple of different solutions to this, but they both involve using multiple audio paths. Some audio programmers like to set up a "pool" of audio paths; when a sound is Played, their code chooses an empty audio path and assigns the sound to it. This is elegant for a number of reasons—for starters, it gives the programmer the ability to easily see and manipulate all the audio paths, without having to iterate through all of the Playing sounds.

The alternative method, and the method I've implemented in the audio engine, is to create a new audio path each time a sound is Played. Once the sound has finished Playing, the audio path sticks around until it's released. Because audio paths are relatively lightweight classes, this is okay, provided you can deal with a little allocation inside your game logic. I think it's significantly simpler to code and teach than the first method, which is what really clenched the decision for me to use it in this book.

To accommodate these audio paths requires a few changes to the Play method of CSound. Here's the new code:

bool CSound::Play(CSoundInstance *newinst)
{
  HRESULT hr;
  if (NULL == m_Segment || NULL == m_Manager) {
    if (newinst) newinst->UnInit();
    return(false);
  }
  static CSoundInstance dummy;
  if (NULL == newinst) newinst = &dummy;

  IDirectMusicPerformance8 *perf = m_Manager->GetPerformance();
  IDirectMusicAudioPath8 * pPath = NULL;
  IUnknown *pConfig;

  if (SUCCEEDED(m_Segment->GetAudioPathConfig(&pConfig)))
  {
    // this has an embedded audio path configuration
    hr = perf->CreateAudioPath(pConfig, TRUE, &pPath);
    pConfig->Release();
    ThrowIfFailed(hr, "Couldn't create audiopath.");
  }
  else {
     // no config... make our own!
     hr = perf->CreateStandardAudioPath(DMUS_APATH_DYNAMIC_STEREO,
        CAudioManager::NUMCHANNELS, TRUE, &pPath);
     ThrowIfFailed(hr, "Couldn't create standard audiopath.");
  }
  m_Segment->Download(perf);

  // initialize the sound instance (set volume, etc.)
  // this will also set it Playing.
  newinst->Init(this, pPath);
  return(true);
 }

This code starts out the same, but once it gets the performance and puts it in perf, things change. First, it looks in the segment to see if there's a configuration for an audio path. The config object is like audio path DNA, so if the code finds a configuration, all it must do to create the audio path is give that configuration to CreateAudioPath, which puts the interface of a new audio path inside the pPath variable. Wave files don't have audio path configurations inside them, but music segments authored in DirectMusic Producer usually do.

If there's no configuration embedded in the segment, the code creates a new standard audio path, and puts that interface in pPath. So either way, by the time the code downloads the segment, it has a valid path for this particular sound instance inside pPath. In the final line, the code hands the pPath variable to the sound instance's Init method. Here's the new code for Init:

void CSoundInstance::Init(CSound *sound, IDirectMusicAudioPath8 *path)
{
  HRESULT hr;
  m_AudioPath = path;
  m_Valid = true;
  // set volume based on sound's volume
  SetVolume(sound->GetVolume());
  // now that everything's set, go ahead and Play it.
  hr = GetAudioManager()->GetPerformance()->PlaySegmentEx(
    sound->GetSegment(), NULL, NULL,
    DMUS_SEGF_SECONDARY, 0,
    (IDirectMusicSegmentState **)&m_SegmentState,
    NULL, path);
  ThrowIfFailed(hr, "PlaySegmentEx failed.");
}

The Init method squirrels away the audio path interface into the m_AudioPath member. It then passes that interface to PlaySegmentEx, as the last parameter. This tells DirectMusic to Play the sound using that audio path.

This Init method uses the m_AudioPath interface inside SetVolume when it sets the sound's volume. You'll learn about that method next.

Segment Specific Volume

As mentioned earlier, you control a sound's volume through its audio path. Here's how to get and set the volume of an audio path.

Setting the Volume

The IDirectMusicAudioPath8 interface has a method called SetVolume. This method takes two arguments—a volume (more precisely, a decibel attenuation) specified in hundredths of a decibel, and a duration over which the volume change occurs (making volume fades really easy). The SetVolume method of CSoundInstance uses this SetVolume method of the audio path to change the volume of a Playing sound.

Atop this design, I've built some more advanced functionality into the audio engine. If you look in the CSound class declaration, you'll notice that there's also a SetVolume method there now. This allows you to specify a default volume for new instances of this particular sound. CSound doesn't have to do much to make this happen—it just remembers whatever value you specify. The real work happens in the CSoundInstance::Init method. Notice the line:

SetVolume(sound->GetVolume());

This sets the volume of the audio path to the current volume of the CSound.

Tip

Note the order in which CSoundInstance::Init does things. In particular, notice that it sets the volume on the audio path before it starts Playing the sound. This guarantees that the sound will Play from the get-go with the correct volume. Had the code set the volume after the call to PlaySegmentEx, there might have been a slight instant in which the user heard the sound at full volume. The order in which you do things is important—keep this in mind.

Querying the Volume

Querying the volume of an audio path is similarly easy—just call the GetVolume method of the audio path interface you're interested in. GetVolume takes one parameter: a pointer to a long integer, which it fills with the current volume on the audio path.

Another Volume Knob

Of course, the audio path volume isn't the only volume you can adjust. The underlying DirectSound buffers (remember those?) also have volume knobs, accessible via the SetVolume and GetVolume methods of IDirectSoundBuffer8.

I find it easiest to work with only one volume knob, and leave all the other ones alone. Adjusting both the DirectSound buffer's volume and the segment's volume can make things confusing. I usually just go with the segment volume.

Start Sidebar
Why Such a Funny Volume Range?

Most programmers, used to working with ranges that start at zero and end at one or some other positive number, are completely perplexed as to why DirectMusic's volume control has the range –9,600 to 0.

First of all, the range is specified in hundredths of decibels. That is, –100 is –1 decibel, and –9,600 is –96 decibels. Okay, so that makes it a little more accurate, but why the negative numbers? In the analog recording days of yore, audio engineers measured the quality of their equipment in terms of negative decibels. A rating of zero was impossibly perfect because it meant that the equipment had not lost anything in the process of recording the sound. As the rating dropped below zero, the shoddiness of the equipment rose.

An undamaged human ear has a hearing range of about 120 decibels. That means that if you turned your speakers up loud enough to where permanent hearing damage would result if you Played a sound with volume 0, then you could just barely hear a sound when Played at volume –120 decibels. Of course, a more sensible range would be slightly less than that, say, –96 to 0 decibels.

This method of only reducing the volume of sounds is called attenuation. Virtually all audio hardware today implements volume control using attenuation; in other words, the source samples are never made louder, only reduced.

End Sidebar

Global Volume

There's yet another volume knob you can adjust inside DirectMusic. DirectMusic lets you specify the volume of the performance as a whole. So, how do you do it? Of course, it wouldn't be as easy as calling the SetVolume method of IDirectMusicPerformance8—that method, curiously enough, doesn't exist.

Instead, you have to set the volume via a call to the SetGlobalParam method of the performance interface. Pass in GUID_PerfMasterVolume as the first parameter, the volume you'd like as the second parameter, and the size of a long as the third parameter.

This performance volume knob uses a different range than the audio path. The range depends on the software synthesizer, and for Microsoft's stock synth, the volume here is specified in hundredths of a decibel—anywhere from –200 to +20 decibels. That means the range is actually –20,000 to 2,000, instead of the audio path's –9,600 to 0 range.

I've added GetVolume and SetVolume methods to CAudioManager, but because they're self-explanatory, I won't waste paper dissecting them line by line.

DirectX Volume Control Wrap Up

I hope you've found this "tour of DirectMusic volume knobs" useful, interesting, and fascinating. Like I said before, powerful APIs make the hard stuff easy but the easy stuff hard. Fortunately, the complexity of DirectMusic's volume controls is nothing compared to the Mixer API, which you'll tackle next.

But before you plow forward, stop and explore the Ch5p4_DMusicVolume sample program, which illustrates per-sample and global volume control.


CONTROLLING VOLUME USING THE MIXER API

The DirectMusic volume controls you've learned about so far are all relative to the system's mixer volumes. The mixer volumes are what the user changes when he double clicks the speaker icon in the system tray and adjusts the sliders. Usually, sliders are provided for the main volume, as well as specifically for wave audio output, line in, and so on (see Figure 5.3).

Click To expand
Figure 5.3: Windows provides a GUI to the mixer volumes.
Tip

Tread lightly—the mixer controls adjust the volume of everything on the system, so most users don't like having them changed behind their back. For example, if your game adjusts the master volume, it will not only change its volume, but also the volume of every other application running as well. This might be really rude—consider a user who is listening to MP3s in the background as he Plays your game. If the user goes to your game and adjusts the sound effect or music volume, he would expect the volume of his MP3s to remain constant.

Do not use the mixer volume controls as a cheap way to avoid learning how to control your own application's output.

Adjusting these mixer values programmatically is possible, though it's a maddening exercise of drilling down through several layers of structures and pseudo-objects. This section will show you how.

Architecture of the Mixer API

The architecture of the mixer is summarized in Figure 5.4. There are a lot of structures and pseudo-objects at Play here, so you need to move slowly. If you move too fast and substitute a mixer ID where a control ID should go, you're going to spend a lot of time in the debugger.

Click To expand
Figure 5.4: The mixer API architecture is divided into several different pseudo-objects.

At the top of the mixer architecture is the mixer. Each mixer contains several sources and destinations. You don't modify these sources and destinations directly; instead, you modify imaginary lines that run between them. You can ask the API for the ID of the imaginary line running between any two sources and destinations, or you can ask for the line ID of all sources to one particular destination (for example, the master speaker output).

Each line has multiple controls of various types. You've got your standard issue volume and panning controls, plus potentially more exotic controls, such as meters and equalizers, depending on what sound hardware is in the box. It's worth mentioning that the API also supports completely custom control types—controls that don't fit into any of the pre-defined types. These controls are responsible for disPlaying their own GUI.

Each control usually has the same number of channels as the line. For example, on a stereo line, each control will usually have two channels (left volume and right volume). This is not always the case (most soundcard drivers give the mute control only one channel), so you need to pay attention to the members of the structures given back to you by the API.

Each channel has a value. Normally this value is a signed or unsigned integer, but it can be other things. The valid range for a given value depends on the sound card.

How to Change a Mixer Parameter

Now that you've gotten a taste of the architecture, here's a recipe for changing a specific value on a mixer:

  1. Open a mixer handle by calling the mixerOpen API function. Specify zero for the second parameter, uMxId, to tell Windows that you want to open the first mixer device. Usually this is okay because the first mixer device is usually the only mixer device, but if you're doing something complicated and/or the user has more than one sound card, you'll need to enumerate the available mixers using the mixerGetNumDevs and mixerGetDevCaps API calls.

  2. Figure out what line you want to modify and specify it in terms of source and destination connections (for example, for the wave output volume, the source would be wave out and the destination would be speakers).

  3. Give this information to mixerGetLineInfo, which will give you back a line ID, and the number of channels on that line (for stereo wave out, this will be 2).

  4. Figure out the control within that line that you want to adjust. The easiest way to do this is to ask the line to give you a control ID of a specific type (for example, volume). Call mixerGetLineControls specifying the MIXER_GETLINECONTROLSF_ONEBYTYPE flag, and put the control type you want inside the dwControlType member of the MIXERLINECONTROLS structure.

  5. You'll get back a MIXERCONTROL structure describing your control. Save the dwControlID of this structure; it specifies the control you want.

  6. Figure out the type of control details you need (for example, a volume control has MIXERCONTROLDETAILS_UNSIGNED), and make an array of "value holder" structures of this type. The array needs to be as big as the number of channels on the line. Fill in a MIXERCONTROLDETAILS structure, giving it the control ID, along with your array of value holder structures.

  7. If you want to query the control, call mixerGetControlDetails. This function will fill in each of the value structures with the current value for that particular channel (for example, the first element in your array will correspond to channel 1, the second to channel 2, and so on).

  8. If you want to set new values for the control, you have to set all of the values or none of them—there's no way to say, "Windows, I'd like to set the value for channel 1 on this 2-channel line." What you must do instead is query for all of the channel values, change the ones you want, and then pump the new values (and the old unchanged values) back into the system.

  9. Close the mixer handle you opened in step 1.

Building CMixer

I bet you never thought volume controls could be so complicated, right? Now is the time to encapsulate all of this complexity into a user-friendly CMixer class. I'm not going to go over CMixer line by line (it's all pretty straightforward); instead, I'm going to explain its interface along with my design philosophy, and then leave it to you to figure out how the innards of the class actually work based on the code and the previous section.

I built CMixer mainly to handle volume and muting controls. You might want to extend it to handle other types of controls (for example, the gain control on a microphone, or bass and treble controls).

CMixer uses a dumb class CMixerLine to store a connection from a source to a destination. I call CMixerLine dumb because it does not have methods for setting the volume and muting. In other words, you don't say, "CMixerLine, mute thyself," you say, "CMixer mute this CMixerLine."

A different designer might have made CMixer more into a CMixerLine factory; in other words, you give CMixer a source and destination, and if that forms a valid line, CMixer gives you back a CMixerLine. You would then call the methods of CMixerLine to change the controls on that line.

Tip

One thing I didn't address in the CMixer class is the preservation of panning. Most sound cards expose two channels for volume: left speaker and right speaker. To preserve panning when setting these values, you need to make sure they're in the same ratio (or, set the panning value explicitly later). I leave this for you to implement should you need it.

For a more complex mixer interface, that's probably the way you want to do it, but for a simple volume-only control, I eschewed flexibility in favor of simplicity; with my design, the only class you need to worry about is CMixer.

If you look in the class declaration for CMixer, you'll notice that it includes several static const CMixerLine members. These store the source and destination of commonly used lines, and masquerade as constants. For example, if you want to set the master volume, you'd use the built-in Master member of CMixer in the call to SetVolume, like so:

// set master volume at 50%
m_Mixer.SetVolume(CMixer::Master, 32767);

Of course, if you want to get fancy, you can always specify your own source and destination connections:

// set telephone (modem) volume at 50%
m_Mixer.SetVolume(
  CMixerLine(
    MIXERLINE_COMPONENTTYPE_SRC_TELEPHONE,
    MIXERLINE_COMPONENTTYPE_DST_SPEAKERS
  ),
  32767); // about halfway

The preceding code creates a temporary CMixerLine class that specifies a telephone source and a speaker destination.

Tip

There are many freeware mixer classes available for download on sites like http://www.codeguru.com and http://www.codeproject.com. Check there, and you just might get lucky.

The Ch5p5_MixerVolume sample program illustrates how to use CMixer, and I've also included a couple links regarding the mixer API on your CD.


CONCLUSION

Congratulations! You've come a long way in this chapter, having learned many disparate topics and APIs. The engine has also come a long way; I'd now consider it ready for integration into a simple game. It's still lacking many things, but I think most of the essentials are in place.

In the coming chapters, you'll learn how to further extend the engine, specifically in regard to Playing music and (later) dynamic music, 3D sound, and other goodness.


Chapter 6: MIDI Playback

Download CD Content

OVERVIEW

The previous chapters have taught you how to load, play, and manipulate sound effects. Now it's time to turn your attention to the other half of the game audio equation—background music.

In this chapter, you'll learn the essentials of using MIDI files. This chapter concentrates on playback of static music—that is, music that doesn't change. In Part Two of the book, you'll learn how to work with dynamic music.

DirectMusic tries very hard to treat WAV files and MIDI files the same way. The Loader object loads both MIDIs and WAVs into segments, which are played by giving them to the performance's Play method, just like with sound effects. So, after several difficult chapters, you finally get an easy one.


HOW MIDI WORKS

MIDI is an acronym for Musical Instrument Digital Interface. Originally invented in the 1980s, today MIDI has become the de-facto standard for communicating between electronic musical instruments.

You can think of MIDI as a simple networking protocol. Essentially, you hook up all of your hardware (synthesizers, drum machines, computers, and so on) using MIDI cables, and assign each unique piece of equipment a channel ID (see Figure 6.1). For example, your synthesizer might be channel 2 and your drum machine might be channel 4. Now, when you press a key on your synthesizer, a "Note On" message gets sent out on channel 2. When you hit a drum on your drum machine, a Note On message gets sent on channel 4. MIDI supports up to 16 different channels.

Click To expand
Figure 6.1: A professional MIDI setup might have several different pieces of hardware, all assigned to different MIDI channels.

Software on your computer (or your synthesizer, if you have an advanced one) can listen to any or all MIDI channels and record the "Note On" events. It can then play them back by sending out messages on the appropriate channels. Or, it can save the sequence of notes it received into a MIDI file.

AUDIO CLIP

Play Audio Clips 6.1, 6.2, and 7.3 to hear a couple of differenct examples of MIDI music.

The Note On message itself contains several different fields—of course, there's a number representing which note of which octave to play (for example, middle C, or C4—higher numbers mean higher pitched notes), as well as options for the volume of the note, the duration, and so on. For drums, the "note" is actually the type of drum to play—when a drum synthesizer receives a C4 note, it plays a high bongo drum. For drum machines, higher note numbers do not mean higher pitched drums, they mean different drum sounds altogether, sort of like the ASCII character set.

There are many different types of messages defined by MIDI. Note On is arguably the most important, but there are also messages used to tell devices to change their voice and perform special effects (like vibrato or pitch-bending).

MIDI channels are not the same as tracks. Sequencer software (for example, DirectMusic Producer or Cakewalk Pro Audio) uses the notion of tracks. Whereas there are only 16 MIDI channels, there can be hundreds of tracks. However, at any given time, you can have at most 16 channels active, even though those 16 channels might be spread over a higher number of tracks (for example, some tracks might be muted because they contain different recordings or different effects). This doesn't mean there's a limit of 16 notes being played simultaneously—each channel corresponds to one device or instrument, but many devices or instruments allow several notes to be played at once (called polyphony).

In DirectMusic, MIDI messages aren't usually sent out the MIDI port, through the MIDI wire, and to external synthesizers. Rather, they're sent to DirectMusic's internal synthesizer, which synthesizes the sound in software and ships it off to the speakers. This is a tremendous benefit for video games because it means that you don't have to have thousands of dollars worth of synthesizers and equipment to get MIDI files to sound good; all you need is the standard DirectMusic synthesizer that comes with Windows.


CLASS STRUGGLE

Before you jump in the ring with MIDI music, you need to decide how MIDI files are going to fit in to the structure of the beloved audio engine. As it stands now, the engine uses the CSound class to represent a sound effect. This is okay, but it isn't perfect. Following the principles of good C++ design, this chapter's code changes that slightly. It makes CSound a base class, with three new derived classes: CDirectMusicSegment, CSoundEffect, and CMIDIMusic. Figure 6.2 illustrates this new class layout.

Click To expand
Figure 6.2: The new class layout for the audio engine.

CSound has been gutted. In previous chapters it contained the DirectMusic segment, the volume, and a whole slew of other stuff. Now it's really just an abstract base class. This is because in this new class hierarchy, CSound is the base class for any audio object that comes out the speakers. This includes things like sound effects and MIDI music (which use DirectMusic segments), but also things that have nothing to do with DirectMusic, like MP3 or Ogg Vorbis music files (which you'll learn about in the next two chapters).

CSound has three pure virtual methods, Play, Stop, and Is Playing, along with two more virtual methods, GetVolume and SetVolume. These five methods represent what I consider to be the minimum interface needed for any type of sound that comes out the speakers. Whether it's an MP3, a WAV file, or a chunk of dynamic music composed on the fly by DirectMusic, at a minimum, you'll want to Play it, stop it, see if it's Playing, and adjust its volume.

Derived from CSound is the next new class: CDirectMusicSegment. This class represents a DirectMusic segment, and it contains everything that CSound used to contain. It has the DirectMusic segment pointer, and it uses that segment pointer in its implementations of Play, Stop, and IsPlaying. In the future, if there are additional things that you want to do to a segment (regardless if it's a music or sound effect segment), you can put those methods in this class.

Derived from CDirectMusicSegment are two more classes: CSoundEffect and CMIDIMusic. Methods that make sense for just sound effects or just MIDI music (for example, setting the tempo) belong here.

Why do this? It may seem silly to break out sound effects and music into separate classes, especially because DirectMusic goes to all that trouble making WAV files and MIDI files load into the same segment object. By doing this, DirectMusic allows you to treat WAV files and MIDI files the same way.

Just because you can, however, doesn't always mean that you should. There might be specific things that we want to do to music files that wouldn't really make sense to do to a sound effect. For example, tempo—tempo makes sense when dealing with MIDI music but not when dealing with WAV sound effects. The formats are interchangeable; WAV files and MIDI can both be used for sound effects and music, but the conceptual differences between "background music" and "sound effect" remain. (And hey, if you really don't want to make this distinction, just use the CDirectMusicSegment class instead of the CSoundEffect and CMIDIMusic derivatives!)

The rest of the class hierarchy of the engine remains the same. The manager still oversees the whole operation, and the various other classes (CWAVFile, CMixer, and so on) have not been modified.


LOADING AND PLAYING MIDI FILES

As I mentioned in the beginning of this chapter, DirectMusic tries to treat MIDI music and WAV files the same way. Both are loaded into segments through the Loader. The good news is that the code to load a MIDI song from disk, memory, or resources into a segment is exactly the same as the code to load a WAV file.

Enhancing the Audio Manager

The bad news is that the audio manager's load routines need to get stirred up a little to accommodate this. The audio manager allows you to load WAVs through its LoadSound methods. A good design would be to make it load MIDI files by creating three new LoadMIDI methods (one for loading from disk, one for loading from memory, and one for loading from resources).

You don't want to just cut and paste the LoadSound routines and change their names. You should never cut and paste large code blocks because whenever you do so, you needlessly increase the amount of code that you need to deal with. A better solution is to isolate the common code into three core functions, and then make both the LoadSound and LoadMIDI routines call those functions.

The three core functions are called LoadSegmentFromDisk, LoadSegmentFromMemory, and LoadSegmentFromResource. They take different arguments. Taking just one as an example, here's the new LoadSegmentFromDisk:

void CAudioManager::LoadSegmentFromDisk(CDirectMusicSegment *dest, std::string
filename)
{
   HRESULT hr;
   // convert filename to wide-string
   WCHAR widefilename[MAX_PATH];
   DXUtil_ConvertGenericStringToWide( widefilename, filename.c_str());
   // tell loader to load this file
   hr = m_Loader->LoadObjectFromFile(
     CLSID_DirectMusicSegment,
     IID_IDirectMusicSegment8,
     widefilename,
     (void**) &dest->m_Segment);
   ThrowIfFailed(hr, "LoadObjectFromFile failed.");
 }

That code contains nearly everything that the disk-loading overload of the LoadSound method used to contain. What's missing is the class instantiation; instead of creating a new class, LoadSegmentFromDisk is passed a pointer to one. This is because of the new design. LoadSound creates CSoundEffect classes and passes them to LoadSegmentFromDisk, whereas LoadMIDI creates CMIDIMusic classes. Thanks to polymorphism, LoadSegmentFromDisk doesn't know the difference between a CMIDIMusic class and a CSoundEffect class; it uses their common base class, CDirectMusicSegment.

Here's the new version of LoadSound:

CSoundPtr CAudioManager::LoadSound(std::string filename)
{
  CSoundEffect *snd = new CSoundEffect(this);
  LoadSegmentFromDisk(snd, filename);
  return(CSoundPtr(snd));
}

As you can see, now LoadSound just creates the class, calls LoadSegmentFromDisk to do the dirty work, and returns a new CSoundPtr. LoadMIDI looks virtually identical:

CSoundPtr CAudioManager::LoadMIDI(std::string filename)
{
  CMIDIMusic *music = new CMIDIMusic(this);
  LoadSegmentFromDisk(music, filename);
  return(CSoundPtr(music));
}

Instead of a CSoundEffect, LoadMIDI creates a CMIDIMusic class, then does exactly the same thing as LoadSound.

The other two LoadSound and LoadMIDI overloads look similar—they just create the appropriate class and pass it to LoadSegmentFromMemory or LoadSegmentFromResource.

Getting a CMIDIMusic Pointer

You might be wondering how to get at the methods specific to CMIDIMusic once you've loaded a MIDI file. For example, you might want to change the tempo. The SetTempo method is contained inside the CMIDIMusic class, but all you have is a pointer to a CSound!

To get out of this jam, use run-time type information. RTTI was designed specifically for cases like this—you have a pointer to a base class but need to know if it's actually a derived class. For example, here's how to get a CMIDIMusic pointer from a CSoundPtr:

CSoundPtr snd = g_AudioMgr.LoadMIDI("test.mid");
CMIDIMusic *music = dynamic_cast<CMIDIMusic *>(snd.Get());
If (music != NULL) {
   // snd was actually a CMIDIMusic pointer! Do stuff!
}
else {
  // snd wasn't a CMIDIMusic pointer! D'oh!
}

Notice the dynamic_cast keyword. This keyword converts a pointer into a different type. Here the code is getting the pointer by calling snd.Get, and then trying to convert it to a CMIDIMusic pointer. If the dynamic_cast works, the pointer is put into music (note that music and snd still point to the same memory address; they're just different types). If for some reason the dynamic_cast can't convert the pointer (for example, if you've passed in a pointer to a CSoundEffect class), it will set music to NULL.

In this way you can get to the methods in CMIDIMusic or CSoundEffect, given a CSound or CDirectMusicSegment pointer.


CHANGING TEMPO

One cool thing you can do with MIDI files (and not with MP3s or WAVs) is adjust the tempo on the fly. Tetris popularized the tempo change to create suspense; when your Play area was nearly full, the music would speed up, and when (or if!) you brought it under control again, the music would fall back to normal speed. This simple trick added a lot to the quality of the game, and you can use the same trick for your own titles.

Track versus Performance Tempo Settings

DirectMusic has two different ways you can change the tempo. You can change the tempo track contained in the segment, or you can disable that tempo track, and tell the performance to change the tempo of any segment Playing on it.

This sounds weird, but I've found that the best course of action is to do both. You want to change the segment's tempo so that if you later query the segment for the tempo, it will report the actual tempo—not the tempo it was when it was first loaded. For example, if you load a MIDI file with a tempo of 109 beats per minute, and then you change the performance tempo to 200 beats per minute, when you query the segment for its tempo, it will return 109 (the segment's tempo really is 109—it just happens to be Playing at 200 thanks to the performance settings).

You also want to set the performance tempo because you probably want all of the music that you're Playing to speed up when you change the tempo, not just that one segment.

Fortunately, the code for both of these methods is pretty easy. Here's how CMIDIMusic::SetTempo looks:

void CMIDIMusic::SetTempo(int beats_per_minute)
{
  // step 1: set the track tempo parameter
  DMUS_TEMPO_PARAM Tempo;

  Tempo.dblTempo = beats_per_minute;
  ThrowIfFailed(
    m_Segment->SetParam(GUID_TempoParam, 0xFFFFFFFF, 0, 0, &Tempo),
    "CMIDIMusic::SetTempo: SetParam failed!"
);

// step 2: set the performance tempo
DMUS_TEMPO_PMSG* pTempo;
ThrowIfFailed(
  m_Manager->GetPerformance()->AllocPMsg(
    sizeof(DMUS_TEMPO_PMSG),
    (DMUS_PMSG**)&pTempo),
  "CMIDIMusic::SetTempo: couldn't allocate PMsg!"
);
// Cue the tempo event.
ZeroMemory(pTempo, sizeof(DMUS_TEMPO_PMSG));
pTempo->dwSize = sizeof(DMUS_TEMPO_PMSG);
pTempo->dblTempo = beats_per_minute;
pTempo->dwFlags = DMUS_PMSGF_REFTIME;
pTempo->dwType = DMUS_PMSGT_TEMPO;

ThrowIfFailed(
  m_Manager->GetPerformance()->SendPMsg((DMUS_PMSG*)pTempo),
  "couldn't send PMsg!"
 );
}

This code is based on some sample code given in the DirectX documentation, at DirectX Audio\Using DirectX Audio\Playing Sounds\Timing\Changing the Tempo.

The first step is to set the tempo track to the given value (beats_per_minute). The code does this by filling a DMUS_TEMPO_PARAM structure, and sending it to the track by calling the SetParam method of the segment interface. The DMUS_TEMPO_PARAM structure contains just one member, dblTempo, which is the new tempo you want to set.

The second step is a bit more involved. First, the code must allocate a new PMsg, and store it in pTempo. Once it's allocated, the code fills in the structure. The most important member here is dblTempo, which should be set to the tempo you want to change the performance to.

Note that there are a lot of members in this structure that got set to zero automatically when the code called ZeroMemory. The dwFlags member, however, can't be zero; it must be set to either DMUS_PMSGF_REFTIME or DMUS_PMSGF_MUSICTIME. It doesn't matter which one because the rtTime and mtTime members (not shown in this code) have both been set to zero thanks to ZeroMemory, so I just picked the one with less letters to type.

Similarly, the dwType member also can't be zero. It has to be DMUS_PMSGT_TEMPO, so that DirectMusic knows it's getting a tempo event.

Finally, once the structure is all set up, the code calls SendPMsg, which fires off the pTempo message and causes DirectMusic to change the tempo.

Getting the Tempo

Getting the tempo is much easier than setting it. To get the tempo, just pass a DMUS_TEMPO_PARAM structure to the GetParam method of the segment, like so:

int CMIDIMusic::GetTempo()
{
  DMUS_TEMPO_PARAM Tempo;
  m_Segment->GetParam(GUID_TempoParam, 0xFFFFFFFF, 0, 0, NULL, &Tempo);
  return(Tempo.dblTempo);
}

Nothing to it! Give GetParam the GUID of the parameter you want to receive (GUID_TempoParam), and a structure it should put it in (Tempo), and the requested information will magically appear! Don't worry about the other parameters between GUID_TempoParam and Tempo—they're not usually used. If you want to know what they are, look up GetParam in the DirectX documentation.

The Tempo Scaling Factor

DirectMusic has a second tempo control, called the master tempo, that you also need to be aware of. The master tempo isn't really a tempo so much as it is a scaling factor for the performance's tempo. It is one by default, but you can change it to values between zero and one to slow the performance down, or you can raise it to values above one to speed things up. The DMUS_MASTERTEMPO_MIN and DMUS_MASTERTEMPO_MAX constants define the minimum and maximum limits.

The master tempo is very handy for doing Tetris-style music speedups because you don't need to know the original tempo to speed up a song. All you do is raise the master tempo above one to speed things up, and then set it back to one to restore normal tempo. It's sort of like a virtual scrub bar!

Getting and setting the value of the master tempo is very easy. I've added a couple of new methods to CAudioManager that accomplish this. First, here's SetMasterTempoScaleFactor:

void CAudioManager::SetMasterTempoScaleFactor(float value)
{
  if (NULL == m_Performance) return;
  ThrowIfFailed(m_Performance->SetGlobalParam(GUID_PerfMasterTempo, &value,
sizeof(value)),
    "CAudioManager::SetMasterTempo: SetGlobalParam failed.");
}

The pattern is identical to what you used to set the master volume—just call SetGlobalParam, passing in the GUID of the thing you want to set, along with the new value and the size in bytes of the new value. Nothing to it!

Next up is GetMasterTempoScaleFactor:

float CAudioManager::GetMasterTempoScaleFactor()
{
   if (NULL == m_Performance) return(0.0f);
   float tempo=0.0f;
   ThrowIfFailed(m_Performance->GetGlobalParam(GUID_PerfMasterTempo, &tempo,
 sizeof(tempo)),
     "CAudioManager::GetMasterTempo: GetGlobalParam failed.");
   return(tempo);
}

This is similarly easy—call GetGlobalParam, supplying the GUID of the parameter you want to query, along with a variable to store the value in, and the size of that variable, and DirectMusic will gladly give you back any global parameter you desire.

These are a couple of simple methods that come in very handy when you don't care what the tempo of a song is but you just want to speed it up.


BEAT DETECTION

With MIDI, you can detect the beat of the music and do something with it. For example, you could make an annoying enemy in your game that actually dances to the background music, or you could make strobe lights and other things that flash to the beat.

Tip

There is a bit of latency (20–100ms) between when a beat actually occurs and when your code hears about it. However, this latency isn't perceptible to the human brain, so you're good to go if you just want to sync graphics to music. If you're doing something more advanced, however, (like syncing another machine or synthesizer to DirectMusic's beat), you might need to account for latency. See the DirectX documentation on how to do this.

DirectMusic Notification

Detecting beats requires a bit of code. DirectMusic doesn't have a "IsPlayingBeat" function that you can call; instead, it communicates a different way, using what it calls notification events.

When something interesting happens (for example, a beat, or a segment loop, or a whole slew of other things), DirectMusic adds an event to its internal queue (sort of like Windows). You pull these events off the queue by using the GetNotificationPMsg method of the performance.

Tip

Remember that you've seen PMsg before: when you set the tempo, you sent the performance a DMUS_TEMPO_PMSG. Now, you're on the receiving side of things.

Before you do that, however, you first have to tell DirectMusic what kinds of events you're interested in. You can do this by calling the AddNotificationType method of the performance. Each event is a GUID; you pass AddNotificationType the GUID of the event you're interested in (see Table 6.1).

Table 6.1: GUIDs for Notifications
GUID | Event
GUID_NOTIFICATION_CHORD | A chord change has occurred. This is used mainly to tell you that the DirectMusic composer has decided to change the chord that's Playing..
GUID_NOTIFICATION_COMMAND | A command event has occurred.
GUID_NOTIFICATION_MEASUREANDBEAT | This event is sent out on the beat. A value of zero inside dwField1 indicates a new measure. Any other value indicates a beat within that measure.
GUID_NOTIFICATION_PERFORMANCE | A performance event has occurred. You can look at dwNotificationOption to determine what specifically it was.
GUID_NOTIFICATION_RECOMPOSE | A track has been recomposed by the DirectMusic composer.
GUID_NOTIFICATION_SEGMENT | A segment event has occurred. You can look in dwNotificationOption to determine exactly what happened.

Figure 6.3 illustrates the basic flowchart for using DirectMusic performance events. Essentially you call AddNotificationType during your initialization routine, and then call GetNotificationPMsg periodically (for example, inside your main game loop).

Click To expand
Figure 6.3: A flowchart of the basic process for using DirectMusic performance events.

GetNofiticationPMsg returns S_OK if it gets a message, and S_FALSE if there are no messages to get. So, to process all of the messages that occurred between now and the last time GetNotificationPMsg was called, write code like the following:

DMUS_NOTIFICATION_PMSG* pPmsg;
while (m_Performance->GetNotificationPMsg(&pPmsg) == S_OK) {
  // do something interesting with pPmsg here
  // free it when you're done
  m_Performance->FreePMsg((DMUS_PMSG*)pPmsg);
}

GetNotificationPMsg takes a pointer to a DMUS_NOTIFICATION_PMSG structure you want it to fill. The DMUS_NOTIFICATION_PMSG structure contains everything you want to know about the message, including its type (guidNotificationType), optional subtype (dwNotificationOption), and any optional parameters (dwField1 and dwField2).

Caution

Once you're finished with the notification message you got from GetNotificationPMsg, you must delete it by calling the FreePMsg method of the performance. Failure to do so will result in memory leaks!

Building Notifications into the Audio Engine

Now that you have a basic understanding of DirectMusic's notification/event scheme, it's time to integrate it into the audio engine.

The biggest issue with this revolves around the method you use to communicate events to the code that's using the audio engine. Different games are going to want to listen for different events, and respond to them in different ways. You need a way for the audio engine code to easily tell the game code that something happened.

In C, most programmers solve this by using function pointers. The client code supplies the address of a function they want called when, say, a beat occurs, and the audio framework simply calls that function pointer when the time comes. This is a completely acceptable C solution to the problem.

C++ programmers, however, go one step further, and use a class instead of function pointers. This class contains virtual functions for every type of event that could occur. The audio engine contains a pointer to that class, and when the time comes, the audio engine uses that pointer to call methods on that class.

For example, consider a class called CNotificationHandler. This class contains virtual functions for every type of event that you might want to react to:

class CNotificationHandler
{
public:
  CNotificationHandler() { }
  virtual ~CNotificationHandler() { }
  // this function is called when a beat occurs
  virtual void OnBeat(DMUS_NOTIFICATION_PMSG &msg) { }
  // this function is called when a new measure is encountered
  virtual void OnMeasure(DMUS_NOTIFICATION_PMSG &msg) { }
  // etc..
};

Notice that all of the functions are virtual, and do nothing. Now, imagine that the audio engine has a pointer to this class called m_NotificationHandler. By default, m_NotificationHandler is NULL, but the game code can change that. When an event occurs, the audio engine uses the m_NotificationHandler pointer to call the appropriate method:

void CAudioManager::DispatchNotificationMessages()
{
  DMUS_NOTIFICATION_PMSG* pPmsg;
  while (m_NotificationHandler &&
       m_Performance->GetNotificationPMsg(&pPmsg) == S_OK)
  {
    if (pPmsg->guidNotificationType ==
      GUID_NOTIFICATION_MEASUREANDBEAT) {
      // if it's a beat, call OnBeat... otherwise call OnMeasure.
      if (pPmsg->dwField1 == 0) {
        m_NotificationHandler->OnMeasure(*pPmsg);
      }
      else {
        m_NotificationHandler->OnBeat(*pPmsg);
      }
  }

      // don't forget to free the message when you're done with it!
       m_Performance->FreePMsg((DMUS_PMSG*)pPmsg);
   } // while
 } // method

The beauty of this design is that now the game code can derive a new class from CNotificationHandler and override the virtual functions to do something useful. Then, the code can set the m_NotificationHandler pointer to point to an instance of that derived class, so it can do whatever it wants in response to an event.

As you've probably guessed by now, this is how I wrote the audio manager. The manager has two new accessor methods, SetNotificationHandler and GetNotificationHandler, which enable you to set and retrieve the m_NotificationHandler pointer. The main game calls DispatchNotificationMessages periodically, and DispatchNotificationMessages uses the m_NotificationHandler pointer to dispatch events by calling the OnMeasure, OnBeat, and other virtual methods.

The sample program for this chapter illustrates how this system works. In Ch6p1_MIDIPlayback is a definition for a class called CCh6p1StupidNotificationHandler. This class is derived from CNotificationHandler, but has different method implementations:

class CCh6p1StupidNotificationHandler : public CNotificationHandler {
  virtual void OnBeat(DMUS_NOTIFICATION_PMSG &msg) {
    cout << "..Beat!..";
  }
  virtual void OnMeasure(DMUS_NOTIFICATION_PMSG &msg) {
    cout << endl << "Measure!..";
  }
};

As you can see, it lives up to its name—the stupid class just prints out the messages. Smarter classes would do something more interesting, but for the purposes of the example program, this works fine.

Multithreading Event Detection

If you're disgusted by the idea of having to call the audio manager's DispatchNotificationMessages method all the time, there is another option: multithreading! You can create a separate thread whose sole job is to watch for and respond to DirectMusic events.

Of course, you don't want to continuously loop in this separate thread—that would burn precious CPU time. What you want is to have this thread sleep until a message comes in, dispatch that message, and then go back to sleep until the next message comes in.

Fortunately, DirectMusic provides a way to do this. You can give it a handle to a Windows event primitive, and it will signal that event when a new message is generated. To do this, create the event using the Win32 API function Create Event, then tell DirectMusic that you'd like it to use the event you created by passing it to the SetNotificationHandle method of the performance.

Caution

Make sure that when you create your event, you make it auto-resetting, or that you reset it yourself once it's signaled. Otherwise, you'll never go back to sleep after getting your first message!

To make an event reset automatically, specify false for the second parameter to CreateEvent. For more information, see the Win32 API documentation for CreateEvent.

Now, you can code your thread using WaitForSingleObject, passing it the same event handle you gave to DirectMusic, and it will automatically sleep until DirectMusic signals the event. Once it awakens, the thread can use GetNotificationPMsg to grab the message and do whatever it needs to with it. Then, it can free the message via a call to FreePMsg, and loop until the next event comes in.

The DirectX documentation has an example of this code. You can find it (along with a more detailed explanation of how it works) at DirectX Audio\Using DirectX Audio\Playing Sounds\Notification and Event Handling.

The audio engine does some of this work for you. It creates an event and tells DirectMusic about it. You can access this event through the GetNotificationEvent method of CAudioManager. Then, write your own code using WaitForSingleObject, and don't bother calling DispatchNotificationMessages until you get a message!

I didn't include multithreaded sample code, but it should be fairly easy to piece together given what I've already explained. Whoo-hoo, an exercise for the reader!


MIDI WITHOUT DIRECTMUSIC

As with most of the other things you've learned, if you don't want to use DirectMusic to Play MIDI files, you can use the Windows MCI to Play them. Note that the Windows MCI doesn't give you precise control over MIDI files—for example, you can't synchronize them to other things and you can't Play certain MIDI files. See the Windows Platform SDK documentation, at Platform SDK\Graphics and Multimedia Services\Multimedia Audio\Musical Instrument Digital Interface (MIDI)\About MIDI\Media Control Interface (MCI).

You can also Play MIDI files a third way: by using the low-level MIDI input and output routines. These API routines are what most sequencer programs use to record, Playback, and manipulate MIDI data on a very low level. Here, there's nothing you would want to do that you can't, but most things are fairly difficult to implement and there's a lot of code that must be written. Most games don't need to bother with the low-level MIDI interface—your best bet is to use DirectMusic if you want advanced control and MIDI synchronization capabilities, or MCI if you just want simple MIDI Playback without the constraints of DirectX.


CONCLUSION

This chapter has accomplished several things. The easiest thing you learned is that MIDI files and WAV files both get loaded into segments and Played the same way. However, you also did a little reorganization of the audio engine (setting the stage for chapters to come), and you learned how to send and receive DirectMusic p-messages.

In the chapters to come, you'll learn how to Play other types of sound files (including Ogg Vorbis and MP3), before you come back to the glorious segment and performance interfaces and really crack them open, when you learn about dynamic music.


Chapter 7: MP3 and WMA Playback

Download CD Content

OVERVIEW

In this chapter, you begin learning how to read and play different music formats other than MIDI. MP3 and WMA (Windows Media Audio) are the formats on the chopping block now; in the following chapters, you'll sink your teeth into Ogg Vorbis, CD audio, and tracked music.

But for now, here's everything you need to know about playing an MP3.


MPEG LAYER 3 (MP3) PLAYBACK

Because you're reading this book, I'm going to assume that you know the kitschy basics of MP3s—that MP3 stands for MPEG layer 3, and that MPEG stands for Motion Picture Experts Group (the same group of people who brought you the video algorithm used in DVDs!). That information is interesting, but what really matters is how MP3s work and how you can write code to play them.

How MP3s Work

The magic of the MP3 is that it sounds like a WAV file but is much smaller than one. A typical WAV file of a song will suck up 20-40MB of space, depending on your recording options and the length of the song. MP3s are usually one-tenth of that size, and they sound almost the same. Magic? Nope, just psycho-acoustic compression.

AUDIO CLIP

Play Audio Clips 7.1, 7.2, and 7.3 to hear how the MP3 format compresses sound.

The reason an MP3 can be one-tenth the size of a WAV file and still sound as good lies in the human brain. Specifically, how stupid the human brain is. A song is nothing but a collection of sound waves of various frequencies and amplitudes, and the human brain can't hear all of those different sound waves at once. Some of them are too high or too low, and some are lost behind louder ones. The magic MP3 compression philosophy is "if you can't hear it, why bother encoding it?"

A loud cymbal crash will mute out the acoustic guitar in the background, so there's no need to spend bytes making sure the acoustic guitar sound wave is re-created perfectly when the cymbal crashes. Additionally, there's no need to spend thousands of bytes recording the shape of a sound wave so high-pitched that it's beyond the audible range of human ears. Your dog might prefer original CD tracks over MP3s, but your ears can't detect the missing high-frequency sound waves, so MP3s sound great to you.

The MP3 File Format

Surprisingly, an MP3 file has no header—it's not like a RIFF file with an embedded hierarchy and several header bytes at the beginning. Instead, an MP3 file consists of many frames, and each frame is a self-contained snippet of the song. Each frame has its own little header, which tells you its bit rate, MPEG version (version three is the most popular, but there are also versions one and two), and other information.

Figure 7.1 shows a detailed layout of a frame header. As you can see, the entire header is 32 bits long, and the information is packed into those 32 bits without regard for byte boundaries. This makes things a little irritating because you have to use bit masks to extract some pieces of information (for example, the bit rate is embedded into the high-order word of byte three). The MPEG folks did this to save space—they wanted the MP3s to be as small as possible, and that meant making sure every bit of every byte contained useful information.

Click To expand
Figure 7.1: What's contained in the four bytes of a frame header.

I'm not going to go into the details of what each individual piece of information means, because for what you're going to learn in this chapter, you really don't need those low-level details (however, if you're curious, I've provided some links on your CD that explain things in detail). What you should be aware of is that an MP3 is divided into frames, and each frame can have its own bit rate.

At the very end of the MP3 is what's called an audio tag. This audio tag contains the title, artist, and album strings for the MP3, as shown in Figure 7.2. It's all ASCII data, except for the genre field, which is a one-byte code that stands for a particular genre, as shown in Table 7.1.

Click To expand
Figure 7.2: Illustrates the format of the 128-byte long audio tag.
Table 7.1: Genre Codes
# | Genre
0 | Blues
1 | Classic Rock
2 | Country
3 | Dance
4 | Disco
5 | Funk
6 | Grunge
7 | Hip-Hop
8 | Jazz
9 | Metal
10 | New Age
11 | Oldies
12 | Other
13 | Pop
14 | R&B
15 | Rap
16 | Reggae
17 | Rock
18 | Techno
19 | Industrial
20 | Alternative
21 | Ska
22 | Death Metal
23 | Pranks
24 | Soundtrack
25 | Euro-Techno
26 | Ambient
27 | Trip-Hop
28 | Vocal
29 | Jazz & Funk
30 | Fusion
31 | Trance
32 | Classical
33 | Instrumental
34 | Acid
35 | House
36 | Game
37 | Sound Clip
38 | Gospel
39 | Noise
40 | Alternative Rock
41 | Bass
42 | Soul
43 | Punk
44 | Space
45 | Meditative
46 | Instrumental Pop
47 | Instrumental Rock
48 | Ethnic
49 | Gothic
50 | Darkwave
51 | Techno-Industrial
52 | Electronic
53 | Pop-Folk
54 | Eurodance
55 | Dream
56 | Southern Rock
57 | Comedy
58 | Cult
59 | Gangsta
60 | Top 40
61 | Christian Rap
62 | Pop/Funk
63 | Jungle
64 | Native American
65 | Cabaret
66 | New Wave
67 | Psychedelic
68 | Rave
69 | Showtunes
70 | Trailer
71 | Lo-Fi
72 | Tribal
73 | Acid Punk
74 | Acid Jazz
75 | Polka
76 | Retro
77 | Musical
78 | Rock & Roll
79 | Hard Rock
80 | Folk
81 | Folk-Rock
82 | National Folk
83 | Swing
84 | Fast Fusion
85 | Bebop
86 | Latin
87 | Revival
88 | Celtic
89 | Bluegrass
90 | Avant-garde
91 | Gothic Rock
92 | Progressive Rock
93 | Psychedelic Rock
94 | Symphonic Rock
95 | Slow Rock
96 | Big Band
97 | Chorus
98 | Easy Listening
99 | Acoustic
100 | Humour
101 | Speech
102 | Chanson
103 | Opera
104 | Chamber Music
105 | Sonata
106 | Symphony
107 | Booty Brass
108 | Primus
109 | Porn Groove
110 | Satire
111 | Slow Jam
112 | Club
113 | Tango
114 | Samba
115 | Folklore
116 | Ballad
117 | Power Ballad
118 | Rhythmic Soul
119 | Freestyle
120 | Duet
121 | Punk Rock
122 | Drum Solo
123 | A Cappella
124 | Euro-House
125 | Dance Hall

To read this audio tag, seek to 128 bytes from the end of the MP3 you're interested in, and look for the three ASCII letters, TAG. If you don't find them, the MP3 doesn't have an audio tag. If you do find them, read the last 128 bytes and parse it using the information in Figure 7.2.

Here's how that looks in code:

bool CMP3AudioTag::Read(std::string filename)
{
  char header[4] = { 0 };
  char title[31] = { 0 };
  char artist[31] = { 0 };
  char album[31] = { 0 };
  char year[5] = { 0 };
  char comment[31] = { 0 };
  char genre = 0;
  int handle = open(filename.c_str(), O_RDONLY);
  lseek(handle, -128L, SEEK_END);
  read(handle, header, 3);
  bool result = false;
  if (!stricmp(header, "TAG")) {
    // valid header... this MP3 has an audio tag!
    read(handle, title, 30);      m_Title = title;
    read(handle, artist, 30);     m_Artist = artist;
    read(handle, album, 30);      m_Album = album;
    read(handle, year, 4);        m_Year = year;
    read(handle, comment, 30);    m_Comment = comment;
    read(handle, &genre, 1);
    m_Genre = (MP3_GENRE)genre;
    m_GenreStr = GenreToString(m_Genre);
    result = true;
  }
  close(handle);
  return(result);
}

This code is from a new class, CMP3AudioTag. If you've done file I/O in C, this is easy. The only thing subversive is the call to lseek, which puts you 128 bytes back from the end of the file. From there, you just read the data in the order described, and you're all set!

Tip

My fingers are tired. I made a giant enum, MP3_GENRE, which contains the contents of Table 7.1. I also made a method, CMP3AudioTag::GenreToString, which converts an MP3_GENRE into an STL string.

MP3s in DirectMusic—When a WAV Is Not a WAV

All right, so now you know never to listen to an MP3 with a genre code of 103. Now it's time to learn how to play MP3s using DirectMusic.

As it turns out, it's not quite as easy as you would think. If you try to load an MP3 file into a segment using the Loader, it will fail, with a DMUS_E_UNSUPPORTED_STREAM error. You'd think, given the popularity of MP3 files, that DirectMusic would have made it easy, but alas, it does not. Try to load any MP3 file into the WAV playback program from Chapter 3 (Ch3p1_WAVPlayback), and you'll encounter this error.

Even with this DirectMusic limitation, there's a way you can get Ch3p1_WAVPlayback to play your favorite MP3, and it doesn't even involve changing code or switching formats! Here's the trick:

Take your favorite MP3, and convert it to a WAV file. You should end up with a file that's around 30 or 40 megabytes. Now, this would be a pretty lame trick if that's all there was to it. Of course you can get WAV files to play, but you want MP3s! Don't give up yet; continue reading.

Take that WAV file and load it into Sound Recorder (you know, the lame recording program that comes with Windows as an accessory, shown in Figure 7.3).

Figure 7.3: The Windows Recorder in all its glory.

Select Properties from the File menu, then click the Convert Now button. A dialog box will appear (as shown in Figure 7.4).

Click To expand
Figure 7.4: The Convert Now dialog box.

From the Format drop-down, select MPEG Layer 3.

Click OK twice to dismiss the little dialog as well as the property page. Nothing will appear to happen.

From the File menu, select Save. Now if you go to Explorer and look at the file size of the WAV file, you'll see it's now on the order of one or two megabytes—the size of an MP3.

Sure enough, now Ch3p1 will load and play this WAV file.

What's going on? First of all, remember that a WAV file is a RIFF file, and a RIFF can store many different types of data. Up until now, you've used WAV files to store PCM formatted data (a.k.a. uncompressed sound). But WAV files can actually store sound in a variety of compressed formats, including MPEG Layer 3.

If you open your converted WAV file in a binary viewer, you'll notice that it still has the RIFF header, except the PCM data has been replaced by MP3 data. This is the secret—the DirectMusic loader can load an MP3 if it's got a RIFF header on it, like a WAV file. The RIFF header says, "Hey, this WAV is compressed using MP3 technology," and the Loader then uses the MP3 codec to decompress the pseudo-MP3 and load it into a segment.

Remember that normal MP3 files have no header data—they just contain a whole bunch of frames, and maybe an audio tag at the end. The DirectMusic Loader doesn't know what to make of this, so it assumes it's an unsupported file format and issues the DMUS_E_UNSUPPORTED_STREAM error. But if you take that same set of MP3 frames and wrap it inside a WAV RIFF header, the Loader can make sense of it, and load it without problems.

Tip

The converted WAV file should actually be smaller than the MP3 you started with. This is because the converter can only convert into MP3s with a 56kbps bit rate, whereas your original was probably somewhere in the 128–192 kbps range. The 56kbps rate severely degrades the sound, so you don't want to use it in an actual game (even though it does make for a cool demo).

So, if you're in a pinch, and you absolutely must load things directly through the Loader, but you still want the small file sizes of MP3 files, this is one way to get the best of both worlds.

Playing MP3s Using DirectShow

To play a regular MP3 file, you'll need to enlist the help of another DirectX component: DirectShow.

DirectShow's specialty is in dealing with complex compressed file formats, like MP3. You can also use it to play WMA files, as well as AIFF, AU, SND, and even WAVs and MIDIs. Happily, the process I'm about to describe to you will work without one change for all of those types of files. This makes it well worth the effort to learn a bit about how DirectShow works.

DirectShow Interface Basics

DirectShow works by stringing together filters. Essentially, a filter is something that does one thing to a multimedia stream. Think of filters as the software equivalent to rack hardware you so desperately want to buy at your favorite music shop. For example, a record player's job is to feel out the grooves on some vinyl and send electronic pulses out the audio L/R output jacks. If you're like me, you'll probably want to connect the output of the record player to the input of an equalizer, whose sole job is to adjust the amplitudes of sound frequencies. Then, you'll probably want to connect the output of the equalizer to the input of the amplifier, whose job is to send those electronic pulses out to your speakers.

It works the same way in DirectShow. For example, to play an MP3 file, you might start with an MP3 codec, which will unpack the MP3 and send the audio bytes out. You might connect the output of the codec to the input of the Default DirectSound device, which is analogous to the amplifier—its job is to send the audio bytes out to the speakers. Of course, you could get fancier than that, but you don't have to. What you've created is a simple filter graph. In DirectShow terminology, a filter graph is a set of filters all wired together.

In fact, you don't even have to deal with connecting the filters at all. You can use DirectShow's filter graph manager to automatically create standard filter graphs for different types of source files. You just say, "Hey, Mr. Filter Graph Manager sir, I want to play this MP3," and the graph manager takes care of all the dirty work of hooking up the filters.

So, getting an MP3 to play takes a couple of steps, but it's nothing too complex. Here's how it works.

Streaming an MP3

The first thing you must do is create some interfaces. You'll need three: an IGraphBuilder interface, which you'll tell to build the filter graph; an IMediaControl interface (no relation to Windows's Media Control Interface API), which has methods for playing, pausing, and stopping a filter graph; and an IMediaEvent interface, so you can listen for events (like the event that says, "Hey, your MP3 is done playing"). Some people like to also create a fourth interface, IMediaSeeking, so they can have virtual fast forward and rewind buttons.

Of those interfaces, the graph builder interface is the only one you have to create. You can query the graph builder for the other three interfaces.

Here's how the creation code works:

CoCreateInstance(CLSID_FilterGraph, NULL, CLSCTX_INPROC_SERVER,
  IID_IGraphBuilder, (void **)&pGraphBuilder);
  pGraphBuilder->QueryInterface(IID_IMediaControl,
    (void **)&pMediaControl);
  pGraphBuilder->QueryInterface(IID_IMediaEventEx,
    (void **)&pMediaEventEx);

Here you can see the code to create a graph builder and query for the other two interfaces. This is pretty standard stuff for you COM programmers. The IIDs are defined in dshow.h.

Once you have your interfaces created, you tell the graph builder to build the filter graph by calling its RenderFile method, like so:

WCHAR widefilename[MAX_PATH];
DXUtil_ConvertGenericStringToWide( widefilename, filename.c_str());
mp3->m_GraphBuilder->RenderFile(widefilename, NULL);

As with most things DirectX, RenderFile wants a wide-character version of the name of the file for which you want it to create the graph. The second parameter is currently reserved, and must be set to NULL.

Once you have the filter graph in place, you can press the virtual play button by using the media control interface:

ThrowIfFailed(pMediaControl->Run(), " Run failed.");

There are also media control interface methods for stopping (Stop) and pausing (Pause). However, Stop doesn't automatically rewind the MP3 to the beginning—for that, you'll need to use the IMediaSeeking interface.

Once the MP3 is done playing, you can Release all of your interfaces. Again, follow good COM programming tactics and release them in the opposite order you created them: media control, media seek (if applicable), media event, and finally IGraphBuilder.

DirectShow Messages

As you play, stop, and otherwise manipulate the filter graph, DirectShow sends you messages, called events. You retrieve these messages using the IMediaEvent interface, like so:

long eventcode, param1, param2;
while (m_MediaEventEx->GetEvent(
  &eventcode, &param1, &param2, 0) == S_OK) {
  if (eventcode == EC_COMPLETE) {
    // MP3 is done playing
  }
  m_MediaEventEx->FreeEventParams(eventcode, param1, param2);
}

This code looks very similar to what you did last chapter with DirectMusic notifications. Essentially, you call the GetEvent method, which pulls the first event off the internal queue and returns it in the spaces you've provided. If there are no more messages, GetEvent doesn't return S_OK, and the loop ends. Once you're finished with the event, you must call FreeEventParams to clean up the memory DirectShow allocated.

Nothing to it! Now that you know all that, here's how to add MP3 support to the audio engine.

Adding MP3 Support to the Audio Engine

For starters, you need to link with strmiids.lib, which contains the IIDs of the interfaces you're using. That's easy enough.

Next up—some new classes. I've created a CDirectShowMusic class which represents any type of audio file you can stream from DirectMusic. CDirectShowMusic is derived from CSound, the top-level generic sound object.

I derived a CMP3 class from CDirectShowMusic just in case you wanted to add any methods specific to MP3s (say, manipulating their audio tags). For that matter, I also created a new CMP3AudioTag class, which you got a glimpse of a couple sections ago.

Right now, CMP3 is essentially an empty derived class. It just contains a CMP3AudioTag. All the interesting code is in CDirectShowMusic. The CDirectShowMusic class implements the pure virtual functions Play, Stop, and IsPlaying (defined in CSound).

The most difficult part of adding MP3 support to the beloved audio engine is handling the events. There's no easy way to ask a filter graph if it's still playing, so the audio engine needs a bulletproof way of emulating this, so that the IsPlaying pure virtual function works reliably.

At first it seems easy: make a Boolean member variable and set it to false in the constructor. Whenever Play is called, set it to true, and whenever Stop is called, set it to false. But this doesn't handle the case in which the MP3 file plays in its entirety, then stops on its own. To capture this requires listening for DirectShow events.

So, I tacked some new code onto the DispatchNotificationMessages method of the audio manager (the method you learned about in the last chapter). DispatchNotificationMessages now calls a static method of CDirectShowMusic, called DispatchDirectShowNotifications.

DispatchDirectShowNotifications loops through all of the CDirectShowMusic classes currently instantiated, and calls the non-static member function ProcessMyDirectShowNotifications. This method works as described in the previous section: it calls GetEvent repeatedly until no more events are in the queue. If it sees an EC_COMPLETE event, it sets the Boolean m_IsPlaying to false. This guarantees that m_IsPlaying is always accurate.

Finally, I added one new method to CAudioManager. The new method, LoadMP3, loads an MP3 into a new CMP3 object and returns it in a CSoundPtr. Pretty run of the mill stuff, but be sure to step through the code if you're unclear on anything.


WINDOWS MEDIA AUDIO (WMA) PLAYBACK

Windows Media Audio is a format engineered by Microsoft and touted as a better compressed file format than MP3. Microsoft asserts that you can store twice the amount of music in the same space if you use WMA files instead of MP3 files. They assert that a WMA file with a bit rate of 64kbps sounds equivalent to an MP3 with a 128kbps bit rate.

You'll have to decide for yourself if they're right. Arguments about which file format is best abound on the Internet, and numerous people have put all sorts of audio file formats through their paces looking for the true "best." I've included, on your CD, links to several studies.

If you do decide to use WMA, you'll be happy to know that it's very easy. Simply use the DirectShow MP3 streaming code! Out of the box, it will play WMA files.

I just wanted to mention it here in case you looked through the table of contents and came straight here. The knowledge you seek is mere pages behind where you now read!


CONCLUSION

This chapter should have been called "MP3 with a hint of WMA." In this chapter, you tamed DirectShow into playing MP3, WMA, and several other types of files for you. You also learned a little about the file format of an MP3, and a clever trick to make an MP3 appear to be a WAV file.

In the next chapter, you'll continue your voyage through MusicFileFormatLand, stopping to learn about Ogg Vorbis, an open-source alternative to the MP3 format.


Chapter 8: Ogg Vorbis Playback

Download CD Content

OVERVIEW

Two compressed audio formats down, one to go. In this chapter, you'll learn how to extend the audio engine to support the Ogg Vorbis codec. In case you haven't heard of it before, Ogg Vorbis is an open-source codec created by Xiphophorus (xiph.org). It's popular on Linux due to its open-source heritage, and is quickly gaining momentum as an alternative to MP3.

Tip

If you do use Ogg Vorbis extensively in your game, I implore you to give something back to the Ogg Vorbis community. Open-source software is rooted in the idea that one person looks at something, makes it a bit better, then releases it as open source so that others can improve it further. Donate some money to Xiph.org, so that they can keep releasing great code, and/or donate some brainpower by contributing to the codec's development.

The thing that makes Ogg Vorbis so great is that it's open source. The specification itself is free—you have every right to write code that reads or writes OGG files, without having to pay for a license. Also, the Ogg Vorbis libraries are available under the BSD license, meaning that you can use them in whole or in part in your own closedsource applications.

Many studies, some scientific and some less so, have been done comparing Ogg Vorbis to MP3 to WMA and so on, but you'll have to evaluate each format yourself and decide which is best for you. I personally think Ogg sounds better than MP3, but people may argue that the high-school years I spent blasting Ozzy and MegaDeth have taken their toll on my audio wetware. Don't take my word for it—rip a WAV file into technically equivalent MP3 and Ogg Vorbis formats and see which one you like better.

AUDIO CLIP

Audio Clips 8.1 and 8.2 are examples that compare a regular WAV file with its Vorbis-encoded equivalent.

Also, you should realize that what I'm about to show you cannot be done with the MP3 file format without paying money. This chapter embeds an Ogg Vorbis decoder directly into the audio engine. MP3 requires you to pay a licensing fee to do that (we skirted the issue last chapter because Microsoft's DirectShow was doing the decoding, not us). Ogg Vorbis, on the other hand, allows you to decode the format yourself without having to pay a fee.


STRUCTURE OF OGG VORBIS

The Ogg Vorbis API consists of a slew of different lib files, each one designed for a particular purpose. To start with, the Ogg Vorbis API allows both static linking and the use of DLLs. The static libraries are self-contained, and require no external DLL files, so for most games you'll want to use those and avoid the hassle of DLLs. However, you can also use the stub libraries, which require certain DLLs to operate. You can find these DLLs in your bin directory.

The Ogg Vorbis API is divided into four different libraries: the Ogg library, the Vorbis library, the Vorbisenc library, and the Vorbisfile library.

The Ogg and Vorbis Libraries

These two libraries contain the basic functions needed to work with Ogg Vorbis files.

Not surprisingly then, the Ogg library contains general-purpose routines for dealing with Ogg files. It knows nothing about Vorbis; in fact, the Ogg library doesn't know (and doesn't care) if you're manipulating audio or video data. It just creates the base framework. The Vorbis library contains the foundation functions necessary for decoding streams using the Vorbis codec.

Start Sidebar
Ogg Vorbis: Funny Name, Serious Codec

Some enlightenment on the funny name: Xiph.org has created a framework for compressing audio and video. This framework is called Ogg. The actual audio codec is called Vorbis. This is analogous to RIFF—just as a RIFF file can contain MP3-encoded data, an Ogg file can contain Vorbis-encoded data. Currently the Vorbis codec is the only codec that uses the Ogg framework, but Xiph.org plans to release additional codecs for a wide variety of applications. These codecs, when released, will also use the Ogg framework, so think of Ogg as sitting above Vorbis.

End Sidebar

The Vorbisenc Library

This library is designed to help you with encoding Vorbis files. In here, you'll find functions to set up and tear down the Vorbis encoder. If you're creating an application that rips audio CDs into Ogg Vorbis files, this is your library, but games don't usually need to use this.

The Vorbisfile Library

This is a convenience library used to deal with Ogg Vorbis files. This library uses the Ogg and Vorbis libraries and adds some code that creates a much simpler interface to them, specifically designed for decoding files. This is the library best suited to games, so it's the library you'll learn the most about during this chapter.

Getting Help

There's one last thing I should mention before we move on, and that's how to get to the documentation for the Ogg Vorbis libraries. Inside your installation folder is a doc folder which contains an HTML file called index.html. That's the root of the documentation. If you know specifically what API you want help with, you can shortcut that and go into one of the folders located under doc (ogg, vorbis, vorbisenc, or vorbisfile), and fire up the index.html file located there for help specific to that library.


FROM ZERO TO VORBIS

It all starts by installing the libraries. On the CD you'll find several Ogg Vorbis ZIP files (in the Goodies folder). The following list shows you how to install those files. (I included everything on the CD if you're curious, but the only two files you need for this chapter are libvorbis-1.0.zip and OggVorbis-win32sdk-1.0.zip.)

The goal is to put all of the files from both zip files into one directory tree. So, you need to unzip each file into a separate directory (making sure to maintain the directory structure stored inside the zip file), and then copy the contents of each directory into one main directory (it's safe to overwrite files).

Either way, you should end up with a tree that contains include, lib, and win32 examples, and a whole bunch of other folders. In particular, make sure that you have both an ogg folder and a vorbis folder inside your include folder.

  1. Start a command shell (Go start, run, then type cmd on Windows NT, 2000, or XP, or command on Windows 95, 98, and Me).

  2. You need to set an environment variable called SRCROOT. To properly install the libraries, SRCROOT must point to the top-level folder where you unzipped the files. For example, if you copied both folders into a single folder called C:\libraries\oggvorbis, then at the command prompt you should type set SRCROOT=C:$libraries\oggvorbis. I'd highly recommend making sure that there are no spaces in the your install path, since then you won't have to deal with quoting this path.

  3. Type cd %SRCROOT%\win32. If you've set your SRCROOT environment variable correctly, you should end up in the win32 folder, C:\libraries\oggvorbis\win32.

  4. Inside this folder are batch files that build all of the different Ogg and Vorbis libraries. If you want to, you can run each batch file, but the only ones you need to run right now are build_vorbis_static.bat, build_vorbis_static_debug.bat, build_vorbisfile_static.bat, and build_vorbisfile_static_debug.bat. These batch files will create vorbis_static.lib, vorbis_static_d.lib, vorbisfile_static.lib, and vorbisfile_static_d.lib, respectively. Note that you will get some warnings when you compile these.

That's all it takes! I followed this exact process when building Ch8p1_OggVorbisPlayback. In the Chapter 8 source code folder, you will find a folder called libvorbis-1.0, which contains the built libraries.

The next step is to set up your DevStudio project to use the Ogg Vorbis libraries. If you want to add Ogg Vorbis to a project, you need to do a few things:

  1. You need to add the Ogg Vorbis include path to your project. To do this, select Settings from the Project menu, and go to the C/C++ Tab. Choose Preprocessor from the Category dropdown. In the Additional Include Directories edit box, type a relative path from your project to the include folder underneath where you installed Ogg Vorbis. For example, for this chapter's example program, I would enter ..\..\libvorbis-1.0\include.

  2. You need to link with the Ogg Vorbis libraries. To do this, go to the Link tab in the settings dialog, and select the Input category. In the edit box labeled Object/library modules, add the Ogg Vorbis libs you want to use. For this chapter's sample program, I've added vorbisfile_static.lib, ogg_static.lib, and vorbis_static.lib.

  3. You need to add the Ogg Vorbis library directory to your library path. In the same place, a couple of edit boxes below Object/library modules, is another edit box called Additional Library Path. Add the relative path to your Ogg Vorbis lib directory here. For the Ch8p1_OggVorbisPlayback sample program, this is ..\..\libvorbis-1.0\lib. Separate paths using commas.

  4. In your code, make sure you include the right Ogg Vorbis headers, and make sure you use vorbis/ in their path. For example, if you wanted to include vorbisfile.h, you'd type #include <vorbis/vorbisfile.h>. The preprocessor tacks this onto the relative path you added in step 1, so you will end up with something like ..\..\libvorbis-1.0\include\vorbis\vorbisfile.h.


USING VORBIS

Hooray! If you're reading this, I'll assume you've successfully completed the most frustrating part of dealing with a third party API—installing and configuring the libraries. Now it's time to start looking at some Ogg Vorbis code.

Using the Vorbisfile API

The vorbisfile API is the simplest Ogg Vorbis API available. It allows you to easily read Ogg Vorbis files and convert them into uncompressed PCM streams. Figure 8.1 shows how to use the API. There are other APIs available (check out the Ogg Vorbis docs), but for this chapter you'll be using the Vorbisfile API exclusively.

Click To expand
Figure 8.1: Flowchart for using the vorbisfile API.

The API is structured in such a way as to resemble the process of reading a normal file. To work with a normal file, you open it, read from it, and then close it when you're finished. Similarly, before you can do anything else with an Ogg Vorbis file, you have to open it using the ov_open API call. This call takes as input a handle to a file that you've already opened with fopen, and the address of a OggVorbis_File structure it should fill up.

Once you've opened an Ogg Vorbis file, you can do a few different things with it. You can get some information about it by calling the ov_info API function. This takes as input a pointer to an OggVorbis_File structure (the same one you got back from ov_open), and the address of a vorbis_info structure it should fill up. The vorbis_info structure, declared in vorbis/codec.h, contains the version of the file, the number of channels, its sample rate, and possibly its bit rate (depending on if the file's bit rate changes or not).

Other useful API functions include ov_pcm_total, which, if it can, gives you the total number of samples, and ov_comment, which gives you the user comments in the file (artist, album, title, and so on).

However, the most useful API function is ov_read. This is what decompresses the file into a PCM stream. The ov_read function takes as input a pointer to an OggVorbis_File structure, a buffer of bytes, the length of that buffer (along with some parameters that tell it if you want big-endian or little-endian output), the size of your platform's word, and whether you want signed or unsigned data.

The last parameter to ov_read is a pointer to an int denoting the current logical bitstream. For simple decoding, what you want to do here is just set an int to zero, and pass the address of that int to all ov_read calls. This frees you from having to worry about logical bitstreams; however, if you're curious, check out the Ogg Vorbis documentation.

The ov_read function may not completely fill up the byte buffer you give it. Thus, it's important to check the return value, which tells you how many bytes you got back. If the return value is zero, you've hit the end of the Ogg Vorbis stream.

Once you're all done reading the file, you close it by using the ov_clear API call. Note that this also closes the FILE pointer; there's no need to call fclose yourself.

Integrating Ogg Vorbis into the Audio Engine

Now that the libraries are installed and you know how the APIs work, you can start slinging some code around. In this section, you'll learn how to load an Ogg Vorbis file into a DirectMusic segment. This sounds tough, but it really isn't. All you need to do is decompress the Ogg Vorbis file into a PCM stream (a.k.a. uncompressed stream), wrap that PCM stream in a wave file header, and pass it off to DirectMusic's Loader for loading into a segment.

You have all of the pieces of the puzzle. One of the Ogg Vorbis example programs (vorbisfile_example.c) illustrates how to use the vorbisfile library to convert an Ogg Vorbis file into a PCM stream, and thanks to Chapter 4, "Loading WAV Files," you've already written a CWAVFile class that can create a WAV file from a chunk of uncompressed data. Things are looking good!

I added a new method to CAudioManager, called LoadOggVorbis. Here's the source code:

CSoundPtr CAudioManager::LoadOggVorbis(std::string filename)
{
   std::vector<unsigned char> pcmdata;
   // this code is based on vorbisfile_example.c,
   // included in the ogg distribution.
   char pcmout[4096]; // ogg decoding buffer
   OggVorbis_File vf;
   int eof=0;
   int current_section;
   FILE *f = fopen(filename.c_str(), "rb");
   if (f == NULL) { Throw("could not open file."); }
   if(ov_open(f, &vf, NULL, 0) < 0) {
     Throw("Input does not appear to be an Ogg bitstream.");
   }
   /* Throw the comments plus a few lines about the bitstream we're
      decoding */
   vorbis_info *vi=ov_info(&vf,-1);
  // resize the pcmdata vector to the size of the ogg file
  // two bytes per channel per sample
  int samples = ov_pcm_total(&vf,-1);
  int channels = vi->channels;
  int samplerate = vi->rate;
  if (samples > 0) {
    pcmdata.reserve(channels*2*samples);
  }
  while(!eof){
    long ret=ov_read(&vf,pcmout,sizeof(pcmout),0,2,1,&current_section);
    if (ret == 0) {
      /* EOF */
      eof=1;
    } else if (ret < 0) {
      /* error in the stream. Not a problem, just reporting it in
       case we (the app) cares. In this case, we don't. */
    } else {
      /* we don't bother dealing with sample rate changes, etc, but
       you'll have to*/
      // transfer the data out of the ogg buffer and into ours.
      for (int q=0; q < ret; q++) {
        pcmdata.push_back(pcmout[q]);
      }
  }
}
/* cleanup */
ov_clear(&vf);

// now that we have the PCM data in memory, convert it to a WAV file
// and handoff to DirectMusic for loading into a segment.
CWAVFile wavfile;
wavfile.m_AudioFormat = WAVE_FORMAT_PCM;
wavfile.m_BitsPerSample = 16;
// 16 bits / 8 bits per byte * channels
wavfile.m_BlockAlign = 2*channels;
wavfile.m_ByteRate = samplerate*channels*2;

wavfile.m_NumberOfChannels = channels;
wavfile.m_SampleRate = samplerate;
wavfile.SetData(pcmdata.begin(), pcmdata.size());
return(LoadSound(wavfile));
}

Most of this source code is a direct cut and paste from vorbisfile_example.c. The vorbisfile_example.c reads an Ogg Vorbis stream from stdin and writes an uncompressed PCM stream to stdout; I've modified it so that it reads an Ogg Vorbis stream from a FILE pointer and writes it into a vector of unsigned chars.

Note that this code knows the output wave is going to be 16 bits, because it specified a 16-bit word size to the ov_read function by passing 2 as the fifth parameter. If you wanted an 8-bit wave file, you'd pass 1 instead.

Once the PCM stream is in the pcmdata vector, the code gives it to a CWAVFile instance (wavfile), along with the details of its format (number of channels, sample rate, and so on). Once wavfile is all put together, the code calls the LoadSound overload you learned about in Chapter 4 to read the wave file class and construct a DirectMusic segment from it. Nothing to it!


CONCLUSION

In this chapter, you learned how to use the open-source Ogg Vorbis codec as an alternative to MP3 and WMA. However, there's much more to Ogg and Vorbis than just how to decode them. I encourage you to experiment around with the other Vorbis libraries, and to look at the technical specs through which Vorbis accomplishes its magic—it's fascinating reading.

This wraps up the tour of compressed file formats. In the next chapter, you'll learn how to read and play tracked music formats, a staple of the demo scene, and as you'll see, also quite useful in game programming.


Chapter 9: CD Audio Playback

Download CD Content

OVERVIEW

If your game is going to end up on CD, congratulations—you have the easiest and most versatile approach to implementing game music. Unless you're packing that disc to the gills (and can spare no space for uncompressed audio), or you need music that changes at runtime, you'll want to use CD audio for your game's music.

This chapter is going to concentrate on the creation of a CCDPlayer class. Here's the class declaration, taken from  CDPlayer.h inside the Ch9p1_CDPlayer program folder:

class CCDPlayer
{
public:
  CCDPlayer();
  virtual ~CCDPlayer();
  void Init(HWND hwnd, char driveletter = 0);
  void UnInit();
  void Play(int track);
  bool IsPlaying();
  bool IsPaused();
  void Stop();
  void Pause();
  void Resume();
  void OpenCDDoor();
  void CloseCDDoor();
  bool IsCDInDrive();
  CTrackMinSecFrame GetCurrentPos();
  int GetTrackLength(int track);
  int GetNumTracks();
  static std::vector<CCDDeviceInfo> GetAvailableDevices();
protected:
  bool m_InitGood;
  bool m_Paused;
  CTrackMinSecFrame m_PausedPos;
  static void HandleError(MCIERROR err);
  HWND m_hWnd;
  DWORD m_DeviceID;
};

As you can see, this is a no-frills CD player—there are no methods for seeking within a track or probing the specific capabilities of a CD drive. This class only knows the simple stuff—everything a game would need, and nothing more.

Notice that this class relies on a couple of other classes—CTrackMinSecFrame, and CCDDeviceInfo. Both of these are really just collections of public variables. CTrackMinSecFrame contains four integers—m_Track, m_Minute, m_Second, and m_Frame, which together define a specific time position in a CD. CCDDeviceInfo contains a few strings that describe a given CD audio drive. The drive letter is stored here (m_DriveLetter), alongside a short description of the drive (m_Description) and the UPC code (m_CDProductCode) and CD ID (m_CDID) of the CD in the drive, if any. All of these tidbits of data come from Windows, and you'll learn more about them in a few sections.

Now, here's how to flesh out that class declaration using the MCI.


USING MCI FOR CD AUDIO

You've gotten a taste of the Windows MCI before, back in Chapter 3. Back then, I told you that MCI stood for Media Control Interface, and that the API was designed for multimedia devices, like CD players. The MCI is the easiest way to use CDs. There are ways to talk to them on a lower level, but unless you're writing audio extraction software, you won't need them.

Figure 9.1 illustrates the process of using MCI: you see what devices are available, pick one and open it, bark commands at it, and close it when you're done. You can interface via API functions, or you can actually format a text command string and send that. For example, the command string "play thissound from 0 to 2000 notify" tells the system to play the "thissound" from its beginning to position 2000, and to notify you when it's done. If you forget the syntax of the command strings, just pretend you're a drill sergeant speaking to a CD player.

Click To expand
Figure 9.1: The process for using the Media Control Interface (MCI).

The MCI library is a powerful tool, capable of doing everything most games will require from a CD player. For more information on the API as a whole, check out the MSDN documentation, at Platform SDK\Multimedia Audio\MCI.

MCI is a dual-interface API—you can use freeform text commands (the command string interface) or a set of message IDs and structures (the command message interface) to communicate with it. I've chosen to ignore the free-form string interface, and do everything through the message interface.

The Mother Brain of the message interface is the mciSendCommand API call. This function takes a command message ID (for example, MCI_PLAY), along with a structure for that message (for example, MCI_PLAY_PARMS). It's a little scary at first, because you have to cast the address of your structures to a DWORD as you pass them to the function:

MCI_PLAY_PARMS mciPlayParms;
ZeroMemory(&mciPlayParms, sizeof(MCI_PLAY_PARMS));
mciPlayParms.dwFrom = MCI_MAKE_TMSF(bTrack, 0, 0, 0);
mciPlayParms.dwTo = MCI_MAKE_TMSF(bTrack + 1, 0, 0, 0);
mciPlayParms.dwCallback = (DWORD) hWndNotify;
MCIERROR e = mciSendCommand(wDeviceID, MCI_PLAY,
    MCI_FROM | MCI_TO | MCI_NOTIFY, (DWORD)(LPVOID) &mciPlayParms);

This is a typical chunk of code taken from the CD player class's Play method. You can see the code setting all of the different members of the MCI_PLAY_PARMS structure, the one required by the MCI_PLAY command. The code specifies the play range via the dwFrom and dwTo members, throws in a window handle for a callback, and then calls mciSendCommand to fire off the command. The second parameter to mciSendCommand is the message ID, and the third parameter is the set of flags for the message.

Don't worry about the details yet—I just want you to become familiar with this pattern because you'll see it throughout the CD player class. If mciSendCommand succeeds, it returns zero; otherwise, it returns an MCI error code which you can decode into a text string via the mciGetErrorString function.

Enumerating, Opening, and Closing Devices

The first order of business is to figure out what kind of hardware your gamer has. There was a time when this was easy because most PCs had zero or one drives. Thank the proliferation of CDRW drives and USB for making this complex—nowadays a user can theoretically have dozens of peripherals from which a CD could be played. Your game disc is in one of them—how do you figure out which one?

It's at this point I need to mention that sometimes, the way "They" want you to do things is needlessly complex. For example, if you were to start browsing through the MSDN documentation on the MCI API, you'd quickly discover that the proper way to enumerate devices is to send the MCI_SYSINFO command once with certain flags, to get the number of MCI devices, and then several more times (with different flags) to get each individual device.

But unless you're doing something above and beyond just simple music playback, you probably don't need to do that. Instead, you can use an easier method.

Windows gives each CD audio-capable peripheral (CD player, DVD player, CD burner, and so on) a drive letter between D and Z (remember, drives A and B are usually floppies and drive C is usually a hard drive). Therefore, all you need to do is loop through drive letters D through Z, and see if a CD audio device is connected to that drive. If one is, you can strike up a conversation with that drive to determine what CD (if any) is in it. This is easier than doing proper MCI enumeration, and it also tells you the drive letter of your game CD (in case you want to play movies from it or something).

Here's the CCDPlayer::GetAvailableDevices method. This method looks at all the devices on the system and returns a vector of device information classes—one device information class for each CD audio drive it finds:

vector<CCDDeviceInfo> CCDPlayer::GetAvailableDevices() {
  DWORD retvalue=0;
  MCIERROR e;
  vector<CCDDeviceInfo> devices;
  // loop through entire alphabet of drive letters
  for (char drive = 'C'; drive <= 'Z'; drive++) {
    char buf[256];
    char driveletter[3] = "C:";

    // see if we can open this drive as a CD audio device
    MCI_OPEN_PARMS openparms;
   ZeroMemory(&openparms, sizeof(MCI_OPEN_PARMS));

    openparms.lpstrDeviceType = (LPCSTR) MCI_DEVTYPE_CD_AUDIO;
    driveletter[0] = drive;
    openparms.lpstrElementName = driveletter;
    e = mciSendCommand(NULL, MCI_OPEN,
      MCI_OPEN_TYPE | MCI_OPEN_TYPE_ID | MCI_OPEN_ELEMENT,
      (DWORD)(LPVOID)&openparms);
    if (!e) {
      // success! Query this CD Audio drive and add it to
      // our vector.
      CCDDeviceInfo devinfo;
      devinfo.m_DriveLetter = drive;
      int deviceid = openparms.wDeviceID;
      // removed from this listing to save space:
      // ask device for its description
      // ask device for the product code of the CD in it
      // ask device for the ID of the CD in it
      // close device
      MCI_GENERIC_PARMS genericparms;
      ZeroMemory(&genericparms, sizeof(MCI_GENERIC_PARMS));
      mciSendCommand(deviceid, MCI_CLOSE, MCI_WAIT,
       (DWORD)(LPVOID)&genericparms);

      // add it to the vector
      devices.push_back(devinfo);
    }
  } // next drive letter

  return(devices);
}

I've stripped out the repetitive guts of this GetAvailableDevices function so you can more easily see the overall structure.

Essentially, the code runs through the alphabet, trying to open each drive letter as a CD audio drive. Of key importance is MCI_OPEN_ELEMENT, which triggers the MCI to look at the lpstrElementName member of the MCI_OPEN_PARMS struct. In this context, lpstrElementName is actually the drive letter of the device.

If the drive letter given is a CD audio device, the code will receive a success return code (zero) from the MCI_OPEN message sent with mciSendCommand. This tells the code that the drive letter refers to a device capable of playing an audio CD. The code then sends a few MCI_INFO messages, asking the drive for its description, as well as the product code and ID of the CD in its drive (if any). I've left that piece of code out of the preceding listing, but it's pretty straightforward.

Tip

A CD's universal product code (UPC) is cut onto the CD by the manufacturer, and is guaranteed unique in the entire universe. It's sort of like the CD equivalent of the Zebra bar codes on your favorite items at the grocery store. For example, you can use the CD's UPC to query a CDDB (CD Database) on the Internet for the title, artist, and track names of the CD. Beware, however: some older CDs don't have a product code.

Unlike the CD product code, the CD ID code is a makeshift identifier, calculated by Windows based on the track lengths and other attributes of the CD. Windows can generate an ID for any CD, but it's possible (though unlikely) that two different CDs will have the same CDID.

Once the code has finished querying the device, it closes it by sending it a MCI_CLOSE command. Notice the MCI_WAIT flag. MCI commands are by default asynchronous; specifying MCI_WAIT tells MCI to not return until the device has processed the message. The code plays it safe by waiting for the close command to complete before adding the device info class to the vector, and moving on to the next drive letter.

This static function returns a vector of CCDDeviceInfo classes to the caller. Once the caller has the list of valid drive letters, it picks one, and calls CCDPlayer's Init method. I haven't included the Init source code here because it's straightforward. Init opens the device corresponding to the supplied drive letter, and stashes its device ID into m_DeviceID. This device ID is like a file handle; the code receives it in response to the open message, and uses it to specify the target for more interesting messages, like MCI_PLAY. This is how, in a multiple CD audio drive system, you can get the right disc to spin up.

Once the device is open, the Init method sends an MCI_SET message to set the time format to something called TMSF, which stands for track, minute, second, frame—four identifiers which you use together to pinpoint an exact location on an audio CD. There are other time formats (for example, plain old milliseconds), but TMSF is your best choice for situations where you're mainly playing entire tracks—you'll see why when you learn about playing tracks.

Finally, the Init method sets the m_InitGood flag to true. Yep, you guessed it—two-phase construction again. This flag is the "seal of approval" that the CD player is ready; all of the other methods of CCDPlayer check this flag before attempting to do anything else.

When you're all done using the CD player class, call its UnInit method, which closes the device by sending an MCI_CLOSE message.

MCI Time Formats

Time to take a quick detour and talk about CD time formats. I've written a helper class called CTrackMinSecFrame which contains integers for track, minute, second, and frame. However, the MCI functions pack all four of these values into a single DWORD, one byte for each value. MCI supplies macros to pack and unpack a TMSF DWORD; you can use these macros to create methods that convert between CTrackMinSecFrame classes and DWORDs:

class CTrackMinSecFrame
{
public:
  // snip
  CTrackMinSecFrame &operator=(DWORD dw) {
    m_Track = MCI_TMSF_TRACK(dw);
    m_Minute = MCI_TMSF_MINUTE(dw);
    m_Second = MCI_TMSF_SECOND(dw);
    m_Frame = MCI_TMSF_FRAME(dw);
    return(*this);
  }
  operator DWORD() { return(MCI_MAKE_TMSF(m_Track, m_Minute, m_Second, m_Frame)); }
};

Here are two nifty operator overloads. The first, operator=, uses four MCI macros to extract the individual track, minute, second, and frame bytes, and put them in their new integer homes. The second method, operator DWORD, uses the MCI_MAKE_TMSF to pack them back up.

These operators are cool because together they allow us to treat a CTrackMinSecFrame as if it were a DWORD—anywhere! The C++ lesson here is that when used to automate banal-type conversions, operator overloading can make your life easier.

Playing a Track

The bad news is that you can't just tell the CD player to "play track 8." You have to give it start and end times. The good news is that because you're using the TMSF format, this is pretty easy:

void CCDPlayer::Play(int track)
{
  if (!m_InitGood) return;
  m_Paused = false;
  MCIERROR e;
  MCI_PLAY_PARMS mciPlayParms;
  mciPlayParms.dwFrom = 0L;
  mciPlayParms.dwTo = 0L;
  mciPlayParms.dwFrom = MCI_MAKE_TMSF(track, 0, 0, 0);
  mciPlayParms.dwTo = MCI_MAKE_TMSF(track + 1, 0, 0, 0);
  mciPlayParms.dwCallback = (DWORD)m_hWnd;
  e = mciSendCommand(m_DeviceID, MCI_PLAY,
    MCI_FROM | MCI_TO | MCI_NOTIFY,
    (DWORD)(LPVOID) &mciPlayParms);
  if (e) HandleError(e);
}

By using TMSF, the code is able to easily specify the playback range—it's from the start of the track we want (track, 0, 0, 0), up to but not including the start of the next track (track+1, 0, 0, 0). TMSF is beautiful here—without it, you'd have to calculate the millisecond range you needed.

Also important in the preceding code is the MCI_NOTIFY flag. You'll learn about that in the Synchronous versus Asynchronous Commands section, in a few pages.

Stopping

To stop a CD from playing, send it an MCI_STOP message:

void CCDPlayer::Stop()
{
  m_Paused = false;
  if (!m_InitGood) return;
  MCI_GENERIC_PARMS genericparms;
  ZeroMemory(&genericparms, sizeof(MCI_GENERIC_PARMS));
  mciSendCommand(m_DeviceID, MCI_STOP, MCI_WAIT, (DWORD)(LPVOID)&genericparms);
}

The messages that take only generic parameters are the easiest. Here, the code sends a MCI_STOP command and supplies a generic parameter structure.

Notice the MCI_WAIT flag, which ensures that the CD player is stopped by the time the Stop method returns.

Pausing and Resuming

Notice that the first thing the Play method does is set the m_Paused flag to false. It may surprise you to learn that MCI doesn't support pause and resume for all CD players. To make a rock-solid CD player class, you'll have to implement pause and resume yourself.

Fortunately, this isn't difficult—to pause, all you have to do is query the current play position, remember it, then send a stop command. To resume, just start playing from where you paused until the end of the track. Add a flag to remember whether or not the CD player is paused, and you're all set.

Implementing this form of "pseudo-pausing" requires a couple of additional member variables. First, there's m_Paused, the Boolean flag. Next, there's m_PausedPos, a CTrackMinSecFrame that stores the exact instant the CD was paused.

Here's how it looks in code:

void CCDPlayer::Pause()
{
  if (!m_InitGood) return;
  m_PausedPos = GetCurrentPos();
  Stop();
  m_Paused = true;
}
void CCDPlayer::Resume()
{
  if (!m_InitGood) return;
  if (!m_Paused) return;
  // play from pause position to end of track
  MCIERROR e;
  MCI_PLAY_PARMS mciPlayParms;

  mciPlayParms.dwFrom = 0L;
  mciPlayParms.dwTo = 0L;
  mciPlayParms.dwFrom = m_PausedPos;

  mciPlayParms.dwTo = MCI_MAKE_TMSF(m_PausedPos.m_Track + 1, 0, 0, 0);
  mciPlayParms.dwCallback = (DWORD)m_hWnd;
  e = mciSendCommand(m_DeviceID, MCI_PLAY,
    MCI_FROM | MCI_TO | MCI_NOTIFY,
    (DWORD)(LPVOID) &mciPlayParms);
  if (e) HandleError(e);
  m_Paused = false;
}

The Pause method is easy (you'll learn about GetCurrentPos in the next section), and the only thing tricky about the Resume method is the play range. The code sets up the MCI_PLAY_PARMS structure so that the CD player starts up where it left off (m_PausedPos), and then continues up to the start of the next track (m_PausedPos.m_Track+1, 0, 0, 0). Finally it sets the m_Paused flag to false, completing the illusion of pause.

Tip

The CD player's Play method isn't terribly intelligent. Play doesn't automatically check the m_Paused flag; it unconditionally starts playing from the beginning of a track. The CD Player class relies on other code to keep track of the pause state and either call Play or Resume, as appropriate.

You'll see a couple of other methods that set the paused flag to false. For example, the method that ejects the CD also sets m_Paused to false.

Determining Play Position

To ask a CD player what it's currently playing, send a MCI_STATUS message. MCI_STATUS lets you do a whole bunch of useful status queries—look at all of the flags in MSDN—but to get the current play position, you need to pass in MCI_STATUS_POSITION as the dwItem in the status parameters structure, MCI_STATUS_PARMS.

Here's the code:

CTrackMinSecFrame CCDPlayer::GetCurrentPos()
{
  CTrackMinSecFrame pos;
  if (!m_InitGood) return(pos);
  if (m_Paused) return(m_PausedPos);
  if (!IsPlaying()) return(pos);
  MCI_STATUS_PARMS statusparms;
  ZeroMemory(&statusparms, sizeof(MCI_STATUS_PARMS));

  statusparms.dwItem = MCI_STATUS_POSITION;
  MCIERROR e = mciSendCommand(m_DeviceID,MCI_STATUS,MCI_STATUS_ITEM
    | MCI_WAIT,
    (DWORD)(LPVOID)&statusparms);
  if (e) HandleError(e);
  pos = statusparms.dwReturn;
  return(pos);
}

Sometimes, knowing where to put an MCI flag is difficult. In the case of status messages, I've found the most straightforward approach is to specify the MCI_STATUS_ITEM flag, which tells MCI to look at the dwItem member of the MCI_STATUS_PARMS structure to figure out which status item you want. MCI then puts the information you've requested into the dwReturn member, which the preceding code looks at, converts into a CTrackMinSecFrame (there's that nifty assignment operator overload!), and returns.

Ejecting a CD

As Figure 9.2 illustrates, writing code that makes things move is neat, so writing code to get the CD player to stick its tongue out is thrilling. Okay, maybe only if you're in severe need of a life, but still—tray ejection code can be useful. Here's how to do it:

Click To expand
Figure 9.2: The effects of sending an MCI_SET_DOOR_OPEN command.
void CCDPlayer::OpenCDDoor()
{
  if (!m_InitGood) return;
  m_Paused = false;
  mciSendCommand(m_DeviceID,MCI_SET,MCI_SET_DOOR_OPEN,0);
}

It doesn't get much easier than that. Just send a MCI_SET message, specify the MCI_SET_DOOR_OPEN flag, and you're done—no need to pass along a structure.

Closing the door is just as easy:

void CCDPlayer::CloseCDDoor()
{
  if (!m_InitGood) return;
  mciSendCommand(m_DeviceID,MCI_SET,MCI_SET_DOOR_CLOSED,0);
}

Here, the MCI_SET_DOOR_CLOSED flag tells the CD audio device to suck its tray back in. If the tray's not sticking out, no harm is done.

Detecting When a CD Has Changed

There are two parts to accurate CD change detection. The first part involves querying the device to see if there's a CD in the drive. Here's that code:

bool CCDPlayer::IsCDInDrive()
{
  if (!m_InitGood) return(false);
  MCI_STATUS_PARMS statusparms;
  ZeroMemory(&statusparms, sizeof(MCI_STATUS_PARMS));

  statusparms.dwItem = MCI_STATUS_MEDIA_PRESENT;
  MCIERROR e = mciSendCommand(m_DeviceID,MCI_STATUS,MCI_STATUS_ITEM |
    MCI_WAIT,
    (DWORD)(LPVOID)&statusparms);
  if (e) HandleError(e); return(statusparms.dwReturn == TRUE);
}

Fairly straightforward code—it sends a MCI_STATUS message asking about MCI_STATUS_MEDIA_PRESENT, and gets a TRUE back if there is indeed a CD in the drive.

The next, and slightly trickier, part is knowing when a CD has changed. You don't want to poll MCI_STATUS_MEDIA_PRESENT continuously. To properly detect a CD change, you need to listen for the WM_DEVICECHANGE Windows message. Windows will send this message to the top-level HWND of all programs when a CD is removed or inserted.

Tip

Technically, the WM_DEVICECHANGE message means that the hardware configuration has changed, so you'll also get this message when you start plugging and unplugging USB and firewire devices.

To properly interpret this WM_DEVICECHANGE message, you'll need to look at its wParam. If a CD has been pulled out, the wParam will contain DBT_DEVICEREMOVECOMPLETE (that's "device remove complete"). When a new CD is put in, you'll see DBT_DEVICEARRIVAL. For more information, consult the MSDN docs for WM_DEVICECHANGE.

Determining What's on a CD

You have almost attained complete mastery over the CD. One thing remains—how do you figure out what's on a CD? Sure, you know how to get a CD's product code and generated ID, but how do you know how many tracks there are?

Bonus points if you guessed that it would involve the MCI_STATUS message. Here's the code:

int CCDPlayer::GetNumTracks()
{
  if (!m_InitGood) return(false);
  MCI_STATUS_PARMS statusparms;
  ZeroMemory(&statusparms, sizeof(MCI_STATUS_PARMS));

  statusparms.dwItem = MCI_STATUS_NUMBER_OF_TRACKS;
  MCIERROR e = mciSendCommand(m_DeviceID,MCI_STATUS,MCI_STATUS_ITEM |
  MCI_WAIT,
    (DWORD)(LPVOID)&statusparms);
  if (e) HandleError(e);
  return(statusparms.dwReturn);
}
Tip

There are many more status commands than what I've covered here. Look at the MSDN docs if you want to learn how to do things like query each track's length or track type (data or audio).

MCI provides a special type of status query, MCI_STATUS_NUMBER_OF_TRACKS, that gets the job done nicely. It puts the number of tracks into the dwReturn member of the MCI_STATUS_PARMS structure you gave it.

Synchronous versus Asynchronous Command Execution

One last topic—back when describing the Play method, I mentioned the MCI_NOTIFY flag. You can use this flag on any MCI message, and it's vital because it tells MCI that you'd like to be notified when whatever you've requested is complete. Tack it onto a MCI_PLAY message and MCI will notify you when the playing is complete, that is, when the CD hits the end of what you've told it to play.

MCI notifies you through a MM_MCINOTIFY message passed to your main window. You can identify what specifically happened by looking at the wParam of the MM_MCINOTIFY message, which will either be MCI_NOTIFY_ABORTED, MCI_NOTIFY_FAILURE, MCI_NOTIFY_SUCCESSFUL, or MCI_NOTIFY_SUPERSEDED. The lParam contains the device ID you sent the message to.

Using MM_MCINOTIFY, you can loop tracks indefinitely, shuffle a CD, and do all sorts of other cool things. Just take the appropriate action when you get a MM_MCINOTIFY message with your CD player's device ID in the lParam and MCI_NOTIFY_SUCCESSFUL in the wParam.

On the other hand, if you want a command to execute synchronously—that is, you want mciSendCommand to wait until the command's complete before returning—you can specify the MCI_WAIT flag. MCI_WAIT allows you to play it safe and know for sure that a message has been completely processed—this is useful if you're sending a bunch of messages in a row.

Check out Figure 9.3 for a comparison of synchronous and asynchronous command execution, and for more information, consult MM_MCINOTIFY in MSDN.

Click To expand
Figure 9.3: Synchronous versus asynchronous command execution.

Controlling CD Audio Volume

You can control the volume of the CD player by using the Mixer API, described in Chapter 5. I thought I'd mention it just in case you missed it back then—simply provide the correct line settings (you can use the pre-built CDAudio constant in CMixer), and set the volume as you wish.


THE CD PLAYER SAMPLE PROGRAM

This chapter's sample program is—you guessed it—a CD player. Using the CCDPlayer class, this simple Win32 GUI application allows you to play (and even pause!) your favorite CDs. You can also jump between tracks and select different CD devices to play from. Check out Figure 9.4 for a screenshot.

Click To expand
Figure 9.4: Screenshot of this sample's chapter program in action.

Most of the code in [image] Ch9p1_CDPlayer.cpp is Win32 GUI code. I didn't use any foundation classes—this is bare-bones Win32 API code. As such, it's a bit difficult to snake through, so I thought I'd explain some of the more interesting bits of its operation.

The CD player class itself is a global variable, g_CDPlayer. When the main dialog is initialized, it calls the PutAvailableDevicesInComboBox function, which in turn calls the static CCDPlayer::GetAvailableDevices method. This method goes through the drive letter alphabet and returns the device info array, which the program uses to populate the combo box.

The CD player class is initialized in response to a combo box selection change message (CBN_SELCHANGE). The OnComboBoxChange function re-initializes the CD player class, passing in the drive letter the user selected. Every combo box item has 32 bits of storage space—the sample program uses eight of these bits to hold the char corresponding to the drive letter, so all it has to do is get these eight bits from whichever combo box item is currently selected, and then pass that drive letter off to the CD player's Init method.

There's a timer that fires every second, which calls RefreshGUI. This function does a few things— first, it either enables or disables all of the control buttons, depending on whether there is a CD in the selected drive. Second, it queries the CD player class for the track position, which it displays in the IDC_LCD label. It also sets the caption on the play button to "Play," "Pause," or "Resume," as appropriate.

I've included handlers for both WM_DEVICECHANGED and MM_MCINOTIFY messages. The device changed message handler just calls RefreshGUI so that the controls get enabled when a CD is inserted, and disabled when a CD is removed. MM_MCINOTIFY moves to the next track on the CD. Feel free to add some repeat or shuffle code to this handler.

Finally, notice that the entire dialog procedure is wrapped in a try block. If the CD player class encounters any MCI errors, it calls its protected HandleError method, which in turn throws a string. The dialog proc catches the string and displays it as a message box to the user. In a real game you probably would want to do something else, but for a simple CD player app this is fine.

In summary, if you haven't done any bare-bones Win32 API programming before, learning this program will feel like drinking from a fire hose. You might want to grab a book on the essentials, such as the excellent Windows Programming for Dummies or any other of several books on the Windows API.


CONCLUSION

It's humbling to stop and think of just how much work the MCI is doing for you. You don't have to worry about a CD's file system (which is nightmarishly complex), you don't have to stream, and you don't have to error check—all you do is hit a virtual play button by sending a MCI_PLAY message.

Of course, you can dive deeper, and there are many freeware and open-source programs from which to learn. In particular, SourceForge (http://www.sourceforge.net) has some useful opensource code for interfacing to an Internet-based CDDB service (to retrieve album title, artist, and track names for a particular CD). It also has open-source code for extracting digital audio using low-level ASPI calls. I've included a couple of tasty links on your CD.

This CD player class, however, should do everything you need for your game. Most games just want to be able to play a certain track at a certain time, and loop it, and this class, sitting atop the Media Control Interface, makes those tasks easy.


Chapter 10: Tracked Music Playback

Download CD Content

OVERVIEW

Long before there was MP3, Ogg Vorbis, and Windows Media Audio, there was tracked music (see Figures 10.1 and 10.2). Tracked music blipped onto the game radar in the early nineties and has remained a very popular way to implement game music, especially for games distributed online. That's because it hits a sweet spot between quality of music and the size of that music on disk. Whereas a typical MP3 file will run you 3-4MB, tracked music requires usually one tenth that amount, and still sounds great.

Click To expand
Figure 10.1: A screenshot of Impulse Tracker, a popular tracked music editor written by Jeffrey Lim.
Click To expand
Figure 10.2: A screenshot of ScreamTracker, another popular tracked music editor, written by Sami Tammilehto.

In this chapter, you'll learn how tracked music works, and how to incorporate it into the audio engine using the MikMod API, a GNU GPL library that makes loading and playing tracked music easy.


INTRODUCTION TO TRACKED MUSIC

In the beginning, most video games used FM synthesis for music. Recall that FM synth generates tones through mathematical formulas—you say you want a sine wave, modulated with another wave, with some mathematical effects or some noise added on for fun. This leads to a wide variety of sounds, but unfortunately, a wide variety of sounds that all sound like a synthesizer. Over the years many valiant attempts were made to represent mathematically the sounds of various instruments. For some instruments (like the flute) this worked well, but for most instruments, the results were less than spectacular, and for a few critical instruments (brass instruments and drums in particular), the results were abysmal.

Tracked music is essentially FM synth on steroids. Instead of using mathematical formulas to represent instrument sounds, the sounds of the instruments themselves are digitized and stored in the song, and the CPU uses math to raise or lower the pitch of a sample to the appropriate note. Now, instruments sounded reasonably close to their real-life counterparts. The sound quality of any one instrument still wasn't perfect, but it was good enough, and when you combined several voices of these instruments you could actually come out with a pretty decent song.

Tracked music does, on a more basic level, what DirectMusic does. The DirectMusic synthesizer allows you to use your own sample sets (called bands) for instruments, and provides more options for using samples (for example, you can have several samples of one instrument, each representing a note played at a different pitch or intensity).


USING MIKMOD FOR TRACKED MUSIC

In this chapter, you'll learn how to use MikMod to add tracked music playback abilities to the audio engine. However, before you get to the code, you must first learn how to install MikMod.

Installing MikMod

Here's how to get MikMod up and running on your system. Note that I've already done these steps for you for this chapter's sample program; they only apply when you're working on your own code.

  1. Inside your Goodies folder on the CD is a MikMod folder which contains the MikMod archive, libmikmod-3.1.10 (version 3.1.10 was the latest as of this writing). Unzip this file into a folder of your choosing.

  2. Open a command window; navigate to the win32 folder underneath your main MikMod folder.

  3. If you want to use the static version of the library, type nmake –f makefile.static. Or, if you want to use the dynamic version (which uses [image] mikmod.dll), type nmake –f makefile.dynamic. Note that both of these makefiles output to mikmod.lib, so be careful.

  4. If you don't have mikmod.h inside the MikMod include directory, copy win32\mikmod_build.h to include\mikmod.h in the include directory. The make file is supposed to do this for you, but on my system it didn't.

To use the MikMod library in your own applications, follow these steps:

  1. Select Settings from the Project pull-down menu inside DevStudio.

  2. Add the mikmod include directory to your project include path. You do this by going to the C/C++ tab, then selecting Preprocessor. Add a relative path to the MikMod include directory. For this chapter's sample program, that would be ..\..\libmikmod-3.1.10\include.

  3. Add the mikmod library directory (which is actually the win32 directory) to your project's library path. To do this, go to the Link tab, select Input as the category, and add the relative path into the edit box labeled Additional library path. For this chapter's sample program, I entered ..\..\libmikmod-3.1.10\win32.

  4. Add the MikMod library to your link libraries by adding mikmod.lib to the edit box labeled object/library modules.

  5. Don't forget to include mikmod.h in whatever source files use MikMod.

MikMod Help

The MikMod API reference is contained in HTML documentation, inside your docs folder. Launch the mikmod.html file, which contains all of the documentation.

Using MikMod in the Audio Engine

To integrate MikMod into the audio engine requires the creation of a new class, as well as some changes to the CAudioManager class. The following sections discuss each in more detail.

CTrackedMusic

The new CTrackedMusic class, derived from CSound like every other form of music or audio class in the engine, represents a tracked music object. Here's the class declaration:

class CTrackedMusic : public CSound
{
public:
  friend class CAudioManager;
  CTrackedMusic(CAudioManager *mgr);
  virtual ~CTrackedMusic();

  virtual bool Play();
  virtual bool Stop();
  virtual bool IsPlaying();
  std::string GetFileName() { return(m_FileName); }
  static void FreeAllModules();
protected:
  static std::vector<CTrackedMusic *> m_AllTrackedMusicInstances;
  std::string m_FileName;
  MODULE *m_Module;
};

At the core of this class is a pointer to a MODULE: m_Module. MODULE is a data type that MikMod uses; essentially, it's a handle to a tracked music song. You get a MODULE address by calling the Player_Load API call.

But I'm getting ahead of myself. There are two other members in CTrackedMusic. One is a vector of CTrackedMusic pointers. This vector keeps track of all instantiated tracked music classes—in the CTrackedMusic constructor, the code pushes its this pointer onto the vector, and in the destructor, it finds and removes its this pointer from the array. The engine uses this list to free all of the modules before it shuts down MikMod. Shutting down MikMod without freeing the modules is sloppy, so this mechanism is in place to ensure that everything gets cleaned up when the time comes.

The other member is m_FileName, which contains the filename the CTrackedMusic class was created from. There's no dire need to have it, but it's handy reference material.

The three mandatory methods, Play, Stop, and IsPlaying, are all one liners, thanks to the MikMod API. For Play, the code just calls Player_Start; for Stop, it calls Player_Stop; and for IsPlaying, it calls Player_Active.

The FreeAllModules method loops through the m_AllTrackedMusicInstances vector, calls Stop on all of the valid modules, and then frees their MODULE_ pointers via a call to Player_Free.

CAudioManager

Now that you know everything there is to know about the CTrackedMusic class, here's the other half of the story. To support MikMod requires a couple of subtle changes in CAudioManager.

For starters, there's some new code added to the Init method:

MikMod_RegisterAllDrivers();
MikMod_RegisterAllLoaders();
MikMod_Init("");

All three of these functions are requirements for setting up the MikMod library. The MikMod_RegisterAllDrivers call chooses an appropriate output driver—on Windows, this is either the driver for windows multimedia audio or the DirectX 5.0 compatible driver. Yes, this means that you can use MikMod on an older OS (say, Windows NT) that doesn't support DirectX 5.0.

The MikMod_RegisterAllLoaders function figures out which file formats (IT, S3M, MOD, and so on) the library can load. Be careful—if you forget to call this function at the beginning of your programs, you won't be able to load anything—you'll just keep getting NULL return values from Player_Load.

Finally, the call to MikMod_Init fires everything up and tells whatever driver MikMod_RegisterAllDrivers picked out to initialize and start up. The character string is where you put any options you want to pass to the driver—usually you don't need to pass anything, so just leave this as an empty string.

Usually, new code in the initialization method means new code in the un-initialization method, and this time is no exception. Here are the new lines:

CTrackedMusic::FreeAllModules();
Player_Stop();
MikMod_Exit();

The first call causes all of the CTrackedMusic instances to free their MODULE pointers. Next, UnInit calls Player_Stop in case something's still playing, then calls MikMod_Exit to shut everything down. As you can see, dealing with MikMod is pretty straightforward.


CONCLUSION

MikMod made this chapter pretty easy, and actually, I didn't talk about some of MikMod's functionality. For example, there are API commands to change the tempo and volume of a song; check the MikMod documentation (mikmod.html inside your docs folder) for details.

Also, I encourage you to look at the MikMod source code to determine what's really going on, and what your computer is really doing when you tell it to play a tracked music file.

This chapter is the last chapter in Part One of this book, which means that you've now implemented all of the basic functionality of the audio engine. Pat yourself on the back, eat a piece of cake, buy a new piece of hardware, or do whatever you do to reward yourself—you've earned it!

In the next part of the book, you'll learn how to really make the audio engine shine by incorporating some advanced techniques, such as 3D sound, audio scripting, and effects buffers. Don't stop now, you're halfway there!


Part Two: Advanced Audio Engine Functionality

Chapter 11: Dynamic Music
Chapter 12: Scripts and Effects
Chapter 13: 3D Sound Using DirectX Audio
Chapter 14: 3D Sound Using OpenAL
Chapter 15: Advanced Topics in 3D Sound
Chapter 16: DirectPlay Voice
Chapter 17: Audio Visualization

Welcome to the advanced section of the book. Now that you have worked through the first chapters, you're ready to extend the audio engine beyond just the basics of sound effects and music playback.

In this section, you'll learn about several advanced techniques to increase the quality of your game's audio. You'll learn how to create and play back dynamic music—that is, music that changes in real time. Dynamic music allows you to change the music in response to game events.

You'll also look at audio scripting, a feature of DirectX Audio that allows you to give full control over how a sound effect or piece of music sounds to your sound designers. No more, "Hey, I made this new sound effect, can you put it in the game for me?" questions—with audio scripting the sound designers can add, delete, and change sound effects without having to recompile your game's code.

Later in this part, you'll get a peek at 3D Audio, a fascinating technology that can trick your brain into thinking that a sound is coming from anywhere in 3D space. Using 3D Audio, when the ogre is sneaking up behind your player, its breathing and footsteps will appear to actually come from behind them, significantly adding to the sense of reality and immersion in your game.


Chapter 11: Dynamic Music

Download CD Content

OVERVIEW

Dynamic music ranks up there with 3D sound as one of the most exciting revolutions in video-game audio technologies today. For decades, video game music has been limited to short musical snippets, repeated often enough to resemble torture. I feel bad for my parents, who were forced to listen to the same two minute, 8-bit Nintendo soundtracks day-in, day-out for weeks over my summer breaks in elementary school.

Computers today have enough power to assemble complex soundtracks in real-time, and in response to game events. They can whimsically shift the music to another key or subtly morph it slowly into something completely different, enhancing the player's immersion in the game. All this assuming, of course, that the composer and the programmer work closely together to create the right parts for the computer to put together.

This chapter and Appendix A go hand in hand. This chapter contains the vocabulary, and the explanation of how to implement dynamic music in code. Appendix A contains examples of how to write the music itself. To minimize confusion, you should start by reading the first section of this chapter (Elements of Dynamic Music). Next, put on your musician hat and jump to Appendix A. Finally, put your programming hat on and jump back here and learn how to control, in code, the dynamic music you created in Appendix A.


ELEMENTS OF DYNAMIC MUSIC

The last several chapters have given you a good taste of DirectMusic, but they haven't really taught you anything beyond surface functionality. You know what a segment is, and how to load and play one, but you really don't know anything about what's inside a segment.

Prepare for a massive vocabulary lesson. This section teaches you the names of all the DirectMusic objects you need to know about.

Don't worry if you feel overwhelmed—there's a lot of new stuff here. If you read it over a couple times and it still isn't making sense, read Appendix A, which gives a hands-on demonstration of all these different objects inside DirectMusic producer.

DLS collections

Without instruments, nothing else matters. A DirectMusic DLS collection is a set of all the digital samples of instruments you will be using in your music. When I begin a new piece of music, the first thing I do is pick out the kinds of instruments I want, and add those to the song's DLS collection.

You should realize that it's entirely possible to create a score without using a DLS collection. Every DirectMusic piece can use all 127 sounds of the General MIDI instrument set; DLS collections are only for additional instruments that you may want to include. Also, realize that the sound of the General MIDI instrument set is standardized; what you hear is what your game player will hear.

If you do need a custom sample, however, you'll be pleased at the incredible amount of flexibility the DLS standard gives you. Just to whet your appetite: you can supply multiple samples, one for a different range of notes for the instrument, and you can adjust the attack, decay, sustain, and release (ADSR) of the instrument (to learn more, check out Appendix A).

Notes

When talking about dynamic music, it's important to remember that most of the notes you input aren't absolute—they're relative to the current chord and key that DirectMusic has decided to play your music in. This is very different than notes in MIDI, which are absolute values.

A note contains the following properties:

  • Pitch. This is the obvious one. Remember, it's relative.

  • Velocity. This is the volume of the note. Velocities range from 0 (no sound) to 127 (full volume).

  • Start Time. The time the note starts. This is specified in terms of bar, beat, grid, and tick, giving you very fine control. In 4/4 time, each beat is divided into 4 grids. Quarter notes start on tick zero of grid zero of a given beat in a given bar. Eighth and sixteenth notes start on different grid numbers.

  • End Time. The time the note ends. Again, specified in bar, beat, grid, and tick.

  • Duration. How long the note plays. Duration is always end time minus start time.

Tip

In DirectMusic, a note doesn't always play at exactly the right time. This may sound appalling, but actually it's a great feature. DirectMusic automatically introduces very small, random delays to note hit times, to simulate a human musician. Think about it—even the best musicians can't hit every note at exactly the right time. They're going to be a few milliseconds early or late. DirectMusic does the same thing, so that the music doesn't sound too "perfect" and mechanical.

Notes also contain other advanced properties which are only occasionally useful, like Inversion Group overrides and playmodes. Refer to the DirectMusic documentation for information on these properties.

Tracks

Notes aren't the only things that are organized into tracks. Sure, inside a pattern, you have a note track, a piano roll that tells DirectMusic what notes to play. But different types of tracks contain different types of information, just as valuable as the notes. For example, a style track can tell DirectMusic that it needs to play one style until measure 13, then switch to a different style. Similarly, a band track can tell DirectMusic when it's time to change the band, and a chord track can tell what chords go well with the music being played.

Parts

A part is a collection of notes played by the same instrument. If you were writing a symphony, you'd have different parts for violin, tuba, banjo, and so on. It's no different in DirectMusic.

Specifically, a DirectMusic part contains three different tracks. The note track, plus a Var Switch Point track, and a CC (Control Curve) track. The Var Switch Point track tells DirectMusic when it's safe to move to another variation within that track (you'll learn more about this as you read the rest of this chapter, and Appendix A). The Control Curve track contains your pitch bends, foot petal movements, and modulation adjustments, captured from your MIDI keyboard while you were playing.

Performance Channels (PChannels)

Think of a performance channel as the link between a part and an audio path. You learned a few chapters ago that you create audio paths for each instance of a sound that you want to play. The audio path controls the sound instance's volume, panning, and ultimately the output device that the sound comes out of.

Every part (see the previous section) has a PChannel that it outputs to (see Figure 11.1). The PChannel specifies the volume and panning, as well as the band instrument used to play the note. You can also specify the parameters each PChannel uses to create its audio path from within DirectMusic Producer. For example, you could make a certain PChannel use reverb; any parts you played on that PChannel would then reverberate.

Click To expand
Figure 11.1: Illustrates the relationship between parts, PChannels, and audio paths in DirectMusic.

A PChannel can be shared by several parts. For example, if you have four violin parts that you want played using the same violin instrument at the same volume and panning, you can tell them all to use the same PChannel. You won't run out of PChannels—up to 999 are available.

Patterns

Patterns are the core objects around which all other DirectMusic objects revolve. Patterns contain multiple parts, and can be any length that you want (usually a few measures long, but sometimes longer or shorter depending on the structure of the music and how quickly you want it to change).

Patterns are to DirectMusic what meters are to physics and grams are to chemistry. In other words, patterns are the "comfortable" unit in which to work. If you've written tracked music before, you are already familiar with organizing your music into patterns.

Tip

Most of the time you'll want your patterns to loop smoothly. The exceptions to this rule are the embellishment patterns (explained in a few sections).

Variations

Each pattern can contain 32 variations. A variation is exactly what you think it is—a subtle difference in a part that's played at random by DirectMusic. This allows you to keep the music fresh. If you handed a pattern to a human musician and said, "Here, play this over and over," the musician would eventually get bored, and as he got familiar with the music, he'd start adding variations to make things more interesting—an extra note here, a syncopation or arpeggio there, and so on. You can have DirectMusic do the same thing, only because it's not a real human, it can't add that personal touch on its own. You have to provide the variations.

Note that a variation is not supposed to be a completely different take on a pattern (though it can be). DirectMusic has a lot of places where you can give it a lot of alternatives and tell it to pick one at random. You should use variations for subtle alternatives only; if you have two completely separate melodies, you should put them in different patterns or perhaps even different styles—"higher up" in the DirectMusic object structure (keep reading).

Motifs

Think of motifs as "special patterns." Motifs are brief musical hooks played when something interesting happens. They're never chosen randomly by DirectMusic; instead, you specify when to play one. For example, you might have a little powerup ditty that you play over (or instead of) the main melody when your player grabs a powerup.

Proper use of motifs can really make dynamic music sound great. One of the easiest ways to determine when to play a motif is to watch movies. Notice in a horror movie how the music swells up right before the zombie ambushes the heroine, or how the music crescendos as the spaceship narrowly escapes from the exploding station. Of course, movies have it easy, because the composer knows exactly when something interesting is going to happen, and can structure the music around that point in time. In a video game, the player is in control, and unpredictable, so the motifs have to be ready to play on a moment's notice.

Embellishments

Embellishments are also special patterns. Embellishments are used when the music makes a significant change, as you can see by the following list of standard embellishments:

  • Intro. Played at the start of the dynamic music.

  • Fill. A short embellishment used to transition to something different, for example, from a verse to a chorus.

  • Break. Similar to a fill, but in a different form. A break is when one or more parts temporarily drop out to emphasize something else—maybe the rhythm and background chords cut out to make room for a blazing guitar solo, or maybe everything melodic cuts out for a drum solo.

  • End. A quick wrap up to the dynamic music.

At its most basic, an embellishment is a pattern with a special flag on it, telling DirectMusic what type of embellishment it is. In addition to the standard types listed above, you can set up custom embellishment types (up to 100) if you have a specific event that calls for something a little heavier than just a motif. The difference between a motif and a custom embellishment is that a motif is usually played on top of the main music, whereas a custom embellishment diverts the entire pattern, allowing for more drastic movement.

The DirectMusic brain will, if you tell it to, automatically play embellishments when it makes sense (for example, when a piece is starting, ending, or when a key change occurs, and so on).

Bands

Bands are the instruments used by a style. A band is a subset of your music's DLS collection (see Figure 11.2). Essentially, the band object tells DirectMusic which DLS instrument to use, as well as its default panning position, default volume, and performance channel. When your pattern says "play D sharp using instrument number three," DirectMusic looks at the current band to figure out what instrument three is.

Click To expand
Figure 11.2: You choose the instruments in your band from your DLS collections and the GM sound set.

At first it might seem useless to have this extra level of indirection for your instruments, but you'll come to find that bands are a powerful feature because they allow you to easily "swap out" your set of instruments. Each style can have more than one band, and you can switch them easily in code, which can make your music more interesting and better suited to respond to game events.

Styles

A style is a group of patterns, embellishments, motifs, and bands. In other words, a style is a big chunk of music. Think of a style as a blind, unorganized manila folder full of sheet music— there's a song in there somewhere, but you need something else (namely a segment) to organize everything.

Programmers can think of styles as akin to individual CPP files in a program. Just as the code for a complex program is organized into multiple CPP files, the music for a complex piece of dynamic music is organized into multiple styles.

Segments

A segment is the grand unifier for all of your styles. Tracks contained in the segment object tell DirectMusic what style to play when, and what patterns to play from that style (see Figure 11.3).

Click To expand
Figure 11.3: The style and groove level tracks tell DirectMusic what pattern to play next.

Two of the most important tracks in a segment are the style track, which points to a specific style, and the groove level track (see the next section), which helps DirectMusic decide what patterns within that style to play next. Segments can also contain band tracks (so you can play the same pattern with different instruments), tempo tracks, and several less common tracks to influence the patterns in different ways.

Most of the time, a given level or phase of your game will have only one segment. If you're making an action game with different levels, the most common approach is to have one segment for each level. These segments may share patterns, bands, and other objects, but when it's time to load level 1, your code will reach for the level 1 segment file (level1.sgt).

In this situation you probably won't need to worry about transitioning between segments. You'll start up level1.sgt when the level loads, and tell it to smoothly stop (perhaps using an End embellishment) when the player completes the level.

If your game isn't so cleanly divided, you may need to perform segment transitions. For example, in a space combat game, you might choose to create different segments for different emotional states. When your player starts the game, the "idle, looking around" segment will play. When the player encounters enemies, you may transition to the combat segment, and when a boss comes in, you may transition from combat to boss, and then from boss back to idle when the player beats the boss. Or, for a multiplayer death-match game, you might have different segments for when your player is winning or losing.

As you can see, there are endless different ways to slice up your music. I like to apply a rule that states that each segment is a self-contained "chunk" of music, able to stand on its own and generate enough variation to keep things interesting. Of course, it's possible to put all of your game music into one segment, and it's possible to put every measure of your music into a separate segment, but both of these strategies are too extreme.

Grooves and Groove Levels

Each pattern you create has a groove range, specified by a minimum and a maximum value, both of which must be between 1 and 100. By default, a pattern's groove range is from 1–100 (the widest possible), but you can change that to be any range (for example, 10 to 20), or even a single number.

DirectMusic uses this groove range to determine which pattern to play. Each segment contains a groove track that tells DirectMusic the current groove level, and therefore which patterns it can play. For example, if the current groove level is 10, and you have two patterns—Pattern A with a groove range of 1–5, and Pattern B with a groove range of 8–14, DirectMusic will choose Pattern B to play because its groove range falls within the current groove.

The groove track also tells DirectMusic what to do when there are multiple patterns that can be played. For example, suppose in the preceding example that Pattern A had a groove range of 9–11. That would mean that both Pattern A and Pattern B could be legally played. The groove track has additional parameters that tell DirectMusic whether it should randomly choose between the two (the default behavior), or whether it should play Pattern A, then Pattern B, or a wide variety of other behaviors.

Tip

Groove ranges don't need to be actual ranges. You could set the low and high numbers of the groove range to the same value, which means the pattern would only play when the groove level is that exact value.

In code, there's a way for you to adjust the current groove level, by using the global groove level parameter. This number is added to the groove level specified in the segment to come up with the actual groove level. For example, if the segment groove track prescribed a groove level of 3, and the global groove level was 7, DirectMusic would attempt to play patterns whose groove ranges included 10.


DIRECTMUSIC IN TRACKED MUSIC TERMS

So, I've just dumped all these new objects and metaphors on you, and you're probably wondering how on Earth you're going to piece everything together. To help out, this section will explain DirectMusic from a slightly different angle—by comparing it to tracked music.

The tracked music (IT, S3M, and so on) hierarchy of objects goes something like this:

  • Rows: The "spots" where you can put notes. Most are empty, but some contain notes, volume, and/or effects and commands (for example, G01, which says, "Don't play this note directly, instead slowly slide to it from the last note.")

  • Tracks: (also called columns or voices): Tracks are essentially arrays of spaces. Tracks are usually 64 spaces long, but some editors (for example, Impulse Tracker) allow you to change the track lengths for a pattern.

  • Patterns: Also known as tables, patterns are the core building blocks of the music. A piece of tracked music consists of at least one pattern, usually more. A 64-row pattern is usually divided up into 4 measures of 16 beats each… put a note on every four rows and you have the most basic quarter note, 4/4 time rhythm; put two notes two rows apart and you have an eighth-note pair (see Figure 11.4).

    Click To expand
    Figure 11.4: A sequence of notes in a tracker and in sheet music form.

  • Order List: Also known as a pattern list or pattern sequence, this is the list that specifies the order in which patterns should be played.

  • Sample list: This is the list of all the digital samples used in the song. Each entry in a sample list contains the name of an instrument, until of course, the song is done and the composer overwrites those names so that they collectively form a few paragraphs of text and/or ASCII art.

  • Sample Parameters: Parameters for each sample (default volume and panning).

Happily, all of these constructs have counterparts inside DirectMusic. Here are the differences between these core tracked music objects and DirectMusic objects.

Rows Are DirectMusic Notes

In DirectMusic, there aren't really "rows" as you know them. Instead, each note has a timestamp and a duration attached to it that tells DirectMusic when to play it, and for how long (give or take a few hundred milliseconds, to simulate a human being). The resolution on these timestamps is incredible, making it easy to do weird things like triplets and odd time signatures.

DirectMusic divides time up into bar, beat, grid, and tick, as well as a fifth optional +/-parameter that specifies how early or late the note can be (in ticks). For example, if I had a note that started on the first bar, first beat, first grid, and 50th tick (yes, an odd note), and I put in a +/− value of 10, DirectMusic would play the note sometime between the 40th and 60th tick. (That was just an example; in real life, 99.9% of your notes are going to fall on tick zero because you hardly ever need that level of fine control). Each note's duration has a similar +/− value to randomize its length.

Notes also have volumes, only they're called velocities now. Again, there's a +/− value to specify a velocity range.

Columns Are DirectMusic Parts

Zooming out one level—tracked music's columns are analogous to DirectMusic parts. You have one part per instrument, so a string quartet would have four parts. In DirectMusic, you can only have one instrument for each part—you can't create one part that contains multiple instrument sounds. This is very different than what you're used to; in tracked music, you overlaid different instruments onto the same track because tracks were a precious commodity, but DirectMusic supports literally hundreds of parts, so there's no need to cram multiple instruments together.

Patterns Are Still Patterns

Zoom out again, and you're up to patterns. Tracked music patterns are akin to (drum roll)… DirectMusic patterns! But, they're not exactly the same. In tracked music, patterns are usually short (four measures is usually the maximum, if your song's calm and your tempo is slow), but in DirectMusic you'll usually want your patterns to be several measures long. A good rule is to make your pattern about as long as the background music for any game on an 8-bit console system (for example, the original Nintendo). In other words, something like 30 seconds to a minute—four or eight measures in style A, followed by four or eight measures in style B, then looping back to style A. Of course, there's no rule that says they have to be like that—if you want you could make each pattern a measure, or you could put your entire song into one pattern.

Good dynamic music should include several different patterns at different "groove levels" (a.k.a. intensity levels).

Order Lists Are Groove Level Tracks and Style Tracks

This brings us to the final analogy, and perhaps the most different. In DirectMusic, there's not really such a thing as an order list (or pattern list). Instead, there are two types of tracks living inside a segment. There's a style track, which tells DirectMusic the "pool" of patterns that segment can pick from (remember, a style is really just a collection of patterns), and the groove track, which narrows down the pool of patterns by dictating a groove level.

For example, let's say your segment's style track is constant, and that it specifies MyCoolStyle.stp as the style for the entire song. Inside MyCoolStyle.stp are two patterns. Pattern A has a groove range of 1–60. Pattern B has a groove range of 50–100. When your segment groove level track specifies a groove level of 1, the only Pattern that can play is Pattern A—Pattern B's groove range is too high for the current groove level. Conversely, if your segment groove level track specifies a groove level of 75, only Pattern B can play. Something interesting happens if you specify a groove level of 55— that's within the range of both patterns, so DirectMusic randomly chooses one to play.

If you're just starting out and this is too confusing to think about, you could pretend you were writing tracked music by doing the following:

  1. Give each pattern a unique groove level—not a range, but a single number. This first pattern is groove level 1, the second is groove level 2, and so on.

  2. Add a style track to your segment and have it always play from the same style (where all your song's patterns are stored).

  3. Now, you can use the groove level as an order list. In the groove track, specify a new groove level, and the corresponding pattern will play. It's still not exactly the same—unlike in tracked music, your groove track can switch patterns at any time, not just on pattern boundaries—but it's pretty darn close.

  4. If you want to then get fancy, you can give multiple patterns the same groove level range and DirectMusic will choose between them randomly.

Actually, I lied. There's more to it than that—when you change the groove level in your segment's groove level track, you can tell DirectMusic what to do when more than one pattern can be legally played. You can tell it to do any one of the following, via the dialog box shown in Figure 11.5:

  • Choose one pattern at random, play it, then choose another pattern at random, play it, and so on until the groove level changes. This is the default behavior.

  • Choose one lucky pattern at random and loop it continuously until the groove level changes. This is called "Repeat Same."

  • Start with the first valid pattern and play all of them in sequence, then loop back to the first one, until the groove level changes. This is called "Sequential from First."

  • Start with a random pattern and play it, followed by the next sequential pattern, wrapping around if necessary until the groove level changes. This is called "Sequential from Random."

  • Shuffle all legal patterns into a random order, play them in that order, and repeat until the groove level changes. This is called either "Shuffle" or "No repeat," depending on if you want patterns to repeat before all patterns have been played.

Click To expand
Figure 11.5: The Groove Level Properties dialog.

Sample Lists Are DLS Collections

To create anything in tracked music, you must have instrument samples. In DirectMusic, you don't necessarily need to provide your own WAV files. All DirectMusic compositions can use the entire General MIDI sound set by default. The General MIDI sound set contains 127 different instruments, ranging from pianos and violins to special effects and shakuhachis. These high-quality samples are built into DirectX, and you're guaranteed that they will sound the same on all platforms.

Thanks to this sound set, it's actually quite common to have a DirectMusic score that contains no custom samples. However, the sound set still can't help you when you want to include things like drum loops, lyrics, and wicked guitar licks in your music. That's why DirectMusic supports DLS collections.

DLS collections are far more advanced than what you're used to in tracked music. Most tracked music formats only allow an instrument to be one sample; in DLS you can have different samples for different note pitches. You can also adjust the ADSR (attack, decay, sustain, and release) of your instruments, both in terms of relative volume and relative pitch. You can also create drum sets. There's a lot of power in there, if you need it.

Sample Parameters Are Bands

There isn't really a tracked-music equivalent to a DirectMusic band. That's because you can't share sample lists between different tracked music files—each file embeds its own samples, and that's that.

But imagine if you could share samples. You'd want each file to use the same samples, but possibly with different volume and panning parameters. For example, in song 1, you might want the default volume of the flute sample to be high, since it's carrying the melody. But in song 2, where the flute is in the background, you'd want its default volume to be much lower.

That's why DirectMusic has bands. In DirectMusic, it's not only possible, but quite common, for two different songs to share the same samples. At the very least, everybody shares the General MIDI sound set, so you need a place where you can say, "Okay, for this song, take that shakuhachi, make it loud, and make it come mainly out the left speakers." Another song might want it at half-volume played from the left speaker; bands provide that flexibility.

Summary: DirectMusic Is More Than a Tracker

Many concepts are similar in DirectMusic and tracked music, and I provided this section to give all of you tracked music pros a foundation on which to stand. That being said, do not limit yourself to thinking of DirectMusic as just another tracker. Writing dynamic music using DirectMusic and writing tracked music are very different, both in terms of concept and process. Take the time to learn the DirectMusic way of doing things, and it will pay off with better dynamic music for your games.


PROGRAMMING DYNAMIC MUSIC

If you're encountering dynamic music for the first time, now would be a good time to switch gears. Stop reading this chapter and go to Appendix A, which walks you through the creation of a simple dynamic score. Once you've created your sample score, come back to this section to learn how to write the code to accompany it.

The CDynamicMusic Class

This chapter's code, like the code of so many chapters before it, starts with the creation of a new C++ class. The new class is called CDynamicMusic, and it derives from the CDirectMusicSegment you already know and love.

Thanks to good C++ design, this chapter's code is going to be pretty easy. Because dynamic music scores authored in DirectMusic Producer are really just segments, the CDirectMusicSegment class is going to do most of the heavy lifting. All you have to do is add a few methods specific to dynamic music to the new CDynamicMusic class and you should be good to go.

Setting Things Up

For starters, here's the code to load a dynamic music segment from disk, memory, or resources. This should be old-hat by now.

CSoundPtr CAudioManager::LoadDynamicMusic(std::string filename)
{
  CDynamicMusic *music = new CDynamicMusic(this);
  LoadSegmentFromDisk(music, filename);
  music->LoadMotifs();
  return(CSoundPtr(music));
}
CSoundPtr CAudioManager::LoadDynamicMusic(unsigned char *data,
  int datalen)
{
  CDynamicMusic *music = new CDynamicMusic(this);
  LoadSegmentFromMemory(music, data, datalen);
  music->LoadMotifs();
  return(CSoundPtr(music));
}
CSoundPtr CAudioManager::LoadDynamicMusic(HMODULE hmod,
  char *type, WORD resID)
{
  CDynamicMusic *music = new CDynamicMusic(this);
  LoadSegmentFromResource(music, hmod, type, resID);
  music->LoadMotifs();
  return(CSoundPtr(music));
}

In the immortal words of David Byrne: "Same as it ever was." The only difference here is that this code news up a CDynamicMusic class, and then calls its LoadMotifs method. After LoadMotifs, it returns a CSound smart pointer.

LoadMotifs loads all the motifs contained in the styles referenced by the segment. CAudioManager calls this method when it loads the song, so that when the time comes to play a motif, the code doesn't have to waste time looking for the particular IDirectMusicSegment8 interface for that motif. LoadMotifs loops through the segment, and assembles a map of motif names (STL strings) to IDirectMusicSegment8 interfaces:

void CDynamicMusic::LoadMotifs()
{
  IDirectMusicStyle8* pStyle = NULL;
  DWORD dwStyleIndex = 0;
  while(1)
  {
     HRESULT hr = GetSegment()->GetParam( GUID_IDirectMusicStyle,
       0xffffffff, dwStyleIndex, 0, NULL, (VOID*)&pStyle);
    if(FAILED(hr)) {
      // not a fatal error... just means there are no more styles.
      break;
    }
    DWORD dwIndex = 0;
    while(1)
    {
      WCHAR wstrMotifName[MAX_PATH];
      CHAR strMotifName[MAX_PATH];
      if(S_FALSE == pStyle->EnumMotif(dwIndex, wstrMotifName)) {
        // no more motifs in this style
        break;
    }
    wcstombs(strMotifName, wstrMotifName, wcslen(wstrMotifName) + 1);
    CDirectMusicSegment *newseg = new CDirectMusicSegment(m_Manager);
    hr = pStyle->GetMotif(wstrMotifName,
      (IDirectMusicSegment **)&newseg->m_Segment);
    ThrowIfFailed(hr,
      "CDynamicMusic::LoadMotifs: pStyle->GetMotif() failed.");
    // add this segment to our map
    m_MotifMap[std::string(strMotifName)] = newseg;
    dwIndex++;
  }
  SAFE_RELEASE(pStyle);
  dwStyleIndex++;
  }
}

This code, based on code from the PlayMotif DirectX sample program, loops through all of the styles in its segment. For each style, it loops again—this time through all of the motifs for that style. For each motif, it grabs a IDirectMusicSegment8 interface, and gives that interface to a new CDirectMusicSegment class. It then puts the new class into the m_MotifMap map.

The CDynamicMusic destructor takes care of deleting all of the new CDirectMusicSegments created by LoadMotif:

CDynamicMusic::~CDynamicMusic()
{
  for (std::map<std::string, CDirectMusicSegment *>::iterator i =
m_MotifMap.begin(); i != m_MotifMap.end(); ++i) {
    delete (*i).second;
  }
  m_MotifMap.clear();
}

There's nothing special going on here. The destructor uses an iterator to loop through the contents of the map, and delete each CDirectMusicSegment it encounters.

Playing Dynamic Music

There's one key thing you must change in the CDirectMusicSegment class in order to get dynamic music to work properly. To understand why, you must first understand what a control segment is.

DirectMusic lets you play more than one segment at once. For example, you could play a segment containing your dynamic music, and another segment containing a WAV of a sound effect.

As you know, each segment contains a set of tracks. Most of those tracks only influence that segment—it's no big deal if you have two segments playing and they both have their own, say, groove track. But, some tracks of a segment need to be applied globally. For example, all segments must share the same tempo and chord map (because it doesn't make sense to play music at two different speeds, in two different chords).

So, if you have one segment with 2CM chords, and another with A#m7 chords, and you play them both, how does DirectMusic decide which segment's chord track "wins?" For that, it relies on the notion of a control segment. Control segments lord over the other playing segments; their global tracks (chords, tempo, and so on) are always what DirectMusic uses.

This is important because you normally want your dynamic music segment to be the control segment. Otherwise, all of the chord changes you've put in won't be used, and your music will sound incorrect.

The good news is that it's easy to tell DirectMusic a particular segment should be the control segment—just pass in the DMUS_SEGF_CONTROL flag to the PlaySegmentEx method of IDirectMusicPerformance8. The bad news is that due to the C++ architecture of the audio engine, making this option "bubble up" far enough to where you can use it will require editing a few different classes.

Start at the highest level—you need to add a parameter to the Play method of CDirectMusicSegment. This new parameter, called control, is a Boolean specifying whether the segment you're playing should be flagged as the new control segment.

This new parameter gets passed directly to the Init method of CSoundInstance because it's CSoundInstance::Init that calls PlaySegmentEx. Init also has a new parameter (also called control—imagine that!), which it looks at to determine whether to feed PlaySegmentEx a DMUS_SEGF_CONTROL flag.

Now, it's a simple matter of overriding CDynamicMusic's Play method to use the new flag:

bool Play() { return(CDirectMusicSegment::Play(NULL, true)); }

This is subtle, but it's important, because it ensures that your dynamic music is heard correctly.

Changing the Groove Level

Now that you know how to play dynamic music, it's time to learn how to make it adapt to the situation at hand. Let's say your main character hops on a small dinosaur, and you want the music to change. You might accomplish this using groove levels—imagine that you create two "without dinosaur" patterns, and assign them groove levels of 1 and 2. Then you'd create the two "with dinosaur" counterpart patterns, and assign them groove levels of 11 and 12. In your segment, you'd build a groove track that alternates between groove levels 1 and 2, so that your two patterns play. (By the way, this is what Appendix A walks you through.)

In your game, when your character hops on the dinosaur, you just set the performance groove level modifier to 10. DirectMusic will add 10 to the current groove level, which is either 1 or 2, depending on where you are in the segment. This means you'll end up with a groove level of 11 or 12, and you'll hear one of the "with dinosaur" patterns.

When your character loses the dinosaur (as dinosaurs inevitably get lost), your code will set the performance's groove level modifier to zero. This will put the groove level back down to 1 or 2, and your "without dinosaur" patterns will play.

What I've just described is an example of a very simple, very "crisp" change in dynamic music. You can also make dynamic music that contains less drastic shifts, say by making your code slowly ramp the performance groove modifer over time, or by having several patterns in several different overlapping groove ranges. It's all just a question of how much effort you want to go to.

But before you can do any of this, you need to know how to set the performance groove modifier in code. Happily, this is very easy—it's exactly the same as setting the tempo! Here's the code to get and set the modifier:

void CAudioManager::SetPerformanceGrooveLevelModifier(int value)
{
  if (value > 100) value=100;
  if (value < 0) value=0;
  char v = value;
  if (NULL == m_Performance) return;
  ThrowIfFailed(
    m_Performance->SetGlobalParam(GUID_PerfMasterGrooveLevel,
    &v, sizeof(v)),
    "CAudioManager::SetMasterGrooveLEvel: SetGlobalParam failed.");
}
int CAudioManager::GetPerformanceGrooveLevelModifier()
{
  if (NULL == m_Performance) return(0);
  char groove=0;
  m_Performance->GetGlobalParam(GUID_PerfMasterGrooveLevel,
    &groove, sizeof(groove));
  return(groove);
}

These two methods are exactly like SetMasterTempoScaleFactor and GetMasterTempoScaleFactor, except instead of manipulating GUID_PerfMasterTempo, this code manipulates GUID_PerfMasterGrooveLevel.

Playing Motifs

When your player picks up a powerup or does something good, you may want to play a musical sound effect known as a motif. Because the audio manager has already told the new CDynamicMusic class to gather up interfaces to all the motifs and map them by name, the code to actually play a motif is only a few lines long:

void CDynamicMusic::PlayMotif(std::string name, bool
  bUseDefaultBoundary, bool bUseGridBoundary,
  bool bUseMeasureBoundary, bool bUseBeatBoundary)
{
  if (m_MotifMap.find(name) != m_MotifMap.end()) {
    CDirectMusicSegment *motifseg = m_MotifMap[name];
    int segflags = 0;
    if (bUseDefaultBoundary) segflags |= DMUS_SEGF_DEFAULT;
    if (bUseGridBoundary) segflags |= DMUS_SEGF_GRID;
    if (bUseMeasureBoundary) segflags |= DMUS_SEGF_BEAT;
    if (bUseBeatBoundary) segflags |= DMUS_SEGF_MEASURE;

    HRESULT hr = m_Manager->GetPerformance()->PlaySegment(
      motifseg->GetSegment(), segflags, 0, NULL);
    ThrowIfFailed(hr, "CDynamicMusic::PlayMotif: PlaySegment failed.");
  }
}

This method takes as inputs the name of a motif to play, and some flags that tell it when it can start playing (on the grid, beat, measure, or on whatever the default value is). These flags have default values instructing the code to use the default boundary, and most of the time this default boundary works well.


CONCLUSION

DirectMusic is a powerful system, and the price you pay for that power is learning a whole bunch of terms and concepts. The upside, however, is that you can choose the amount of complexity you want. Start simple, by creating dynamic music with only a few components—music that sounds almost exactly the same every time it's played. Then, slowly build both the music itself, and the code, in parallel, until you've got a good grip on how all of these pieces fit together.


Chapter 12: Scripts and Effects

Download CD Content

OVERVIEW

There comes a point of no return in every book—a point where you realize your goal isn't to start reading it, but to finish reading it. I think this chapter is that point. You've learned the really difficult stuff—dynamic music in particular—and now you can start rounding out your knowledge with some advanced techniques.

Two of those advanced techniques are addressed in this chapter. Here, you'll learn about DirectX Audio's audio scripting capabilities, and about its real-time effect capabilities.


AUDIO SCRIPTING

Included in DirectX starting with version 8.0, audio scripting provides a powerful way for development teams to parallelize their development process. At first glance, it may seem like overkill to use a scripting language to play sounds, but in reality it's very useful for team game development, because it allows sound designers to work faster and with less dependency on the programmers.

Why Use Audio Scripting?

Before audio scripting, it was difficult for sound designers to work autonomously. Picture this—you are a sound designer, working on a team consisting of several other sound designers and programmers (and artists, animators, producers, and all sorts of other people too). Your job today is to make sure the orc grunts in your game sound good.

So you work your magic and you come up with what you think is a pretty good WAV file of an orc grunt. You check it into your asset database, and send a note off to the sound programmer asking him to put your sound in the game. The programmer does so, and gives you back a new build of the game so you can hear your sound.

You hear your sound in-game, and that's when you realize it sounds awful. Your orc grunt, which sounded so great in your WAV editor, is playing too soft in the game, and you can barely hear the grunts. So you send an e-mail to the programmer asking him to raise the volume of the orc sound effect. The programmer does so, and makes you a new build of the game.

And then you realize that the orc grunt sounds good on the blue orc but that it sounds silly for the red orc to make the exact same grunting sound as the blue orc. So you make a new blue_orc_grunt.wav file and send an e-mail to your programmer, asking him to put this new sound in the game.

Do you see what's going on here? Every time you want to change something with the audio, you've got to bug the programmers for a new build. This wastes everybody's time—the programmers don't want to be constantly interrupted, and you don't want to constantly have to interrupt them and wait for a new build.

Audio scripting solves this problem by moving the code that plays sound effects out into a script which is loaded at runtime. With this system in place, sound designers no longer have to rely on programmers to compile their changes into the game. They can simply edit a script (text file) and re-start the game to see their changes.

Start Sidebar
Sidebar: Cycle Time

Professional game development teams live and die by what they refer to as "cycle time." Cycle time is simply the time between two tests, minus any time you spent fixing things.

For a programmer, cycle time refers to how long it takes to stop the debugger, re-compile the code, re-launch the debugger, and get back to the place in the game that's having the problem. For the artist or animator, cycle time refers to how long it takes to export the new art asset (texture, model, animation, and so on) and see it inside the game.

Obviously, the shorter the cycle time, the more "tweaks per minute" you can have, and the better your end product will be. If you're an artist, and it takes you several steps and several minutes to see your new art inside the game, you may only go through two or three cycles before you decide that it's "good enough" and move on to something else. If, however, your cycle times are short—say you can see your art inside the game just by typing a couple of commands—you're going to be able to go through several rounds of view art/change art, and the game's art will look better because you've spent time perfecting it.

So, if you're working in a team, strive to keep everyone's cycle times as short as possible. You can do this by spending time creating WYSIWYG (or, in the case of game audio, "what you hear is what you get") game editors, and by making as many things data-driven as possible. The end result will be a better game and happier teammates.

End Sidebar

Of course, the programmer and sound designer still have to agree on the list of available sound effects the game will have. But the programmer no longer needs to be concerned with the details of how those sound effects are played (for example, whether one wav file is used for several sound effects, how loud to play, and so on). The programmer just calls a subroutine defined in an audio script, and the audio script takes care of telling DirectMusic what sound to play, and how.

Tip

Audio scripts aren't particularly useful if you're a lone-wolf developer or if your sound designer also happens to be a programmer and can edit game code directly. However, if you're working in a large team, audio scripts can really help out.

The Basics of Audio Scripting

Imagine that you're creating a game involving cats (Hey, it worked for the theater—why not for the gaming world as well?). The player controls a hero-cat, and can find powerups that over time causes his cat to grow bigger.

Tip

It's crucial to come up with an appropriate audio interface—that is, the list of routines and variables used in your audio script. Audio scripting works best when there is a well-defined interface between the game code and the audio. In a perfect world, you'd define your audio interface once, and only once. The programmers and sound designers would meet, decide what variables the game code would set, what routines it would call, and then they would both go their separate ways. The audio guys would work on the audio script, and the programmers would work on the rest of the game code.

In practice, that never happens, but it's a goal to strive for. A few changes to your audio interface will always occur, but if you find yourself constantly re-inventing your entire interface, you'll also find yourself missing out on the power of audio scripting.

You're the sound designer for this (revolutionary) game, so it's your job to create the feline sound effects. You and your programmers have decided on a simple audio interface. The game code will keep track of how big the hero's cat is, and will convey that information to you via a single variable named CatSize. There will also be single function, sfxCatMeow, which the game code will call whenever the cat should meow. This sfxCatMeow function should look at the CatSize variable and create a meow of the appropriate power. Ideally, there should be a handful of different meowing sound effects, so that the cat doesn't meow exactly the same over and over again.This situation is specially designed to illustrate the usefulness of audio scripts. With an audio script, the sound designers are free to create as many different meowing sounds as they like, and they're free to organize them in whatever fashion they choose, without having to edit game code. Likewise, the programmers are free from the details of how to make a cat meow—they just call a function in a script, and the right meow comes out the speakers, as if by magic.

The following two sections will show, from a sound design and coding perspective, how this audio script is implemented. This audio script will have only one routine and only one variable; keep in mind that an audio script for a real game will have dozens of routines and variables.

You'll first learn how to create the script (that is, code the script itself), followed by how to integrate the script into your C++ game code.

The Sound Designer's Perspective

The audio scripting language DirectX uses is based on Visual Basic, so if you've used any of the other VB variants (Active Server Pages, Office macros, and so on), you already know most of what you need. If you haven't done anything with VB before, you'll probably want to read some online tutorial to pick up the basic flow control and syntax (see some of the links I've provided on your CD).

You can create AudioVB scripts using DirectMusic Producer, but it's a little more involved than just creating a new script object and typing your script into the editor. This is because you can't play WAV files directly from an AudioVB script—you can only play segments. So, you need to create a WAV segment for each sound effect you want to play through the script.

Think of a WAV segment as a segment that contains WAV files instead of patterns. In this chapter, your WAV segments won't have groove levels, they won't depend on styles, and they won't play patterns. Instead, they will generate their sound by using a WAV track. I think of segments with a WAV track as DirectMusic shortcuts; sure, you could create a style and a bunch of patterns, create a style track and groove track in your segment, and get your sound effects to play that way. That's useful if you're going to be constructing complex sound effects from simple WAV elements. But if you just want to play a simple sound effect, you don't have to deal with all that. You can just embed a WAV file directly into your segment and be done with it.

Tip

AudioVB scripts aren't limited to just playing sound effects (though that's all you will learn about here). You can also use them for playing regular segments, changing groove levels, playing motifs—virtually anything you can manipulate with DirectMusic Producer, you can manipulate in an audio script.

Creating Sound Effects Segments

That's what you're going to learn now. Fire up DirectMusic Producer and create a new project (I called mine CatProject). You'll need an audio script and two segments. Name the script [image] catsfx.spp, and name the segments [image] BigCat.sgp and [image] SmallCat.sgp.

Next, insert the four cat meow WAV files into your project by right-clicking on the CatProject root node in the tree, and selecting Insert Files. (You can find the cat meow wav files in the folder for this chapter. There are four files, two that are of a little cat meowing, and two that are of a big cat growling.) Browse to and select the four WAV files, and you should see four WVP objects appear.

Now you need to craft these four WAV files into two segments of two variations each (the wave track, just like a pattern part, can have up to 32 variations). Go to the BigCat segment, right-click on the Time Signature Track, and select Add Track(s) from the context menu. Add a Wave track.

Tip

Instead of clicking 29 times to enable/disable each variation, you can toggle the entire set of variations by clicking the narrow button to the left of all the variation buttons (see Figure 12.1).

Click To expand
Figure 12.1: Clicking the narrow button on the left will enable/disable the entire variation set.

The segment is now beginning to look a lot like a pattern. Notice that your new wave track (a.k.a. Wave Part) has variation buttons just like a pattern does. Disable everything except variation 1, move your cursor to the beginning of the track, and hit Insert. The wave properties, shown in Figure 12.2, will appear. Choose CatProject:BigCat1 to tell DirectMusic that it should play this wave file for variation 1 (note that only WAV files that are part of a project appear in this list; that's why you had to import them before getting here). If all goes well, you should see a graphical representation of your WAV file depicted at the start of the track.

Click To expand
Figure 12.2: The wave properties window.

Follow the same process to make the [image] bigcat2.wav file play at the beginning of variation 2. Remember to disable variation 1 so that you don't accidentally insert [image] bigcat2.wav into both variations. You'll know when you have it right because you'll see [image] bigcat1.wav appear and disappear when you toggle the button for variation 1, and [image] bigcat2.wav appear and disappear when you toggle the button for variation 2.

One more very important thing to do—in the segment window, right-click on the name of the wave track with the purple background, and select Properties. DirectMusic Producer will show you the Wave Track properties (shown in Figure 12.3). Navigate to the Wave Track tab and make sure that the Reset Variation order on Play checkbox is checked. This is a must for creating a segment that plays [image] bigcat1.wav or [image] bigcat2.wav at random.

Click To expand
Figure 12.3: The wave track properties.

If you play the segment using your toolbar, you should now hear [image] bigcat1.wav or [image] bigcat2.wav played at random. (If you hear both at once, it's because you accidentally had one variation enabled while editing the other.) Figure 12.4 shows the final segment (with both variations active).

Click To expand
Figure 12.4: The final BigCat segment,with both variations active.

Duplicate the same recipe to create a SmallCat segment that randomly plays either [image] smallcat1.wav or [image] smallcat2.wav. Again, pay attention to your variations and be sure to test out your segment using the toolbar buttons.

Authoring a Simple Script

With your segments set up, you're now ready to create the audio script itself. Begin by clicking the New toolbar button and selecting Script from the dialog that appears. Rename the new script catsfx.

There's one thing you need to do before you start writing the code for the audio script—you need to tell DirectMusic which objects of your project your script will reference. The script you're about to input will use the two segments SmallCat and BigCat, so you need to add those two segments to the script's references. Open the script content node underneath your script in your treeview, and right-click on Reference Runtime. Add  BigCat.sgp and  SmallCat.sgp to the list of references (see Figure 12.5). Click OK and you should see shortcuts to your segments contained underneath the Reference Runtime folder in your script.

Click To expand
Figure 12.5: Adding references to an audio script.
Caution

It's crucial that you add references to all objects your audio script references. If you don't, your script will not work.

Once you've got your references set up, go back to the script editor (double-click on your catsfx script) and enter the AudioVB code shown next:

dim CatSize
sub sfxCatMeow
     if CatSize < 10 then
          SmallCat.Play(IsSecondary)
     else
          BigCat.Play(IsSecondary)
     end if
end sub

Watch your syntax—AudioVB is different than C++ (for example, putting semicolons at the end of statements in AudioVB will really screw you up).

As for the script itself, it really is that easy—it declares one variable, CatSize, and one subroutine, sfxCatMeow. The sfxCatMeow subroutine looks at the CatSize variable (which the game will set). If it's less than ten, it plays the SmallCat segment; otherwise, it plays the BigCat segment. Note the use of the IsSecondary flag, which tells DirectMusic the segment should be played as a secondary segment (usually you'll want your sound effects to be secondary segments so they don't usurp control from any dynamic background music that might be playing).

Tip

When editing your script, you can use DirectMusic Producer to help check for syntax errors. If you hit the refresh button above the routines list to the right of your editing area, and the list box gets filled in with the names of your subroutines, you'll know your script has no syntax errors. Conversely, if you don't see your subroutines and variables populated when you hit refresh, check your syntax!

Save this script, save your project, and then save everything in Runtime format, and you're done! Put the runtime files into the working directory of the Chapter 12 sample program—the folder named Ch12p1_AudioScripts. You can do this by saving everything directly to that folder, or by copying the runtime files once they're saved inside your project folder. Alternatively, you could also write code to tell DirectX to search in a different directory—by default it searches in the current directory, which, when you run your sample program through DevStudio, is Ch12p1_AudioScripts.

Obviously, a real game would have a more complex audio script. A real game would probably have functions for each sound effect, as well as functions to manipulate the dynamic background music, play motifs, and do whatever else the game demanded. It's entirely up to you to decide how to organize all these subroutines. You can have as many different script files, variables, and subroutines as you want, though a good rule is to use one script file for each level of your game.

Tip

Notice in the audio script that the name of the subroutine is prefixed with sfx. It's a good idea to come up with some sort of standard naming convention for your audio scripts—for example, decree that any subroutine that's concerned with playing a sound effect should start with sfx. Creating (and following!) standard naming conventions can help keep large scripts organized.

The Developer's Perspective

Once you've created your segments and your script, and saved them out in runtime format, you can turn your attention to writing the game code that will interface with the audio script.

DirectX Audio provides a very simple, yet 100% complete, interface to your audio scripts. Essentially, after initializing a script interface, you can ask that interface for a list of subroutines and variables, you can set and retrieve variable values, and you can call subroutines. That's really all you have to do, and most of the time you don't even need to get the list of subroutines and variable names, because just hard-coding the names of the subroutines and variables works just fine.

So, this makes for a short list of code enhancements to perform. The following sections will guide you through adding audio scripting support to the audio engine. Most of this support will come via a new CAudioScript class, backed up by a few new CAudioManager methods. But before you can dive into CAudioScript or CAudioManager, you need to create a new smart pointer.

A New, Slightly Smarter Pointer Class

Up until now, you've gotten away with having only one smart pointer class, CSoundPtr. That's because everything you've learned so far can, at its most basic level, be treated as something you can play, stop, and set the volume on. You can't really play an audio script like you can a segment or a wave file.

So, it doesn't make sense to derive your new CAudioScript class from CSound. CAudioScript doesn't derive from anything, because it's unlike anything you have currently written. This means that when you ask CAudioManager to load an audio script, you're not going to get a CSoundPtr back.

But you have to get something back! Sure, you could just write the load routines and make them return a CAudioScript pointer, but it's very simple to create a new smart pointer (derived from the CRefCountPtr template) specifically designed for CAudioScript. Here's how it looks:

// smart pointer to CAudioScript
class CAudioScriptPtr : public CRefCountPtr<CAudioScript>
{
public:
  explicit CAudioScriptPtr(CAudioScript* p = NULL);
  void Release();
};

That's not so bad—the heavy lifting is done by CRefCountPtr<CAudioScript>. All CAudioScriptPtr needs is an explicit constructor and a Release method. Here are those methods:

CAudioScriptPtr::CAudioScriptPtr(CAudioScript* p) : CRefCountPtr<CAudioScript>(p)
{
  stringstream str;
  if (p) {
    str << "CAudioScriptPtr::CAudioScriptPtr: constructed for pointer "
        << p << " name: " << p->GetName() << " (count = "
        << itsCounter->count << ")..." << endl;
    OutputDebugString(str.str().c_str());
  }
}
void CAudioScriptPtr::Release()
{
  stringstream str;
  if (itsCounter) {
    str << "CAudioScriptPtr::Release(): pointer " << itsCounter->ptr
        << " name: " << itsCounter->ptr->GetName()
        << " (count = " << itsCounter->count << ")..." << endl;
    OutputDebugString(str.str().c_str());
  }
}

Neither of these methods do anything critical—they just talk a lot. The constructor echoes a status line to the DevStudio trace output, so you can see exactly when smart pointers are constructed, and the Release method echoes a line so you can make sure things are freed correctly. Notice that these routines output the name of the CAudioScript class, fetched via a call to the GetName method that you'll create later. Note that Release doesn't actually do anything—the real pointer deletion takes place inside CRefCountPtr.

The CAudioScript smart pointer is just like the CSound smart pointer, only with different trace output and a different template parameter to CRefCountPtr. The CAudioScriptPtr displays the name of the audio script it's working on, so you could say that CAudioScriptPtr is slightly smarter than CSoundPtr, though since both know exactly the right time to delete their pointers, they're both pretty smart.

CAudioScript

With the new CAudioScriptPtr smart pointer class created, you can now create the CAudioScript class itself. Here's the declaration:

class CAudioScript
{
public:
  friend class CAudioManager;
  CAudioScript(CAudioManager *mgr);
  virtual ~CAudioScript();
  void SetVariable(std::string varname, int value);
  int GetVariable(std::string varname);
  void CallRoutine(std::string procname);
  std::vector<std::string> EnumVariables();
  std::vector<std::string> EnumRoutines();
  std::string GetName() { return(m_Name); }
  std::string AssembleErrorInfoString(std::string header,
    DMUS_SCRIPT_ERRORINFO &errInfo);
protected:
  std::string m_Name;
  void Init(std::string name);
  CAudioManager *m_Manager;
  IDirectMusicScript8 *m_Script;
  unsigned char *m_OriginalData;
};

There should be a lot of things here that you recognize, and a few things you don't. Starting with the familiar, notice that CAudioScript has an m_OriginalData pointer, just like CSound does. This m_OriginalData pointer is set by the audio manager when an audio script is created from raw memory—just like the CSound derivatives, the CAudioScript class remembers the memory it was born from, and it frees that memory when it's destroyed. Notice also that just like CSound, CAudioScript is a friend of CAudioManager (so that CAudioManager can access CAudioScript's private and protected members directly when it's loading a new audio script), and that a new CAudioScript class can only be constructed by providing a CAudioManager pointer (so that every CAudioScript has a pointer to a CAudioManager instance, and therefore a DirectMusic performance interface).

And now, here are the new things. At the heart of CAudioScript is m_Script, the IDirectMusicScript8 interface, provided by DirectX Audio. You can also see that the audio script has an m_Name member, accessed through the GetName and SetName methods. This is just to make things simpler for the game programmer—the name of the audio script will be echoed to the trace output by the CAudioScript smart pointer on creation and release. You'll see how the code decides on a name in the next section.

CAudioScript also has a private Init function, which takes care of performing one-time initialization on the script. This Init function is called only by CAudioManager (a friend, able to call protected functions) when a new script is loaded. Finally, there are methods to get and set variables, call a routine, enumerate the available variables and routines, and assemble error strings, all of which you'll learn about in the following sections.

Initializing and Loading a Script

Before you can do anything with an audio script, you must load and initialize it. This means you need three new methods inside CAudioManager: one method to load from disk, one to load from resources, and one to load from memory. Here's a look at the disk-loading method:

CAudioScriptPtr CAudioManager::LoadScript(std::string filename,
  std::string name)
{
  CAudioScript *script = new CAudioScript(this);
  HRESULT hr;
  // convert filename to wide-string
  WCHAR widefilename[MAX_PATH];
  DXUtil_ConvertGenericStringToWide( widefilename, filename.c_str());
  // tell loader to load this file
  hr = m_Loader->LoadObjectFromFile(
    CLSID_DirectMusicScript,
    IID_IDirectMusicScript8,
    widefilename,
    (void**) &script->m_Script);
  ThrowIfFailed(hr, "LoadObjectFromFile failed.");
  if (!name.length()) { name = filename; }
  script->Init(name);
  return(CAudioScriptPtr(script));
}

By now, you should be familiar and comfortable with all of the actions performed by this LoadScript method. It creates a new CAudioScript, converts the given filename into wide-string, tells DirectMusic to load the file (specifying CLSID_DirectMusicScript and IID_IDirectMusicScript8 for the class and interface, respectively), and stuffs the interface DirectMusic provides into the script object's m_Script variable. Once all that's complete, it sets the script object's name and calls its Init method. When Init is complete, it returns a new smart pointer to the new script.

The only thing mysterious here is the Init method of CAudioScript. Here's what that looks like:

void CAudioScript::Init(string name)
{
  m_Name = name;
  DMUS_SCRIPT_ERRORINFO errInfo;
  HRESULT hr;

  errInfo.dwSize = sizeof(DMUS_SCRIPT_ERRORINFO);
  hr = m_Script->Init( m_Manager->GetPerformance(), &errInfo);
  if (FAILED(hr)) {
    ThrowIfFailed(hr,
      AssembleErrorInfoString("m_Script->Init() failed!", errInfo));
  }
}

Init is short and sweet. All it's really doing is calling the Init method of the audio script interface. This is a requirement by DirectX Audio; before you do anything with a script interface, you must initialize it by telling it what performance it's tied to.

Notice the DMUS_SCRIPT_ERRORINFO object, errInfo. The address of this structure is passed as the second argument to IDirectMusicScript8::Init, and the code gives the structure to AssembleErrorInfoString when something goes wrong.

The DMUS_SCRIPT_ERRORINFO object is cool. When you Init an IDirectMusicScript8 interface, you're actually compiling the AudioVB code, so if there are any errors in the code, initialization will fail, and errInfo will be filled with all sorts of useful information. You'll get the file and line number of the error, along with a description of the error and the component that generated the error (that is, the name of the DirectX component that has a problem with your script).

The AssembleErrorInfoString uses the information in the DMUS_SCRIPT_ERRORINFO object it's given to create a detailed error string. This error string is then thrown via the ThrowIfFailed macro, and shown in a dialog similar to that in Figure 12.6.

Click To expand
Figure 12.6: The error dialog generated by AssembleErrorInfoString.

Here's what AssembleErrorInfoString looks like:

string CAudioScript::AssembleErrorInfoString(string header, DMUS_SCRIPT_ERRORINFO
&errInfo)
{
   stringstream str;
   str << header << endl;

   // put error info into string!
   char file[DMUS_MAX_FILENAME];
   char desc[DMUS_MAX_FILENAME];
   char component[DMUS_MAX_FILENAME];
   DXUtil_ConvertWideStringToGeneric(file,
     errInfo.wszSourceFile, sizeof(file));
   DXUtil_ConvertWideStringToGeneric(desc,
     errInfo.wszDescription, sizeof(desc));
   DXUtil_ConvertWideStringToGeneric(component,
     errInfo.wszSourceComponent, sizeof(component));

   str << "There was an error in script file \"" << GetName()
       << "\"." << endl << endl;
   str << "File: " << file << ", line " << errInfo.ulLineNumber
       << ": " << desc << endl
   << endl; str << "Error reported by: " << component << endl;
   return(str.str());
}

The only irritating thing about the DMUS_SCRIPT_ERRORINFO object is that all of its string members are wide-character, so you have to convert them to normal ASCII strings (usually via the DXUtil_ConvertWideStringToGeneric function). Once you do that, you have detailed information about the error, which you can use as you please—the AssembleErrorInfoString function outputs everything into one big STL string, and returns that.

Enumerating Variables and Routines

Once you've got an audio script loaded and initialized, you can peek inside it using methods that enumerate its subroutines and variables. Here's how CAudioScript's EnumRoutines method gets a list of all the routines in the script:

vector<string> CAudioScript::EnumRoutines()
{
  vector<string> routines;
  int count = 0;
  HRESULT hr;
  bool done=false;
  while (!done) {
    WCHAR widename[MAX_PATH];
    char name[MAX_PATH];
    hr = m_Script->EnumRoutine(count, widename);
    ThrowIfFailed(hr, " EnumRoutine failed.");
    if (hr == S_FALSE) done=true;
    else {
      DXUtil_ConvertWideStringToGeneric(name, widename, sizeof(name));
      routines.push_back(name);
      count++;
    }
 }
 return(routines);
}

This code loops, repeatedly calling the EnumRoutine method of IDirectMusicScript8. That method takes two parameters—the index of the routine you're interested in, and a pointer to a wide-character array that will be filled in with its name. If the given index is too high, it returns S_FALSE.

This makes getting a list of routines very easy. The code starts with index zero, calls EnumRoutine, and looks at the return value. If it's S_FALSE, it stops the loop; otherwise, it converts the name it got back into a normal string, and pushes it onto the vector.

The implementation of CAudioScript::EnumVariables works the same way, only instead of calling EnumRoutine, it calls EnumVariable.

Setting Variable Values

So, you've loaded a script, initialized it, and maybe you've looked inside it by enumerating its variables and routines. Now you want to communicate with it by setting some of its variables, in preparation for calling one of its routines. Look no further than the CAudioScript::SetVariable routine:

void CAudioScript::SetVariable(string varname, int value)
{
  DMUS_SCRIPT_ERRORINFO errInfo;

  WCHAR widevarname[MAX_PATH];
  DXUtil_ConvertGenericStringToWide(widevarname, varname.c_str());
  HRESULT hr = m_Script->SetVariableNumber(widevarname,
    value, &errInfo);
  if (FAILED(hr)) {
    ThrowIfFailed(hr,
      AssembleErrorInfoString("SetVariableNumber failed!", errInfo));
  }
}

Setting an integer number is as easy as calling the SetVariableNumber method of IDirectMusicScript8. Give this method the name of the variable you want to set (as always, in wide-character format!), along with a value, and DirectX Audio will set that variable if it exists (for example, if the audio script has dim'd it), or it will fill up a DMUS_SCRIPT_ERRORINFO structure if something goes wrong.

Tip

You can get and set variables as string values, using a process similar to that of integer values. Check out the DirectX Audio documentation for the SetVariableVariant method.

Getting Variable Values

Retrieving a variable's current value is just as easy:

int CAudioScript::GetVariable(string varname)
{
  DMUS_SCRIPT_ERRORINFO errInfo;
  WCHAR widevarname[MAX_PATH];
  long value=0;
  DXUtil_ConvertGenericStringToWide(widevarname, varname.c_str());
  HRESULT hr = m_Script->GetVariableNumber(widevarname,
    &value, &errInfo);
  if (FAILED(hr)) {
    ThrowIfFailed(hr,
      AssembleErrorInfoString("GetVariableNumber failed!", errInfo));
  }
  return(value);
}

The GetVariableNumber script puts the value of the given wide-character variable name into a long pointer, or it fails and fills in a DMUS_SCRIPT_ERRORINFO structure if something goes wrong.

Calling a Routine

Once you've set up your variable values, calling a routine is as easy as calling the CallRoutine method of IDirectMusicScript8, as shown by the following code:

void CAudioScript::CallRoutine(string procname)
{
  DMUS_SCRIPT_ERRORINFO errInfo;
  WCHAR wideprocname[MAX_PATH];
  long value=0;
  DXUtil_ConvertGenericStringToWide(wideprocname, procname.c_str());
  HRESULT hr = m_Script->CallRoutine(wideprocname, &errInfo);
  if (FAILED(hr)) {
    ThrowIfFailed(hr,
      AssembleErrorInfoString("CallRoutine failed!", errInfo));
 }
}

Again, aside from the now-standard wrapper for dealing with the errInfo structure, there isn't much to this. Convert the procedure name you want to call into a wide-character string, call CallRoutine, and DirectX does the rest!

To wrap up, here are a few final notes about audio script subroutines. First, they execute synchronously—that is, CallRoutine doesn't return until it hits the end sub line of the routine you called, so your game code can fall victim to infinite loops and hanging caused by a buggy script.

Second, realize that you can't pass arguments to audio script routines; everything must be in the form of global variables that the subroutine can access. This underscores the need for a consistent naming convention; consider making fake subroutine arguments by prefixing the name of the subroutine onto all arguments used by that subroutine.

Finally, be aware that audio scripts can be called by DirectMusic itself; for example, you can embed calls to audio script subroutines inside other DirectMusic objects. This isn't necessarily something to fear, just something for you and your sound designer to be aware of, as it can be useful in certain situations.

The Audio Script in the Game

You now have a complete CAudioScript object, just dying to be used in an actual program. Here are some quick notes on how to use CAudioScript to accomplish the cat meow scenario discussed at the beginning of the chapter.

The complete code for that example is contained in Ch12p1_AudioScripts. Obviously, it's a far cry from a real game example, but it shows you everything you need to implement audio scripting in your own game.

Audio Scripting Wrap-Up

You've been given a brief overview of audio scripting, and if you've never dealt with integrating a scripting language into a game, you've probably learned many new things. If you'd like to learn more about audio scripting, consult the DirectX Audio documentation, at DirectX Audio\Understanding DirectX Audio\Audio Scripts, and DirectX Audio\Using DirectX Audio\Using Audio Scripts. Also, check out the links on your CD.


EFFECTS

Proper use of real-time effects can make your game audio really shine. Add a slight echo when the player steps into a cave or a slight distortion when the communication signal breaks up, and you can really add to your player's sense of immersion.

DirectX Audio provides several stock effects, such as echo, flange, chorus, distortion, and gargle (for a complete list, consult your DirectX documentation, at DirectX Audio\Using DirectX Audio\Using Effects\Standard Effects). Also, developers can write custom effects and use them from within their games.

There are two main ways to use effects: you can set them up from inside DirectMusic Producer or you can apply them at runtime. Which method you should use depends on how your game audio is designed.

Adding effects using DirectMusic Producer is handy if the effects won't change during the course of playback. For example, if you're authoring a DirectMusic project for a level of your game that takes place entirely within a cave, you could easily edit your default audio path so that all of your sound effects play through an echo effect. Conversely, if your player is going in and out of caves, this method won't work—you'll want to engage and disengage the echo effect at runtime, based on, say, the type of terrain (cave or not cave) underneath the current player position.

The following sections will teach you both methods.

Adding Effects Using DirectMusic Producer

DirectMusic Producer makes adding effects easy. All you need to do is create an audio path for the segment you want to add an effect to. Do this by right-clicking on the segment and selecting New Audiopath. Usually, you'll want to base your audio path on a standard path, such as stereo sound or 3D sound, so select the appropriate option (when in doubt, just base it on standard stereo). The audio path editor, shown in Figure 12.7, will appear.

Click To expand
Figure 12.7: the audio path editor.

Right-click on the 1-128 mix group, and select Properties. To add custom effects, you'll need to uncheck the Use Standard Buffer checkbox (see Figure 12.8). Then, close the Properties dialog, and just drag-and-drop the effects you want from the effects palette on the right-hand side of your editing window to the mix group you want to apply them to. You should see a new effect instance appear in the Effects List column.

Click To expand
Figure 12.8: Ucheck the Use standard Buffer check - box to add custom effect to your audio path.

Once you've got an effect hooked up, you can edit its properties by right-clicking on it and selecting Properties, then clicking Edit Effect Properties. Clicking that button will bring up the Effect Properties dialog (see Figure 12.9). You can also cut, copy, and paste effects, and you can hear your changes at any time by playing the object whose audio path you're editing. Cool, isn't it?

Click To expand
Figure 12.9: The Effect Properties dialog for the echo effect.

Adding Effects at Runtime

Adding effects at runtime isn't as easy as using DirectMusic Producer, but it's still pretty straightforward, and it also teaches you another critical technique: namely, how to get an interface to an audio path's underlying DirectSound buffer.

You worked directly (no pun intended) with DirectSound back in Chapter 2, when you learned how to put raw bytes into a secondary DirectSound buffer and mix them out the primary buffer to the speakers. Underneath DirectMusic, this same process is happening, only it's not your code generating those bytes, it's the DirectMusic synthesizer.

Every DirectMusic audio path is actually a DirectSound buffer, and every buffer can have multiple effects attached to it. As these secondary buffers get mixed onto the primary buffer, DirectX Audio applies the effects attached to them. Turning an effect on or off at runtime is accomplished by grabbing the correct DirectSound buffer and inserting or removing an effect from it.

Each effect attached to a buffer has its own parameters structure and its own interface to set or retrieve the specific values for that effect. For example, if you wanted to change the delay of an echo, you'd grab the IDirectSoundFXEcho8 interface that you're interested in, fill in the appropriate members of a DSFXEcho parameter structure, and pass that structure to the SetAllParameters method of IDirectSoundFXEcho8.

Programming DirectX Audio Effects

Every effect has a GUID, defined in dsound.h. For example, the name of the echo effect's GUID is GUID_DSFX_STANDARD_ECHO. These GUIDs are how you tell DirectX Audio what kind of effects you want.

To use an effect, you do two things. First, you have to get the DirectSound buffer interface of the audio path you want to add an effect to. Once you've got the buffer's interface, you call its SetFX method to attach all of your effects onto the buffer. Once the effect is attached, you can retrieve its interface and adjust its parameters.

Hypothetically, here's how the process might look in code (assume that audiopath is the interface of the audio path you want to add an effect to):

// get audio path's underlying directsound buffer
hr = audiopath->GetObjectInPath(DMUS_PCHANNEL_ALL, DMUS_PATH_BUFFER, 0,
  GUID_NULL, 0, IID_IDirectSoundBuffer8, (LPVOID*)&directsoundbuffer);
ThrowIfFailed(hr, "Couldn't get DirectSound buffer interface.");
// Add an echo effect.
DSEFFECTDESC dsEffect;
dsEffect.dwSize = sizeof(DSEFFECTDESC);
dsEffect.dwFlags = 0;
dsEffect.guidDSFXClass = GUID_DSFX_STANDARD_ECHO;
dsEffect.dwReserved1 = 0;
dsEffect.dwReserved2 = 0;

DWORD dwResults;

hr = directsoundbuffer->SetFX(1, &dsEffect, &dwResults);
ThrowIfFailed(hr, "Couldn't set effect.");
// get the effect's interface IDirectSoundFXEcho8* effect = NULL;
hr = audiopath->GetObjectInPath(DMUS_PCHANNEL_ALL,
    DMUS_PATH_BUFFER_DMO, 0, GUID_All_Objects, 0, IID_IDirectSoundFXEcho8,
    (LPVOID*) &effect);
ThrowIfFailed(hr, "Couldn't get effect interface.");
// set the effect's params
DSFXEcho params;
params.fWetDryMix = 50; // 50% echo (wet), 50% original wave (dry)
params.fFeedback = 50; // 50% of output fed back into input
params.fLeftDelay = 750; // wait 750ms, the echo out left speaker
params.fRightDelay = 500; // wait 500ms, then echo out right speaker
params.lPanDelay = 0; // don't swap echo
hr = effect->SetAllParameters(&params);
// release the effect's interface
SAFE_RELEASE(effect);
// release the buffer interface
SAFE_RELEASE(directsoundbuffer);

This code doesn't exist as written anywhere—in real-life it's scattered in a few different places (that you'll learn about later). But this is how it would look if you were dealing strictly with DirectX Audio; this code illustrates the minimum steps needed to apply an echo effect to a particular audio path (or should I say to the DirectSound buffer of a particular audio path).

The code starts by grabbing the IDirectSoundBuffer8 interface contained in the audio path. That interface is stored in directsoundbuffer. Next, the code assembles a DSEFFECTDESC structure. This structure tells DirectX Audio what kind of effects are wanted. The SetFX method of the buffer interface allows you to add several effects at once; it expects an array of these DSEFFECTDESC structures, with each structure containing the GUID of an effect to be applied.

The first parameter to SetFX specifies the number of DSEFFECTDESC structures you're sending; the second parameter is the address of the first structure in the array. The third parameter is the address of a DWORD array, with as many elements as there are DSEFFECTDESC structures. If anything goes wrong with a particular effect, DirectX Audio puts a code into the corresponding member of this array. This can be used to determine what specifically caused the call to SetFX to fail.

Once SetFX returns successfully, the audio path has the echo effect attached to it, but the echo effect is using default values. So, the code grabs the interface for the echo effect's interface by calling GetObjectInPath. It then adjusts the parameters contained in the DSFXEcho structure, and sends the new values to the effect via a call to SetAllParameters. Finally, it releases the interfaces it has gathered.

Again, that's a hypothetical code path—in reality, the different steps are spread over several classes, which you'll now learn about.

Adding Effects to the Audio Engine

The process of adding effects support to the audio engine is one that involves a few edits to the CSoundInstance class, as well as the creation of a handful of new classes (one for each type of standard effect supported by DirectX Audio: echo, flanger, chorus, and so on).

When you're finished, you'll be able to use effects very simply, thanks to the C++ audio engine code provided in this section. Recall from Chapter 5 that if you want to play multiple sounds concurrently, you need a CSoundInstance for each instance of the sound:

CSoundInstance inst[numinsts];
for (int q=0; q < numinsts; q++) {
  snd->Play(&inst[q]);
  Sleep(500);
}

This code, taken from Ch5p2_SoundInstances, illustrates the basic idea: You create several CSoundInstance structures, and each time you play a sound, you give the Play method a unique instance it can use.

I've designed the effects enhancements so that effects can be used following the same procedure. To use an effect, all you need to do is add it to a sound instance. Anything played on that sound instance will then have that effect. For example, here's how to do the same thing as above, only with an echo effect added to each instance:

CSoundInstance inst[numinsts];
for (int q=0; q < numinsts; q++) {
  CEchoEffect *echo = new CEchoEffect;
  // set parameters for echo effect
  inst[q].AddEffect(echo);
  // once you call AddEffect, the engine owns the effect pointer.
  snd->Play(&inst[q]); // with echo!
  Sleep(500);
}

Here you can see the required steps—first, you create a new echo effect class (CEchoEffect). Normally, right after you create the effect class, you'll want to fill up its m_Params member with the specific parameters for the effect, but I left that code out here to save space. Once you do that, you call the AddEffect method of CSoundInstance (which you'll learn about in the following sections), and the effect is magically applied to that instance. Any sounds played from that instance will have an echo applied to them!

You can call AddEffect repeatedly to add additional effects, and if you want to clear everything and start over, you can call RemoveAllEffects. I didn't bother with making it easy to remove individual effects because games don't usually do that, but if you want to, it should be easy for you to add that method.

Tip

Be aware that the audio engine only supports one instance per effect type. That is, you can't have two echo effects on the same audio path. I did this so that the code would be easier to follow; if you want to implement support for multiple instances of the same effect, feel free!

Now that you see how things work from a client's perspective, here's how to implement this functionality by changing the audio engine's code.

The Effects Classes

In the audio engine, each of the different effects is represented by an effects class. All of these effects classes derive from a common CEffect base class. CEffect looks like this:

class CEffect
{
public:
  CEffect(); virtual ~CEffect();
  virtual GUID GetGuid() = 0;
  virtual void SetAllParameters(IDirectMusicAudioPath8 *audiopath) = 0;
};

The two important methods in the CEffect base class are GetGuid and SetAllParameters. These are both pure virtual functions, so they must be defined by all the derived effect classes. The GetGuid method returns the GUID for that particular effect (for example, GUID_DSFX_STANDARD_ECHO), and the SetAllParameters method applies the class's parameters to the effect on the given audio path. You'll learn more about SetAllParameters in the next section.

Here's the echo effect class you saw earlier:

class CEchoEffect : public CEffect
{
public:
  CEchoEffect() { ZeroMemory(&m_Params, sizeof(DSFXEcho)); }
  virtual ~CEchoEffect() { }
  virtual void SetAllParameters(IDirectMusicAudioPath8 *audiopath);
  virtual GUID GetGuid() { return(GUID_DSFX_STANDARD_ECHO);}
  DSFXEcho m_Params;
};

Here you can see that the CEchoEffect derives from CEffect and implements the two required pure virtuals. Notice also that I was lazy when writing the class, and just made the m_Params structure a public member. If you wanted to, you could protect this structure and create accessor functions to get and set a specific member of it (for example, GetWetDryMix and SetWetDryMix). You could also make the constructor fill in the m_Params structure with sensible default values, instead of just setting everything to zero.

Enhancements to CSoundInstance

The CSoundInstance class contains a few new functions and variables:

class CSoundInstance
{
public:
  // existing stuff snipped to save space
  void AddEffect(CEffect *effect);
  void RemoveAllEffects();
  void ApplyEffectsToAudioPath();
protected:
  void ClearEffectsVector();

  std::vector<CEffect *> m_Effects;
  IDirectSoundBuffer8 *m_DirectSoundBuffer;
};

First, there's m_Effects, an STL vector of pointers to CEffect classes. This vector represents the list of all the effects attached to the instance. Second, there's a DirectSound buffer pointer, m_DirectSoundBuffer. This buffer is fetched from the audio path and is used to apply the effects by calling its SetFX method.

There are also some new methods in CSoundInstance. The public effects interface is comprised of three methods: AddEffect, RemoveAllEffects, and ApplyEffectsToAudioPath. The ApplyEffectsToAudioPath method is a refresh method; if you change the parameters of an effect after you add it, you will need to manually call this function to send the new parameters to the effect. There's also a new protected method, ClearEffectsVector, that's called at destruction time and by RemoveAllEffects, and deletes all of the effects pointers in the m_Effects array.

The AddEffect method is simple:

void CSoundInstance::AddEffect(CEffect *effect)
{
  m_Effects.push_back(effect); ApplyEffectsToAudioPath();
}

As you can see, all it does is push the given effect pointer onto the vector, and then calls ApplyEffectsToAudioPath, which does the real work in getting the new effect onto the audio path.

RemoveAllEffects is similarly easy:

void CSoundInstance::RemoveAllEffects()
{
  ClearEffectsVector();
  ApplyEffectsToAudioPath();
}

The RemoveAllEffects method just clears the effect vector, and then relies on the ApplyEffectsToAudioPath method to inform DirectX Audio of the change.

Here's the important method:

void CSoundInstance::ApplyEffectsToAudioPath()
{
  HRESULT hr;
  if (NULL == m_AudioPath || NULL == m_DirectSoundBuffer) return;
  // deactivate audiopath
  m_AudioPath->Activate(false);
  // remove any effects possibly attached.
  hr = m_DirectSoundBuffer->SetFX(0, NULL, NULL);
  ThrowIfFailed(hr, "Couldn't remove effects chain.");
  if (m_Effects.size() == 0) {
    m_AudioPath->Activate(true);
    return;
}

// our effects vector isn't empty, so loop through it and
// apply the effects to the audiopath
std::vector<DSEFFECTDESC> effectdescs;
std::vector<DWORD> effectdescresults;
for (std::vector<CEffect *>::iterator i = m_Effects.begin();
i != m_Effects.end(); ++i) {
  DSEFFECTDESC edesc;
  edesc.dwSize = sizeof(DSEFFECTDESC);
  edesc.dwFlags = 0;
  edesc.guidDSFXClass = (*i)->GetGuid();
  edesc.dwReserved1 = 0;
  edesc.dwReserved2 = 0;
  effectdescs.push_back(edesc);
  effectdescresults.push_back(0); // dummy value
}

  hr = m_DirectSoundBuffer->SetFX(effectdescs.size(),
    &effectdescs[0], &effectdescresults[0]);
  ThrowIfFailed(hr, "Creation of effects chain failed.");
  // feel free to add more detailed error msg processing here
  // now that the effects are created, grab each of their
  // interfaces and set their params.
  for (std::vector<CEffect *>::iterator q = m_Effects.begin();
  q != m_Effects.end(); ++q) {
    (*q)->SetAllParameters(m_AudioPath);
  }
  // activate audiopath
  m_AudioPath->Activate(true);
}

This code's job is to look at the m_Effect vector and put those effects onto the audio path. It starts by deactivating the audio path—this is a requirement, because the SetFX call will fail if the audio path is active. Next, it removes any effects that might already be present, by passing a zero and a couple of NULLs to SetFX. This is done in case any effects have been removed from the m_Effects vector. Note that at this point, the code returns if the m_Effects vector is empty.

If the vector isn't empty, the code loops through it and creates a vector of DSEFFECTDESC structures. Vectors are really just arrays, so by using a vector, you avoid the hassle of having to allocate an array of the right size; instead, you just push things onto the vector. For each effect in the m_Effects vector, the code asks it for its GUID, puts that GUID into a new DSEFFECTDESC structure, and adds that structure to the effectdescs vector. Notice that it also pushes a zero onto the effectdescresults array. This is just so that by the time the loop ends, the effectdescresults array is big enough for SetFX.

Once the loop ends, the code calls SetFX, giving it the vectors it created, along with the number of effects to create (a.k.a. the size of the m_Effects vector). The code doesn't do detailed error checking on SetFX—if it fails, you could enhance it by having it look at the effectdescresults array to figure out what effect caused it to fail.

After SetFX returns successfully, the code again loops through each class in the m_Effects vector. It tells each effect class to apply its parameters to the audio path. Once that's done, the code activates the audio path and returns—everything's now set up!

Now, all that remains to be seen is how the effects classes apply their parameters to the audio path.

Applying Effect Parameters

Each effect class has a SetAllParameters method that's supposed to apply the parameters contained in that effect class to the DirectX Audio effect object on the audio path. Here's what that code looks like:

void CEchoEffect::SetAllParameters(IDirectMusicAudioPath8 *audiopath)
{
  HRESULT hr;
  IDirectSoundFXEcho8* effect = NULL;

  // get the effect's interface
  hr = audiopath->GetObjectInPath(DMUS_PCHANNEL_ALL,
    DMUS_PATH_BUFFER_DMO, 0, GUID_All_Objects, 0,
    IID_IDirectSoundFXEcho8, (LPVOID*) &effect);
  ThrowIfFailed(hr, "couldn't get effect interface.");
  // set the effect's params
  hr = effect->SetAllParameters(&m_Params);
  // release the effect's interface
  SAFE_RELEASE(effect);
}

The three main steps are to get the effect interface, call the SetAllParameters method of that interface, and release the interface. Notice that the interface is retrieved from the given audio path, not from a DirectSound buffer. Also, notice that all the effects use the DMUS_PATH_BUFFER_DMO flag; they differ only by their IID. This is because inside DirectX Audio, effects are actually DirectX Media Objects (DMOs)—they are used by other DirectX components, such as DirectShow. They're also used directly by other programs.

Each effect class needs its own version of SetAllParameters, because each effect class must request a different DirectX Audio effect interface, and supply a different m_Params structure. You could probably write a template to eliminate some of the code duplication—if you're feeling adventurous, go for it!

Runtime Effect Programming Wrap-Up

You've now seen the complete workings of one particular effect: the echo effect. The source code on your CD contains complete implementations of the standard DirectX effects (for example, there's also a CGargleEffect, CFlangeEffect, and CParamEqEffect). The Ch12p2_Effects sample program demonstrates how a few of these effects sound. Feel free to edit the sample program to use the other effects, and to change the effects' parameters to achieve different sounds. (Although it's probably easier to do this in DirectMusic Producer so that you don't have to recompile each time you change an effect parameter!)

Effects Wrap-Up

A word of caution: use effects sparingly. Like lens flares and graphic violence, effects work best when applied strategically, and for a specific purpose. They are often applied with great subtlety. A common mistake is to look at all these cool new effects and start blanketing your game's audio with them. Resist the temptation to do this. Instead, apply effects with restraint and precision, and your game's audio will truly sound great.

To learn more about effects, consult the DirectX documentation at DirectX Audio\Using DirectX Audio\Using Effects.


CONCLUSION

In this chapter, you learned a bit about a couple of different DirectX Audio technologies. These technologies are by no means requirements for a game, but in certain situations, using them can be helpful.


Chapter 13: 3D Sound Using DirectX Audio

Download CD Content

Have you had enough of 2D sound? Are you sick of having to manually set a sound's volume and panning position? Do you want cool effects (like the Doppler effect) in your game? Then read on, because this chapter will teach you how to do all that and more! This chapter introduces you to the concepts and techniques behind 3D sound using DirectX Audio.

PRINCIPLES OF 3D SOUND

Before you start learning how to code 3D sound, you first need to spend some time thinking about what 3D sound actually is. When you say a game has 3D sound, what do you mean? This is similar to saying that a game is 3D—for both graphics and sound, there are different degrees to which a game can be 3D. Obviously, first-person shooters have fully 3D graphics (and probably fully 3D sound as well), but what about an RTS whose background is 2D but whose units are 3D?

To understand what 3D sound means, you need to understand the principles of how the human brain interprets sound. Scientists studying the perception of sound have discovered that your brain relies on the following cues to orient sound relative to itself:

  • Loudness: The louder the sound, the closer your brain thinks it is. Of course, different things have different natural volumes, but for any given sound sample, if it's louder, your brain will think it is closer.

  • Interaural intensity difference: A sound directly to your left will still come in through your right ear, but it will be a little less loud, cueing your brain to interpret the sound as coming from your left.

  • Interaural time difference: This is similar to the concept above; sounds coming from your left will hit your right ear a millisecond or so later than your left (depending on how far apart your ears are).

  • Muffling: The shape of your ear is best suited to hear things in front of you. If a sound is behind you, it will sound muffled. To exaggerate this effect, cup your hands behind your ears, with your palms facing out, and then listen to a sound while facing it and with it behind you. It will sound different, and amazingly, scientists can simulate this difference mathematically.

Fortunately, DirectX Audio takes care of all these details for you. All you need to do is tell it where in the virtual world the sound is coming from, and where the player's ears are. To do that, you use listeners and buffers.

Listeners and Buffers

In DirectX Audio's world of 3D sound, there are only two kinds of objects—buffers, which create sound, and listeners, which "hear" sound. Put a more silly way, in THX, the audience is listening, but in 3D sound, the audience is a listener. Usually, you'll have several buffers and one listener. To compute the sound coming out of your speakers for a particular buffer, DirectX looks at the position of that buffer relative to the position of the listener.

Tip

If a tree falls in a forest, and no one is around, does it still make a sound? If a 3D sound buffer containing a sample of a tree falling starts playing, and there's no 3D listener, it doesn't make a sound. In 3D sound, you need at least one buffer and one listener to hear anything out your speakers.

For example, if your listener is at position (0,0,0) in your 3D world, and there's a chainsaw buffer at (-10,0,0), the sound will come (mostly) out the left speaker. If the chainsaw buffer is at (-20,0,0), it'll still be mostly out the left speaker, but its volume will be reduced.

Tip

Again, just because a 3D sound buffer is positioned directly left of a listener doesn't mean that sound from that buffer will only come out the left speaker, and the right speaker will be completely silent. In real life, if a sound is generated to your left, you hear it through both ears, but the sound waves arrive at your right ear just a little bit later and softer than at your left, and your brain interprets this to mean that the sound is coming from the left. 3D sound simulates what you'd hear in real life, so sound will come out the right speaker, but it will be softer and slightly delayed (based on mathematical principles of sound perception), so that together both speakers "trick" your brain into thinking the sound is coming from your left. This is an important principle of 3D sound—it's not about just scaling a sound's volume and panning it to one speaker or another; it's about simulating actual sound waves.

The Doppler Effect

If you haven't heard of the term before, imagine you're standing next to some train tracks. In the distance you see a train. It's going fast. Just as the train is about to pass you, it sounds its whistle, at a constant pitch. As the train passes you, the pitch of the whistle appears to drop. That's the Doppler effect, named for the Austrian mathematician and physicist, Christian Doppler, who discovered it and first modeled it mathematically in the nineteenth century.

DirectX Audio can simulate the Doppler effect, but in addition to the positions, it also needs the velocities of the listener and the buffer.

AUDIO CLIP

Listen to Audio Clip 13.1 for an example of the Doppler effect.

Properties of Buffers and Listeners

Now that you've got a grip on all of the concepts, here are the actual variables that DirectX Audio uses to properly simulate 3D sound.

Position

For starters, both listeners and buffers have a position, a coordinate triplet (x,y,z) specifying their position in world space. Note that there are no matrices or local coordinate spaces; you simply tell DirectX Audio, "This chainsaw buffer is at (50,-50,10) in world space."

Velocity

Listeners and buffers also have velocities. These are specified as direction vectors with magnitude. In practice, you can form a velocity vector by using the object's position in the last frame, and its position on the current frame.

Minimum and Maximum Distance (Buffers Only)

A 3D sound buffer has two properties that specify how far away you can hear it from, and how close you have to get until it becomes maximum volume. These two properties are the minimum and maximum distances (see Figure 13.1).

Click To expand
Figure 13.1: The minimum and maximum distances tell DirectX Audio how far away a sound can be heard.

The minimum distance specifies how close you have to get for full volume on the buffer. If you get closer than this, the volume is capped—DirectX Audio won't amplify your buffers. Similarly, the maximum distance is the distance at which the sound fades to nothing. If you're a smidgen closer than this distance, you can just barely hear the sound; if you're slightly farther away, you hear nothing. Skip to the distance factor property section if you want to know how DirectX Audio interpolates between the minimum and maximum volumes of a buffer.

Orientation (Listener Only)

A DirectX Audio listener also has an orientation, so that DirectX Audio knows which direction is "front," and can properly muffle sounds coming from the back. Orientation is specified by two vectors: an up vector and a front vector. If you're making a FPS shooter, this is easy—you can get your orientation vectors from your camera.

Sound Cones (Buffers Only)

Buffers don't have orientation, though they have something similar called a sound cone. A sound cone is akin to a directional light's light cone. Essentially, you specify an inner and outer cone. A listener inside the inner cone will hear the sound at the maximum volume possible based on his distance from the sound (see Figure 13.2).

Click To expand
Figure 13.2: The various buffer properties that influence the volume of a sound.
Tip

By default, a sound's cone is set up so that the outer cone and inner cone are the same, and the angle of both cones is 360 degrees. If you think about it, when the angle of the sound cone is 360 degrees, it's not really a sound cone any more. Instead, it becomes a sound sphere, and the sound no longer has any orientation—it's like a point light.

A listener in between the inner and outer cones will experience two volume cuts—one based on how far away he is from the object, and one based on his position between the inner and outer cone. (Actually, there might be more than just two volume cuts, as the listener's orientation and a few other things are also thrown into the mix, but the basic idea is that a sound cone, and specifically the listener's position between the inner and outer sound cone, is just one more factor in determining the overall volume of the sound.)

Rolloff Factor (Listener Only)

The rolloff factor is a property unique to the listener (buffers don't have rolloff factors). It specifies the attenuation of all the buffers, in other words, how quickly sounds fade out as the listener moves away from them, or how quickly they fade in when the listener travels toward them. DirectX Audio's rolloff factor is actually a multiplication factor specifying how much to deviate from the real world. With a rolloff factor of one, sound volumes in the game are exactly as they would be in real life. A rolloff factor less than one (but greater than zero—negative values are illegal) specifies that sound attenuation happens less in DirectX Audio than it does in the real world. In other words, a value of 0.5 means that you can hear sounds from farther away.

Specifically, if a sound becomes inaudible at 40 meters away in the real world, in DirectX Audio's virtual world it becomes inaudible at 80 meters away. If the rolloff factor is zero, sound volumes don't change based on distance—you can either hear something, or you can't.

Distance Factor (Listener Only)

There's also a property of the listener called the distance factor. Similar to the rolloff factor, the distance factor is a multiplier for the unit of measure used in the game. By default, DirectX Audio assumes that all units are in meters. That is, if you have a listener at (10,0,0) and a sound at (40,0,0), DirectX Audio interprets that as "the buffer is 30 meters away from the listener." You need to make sure that if DirectX Audio thinks something is 30 meters away, then it appears on the screen as if it were 30 meters away. Sure, you could do this by modeling your game world using meters, but if you've already modeled using some other real-world units, you'll need to set this scaling value appropriately. For example, if all of your models are in feet (that is, the two local coordinates for a yardstick in your virtual world are (-1.5, 0, 0) and (1.5, 0, 0)), you'll need to set your scale value to 0.3048 (the number of feet in a meter). The distance factor is mainly used when DirectX Audio calculates the Doppler effect.

Tip

At first thought, it may seem weird that DirectX Audio made the distance and rolloff factors a property of the listener, and not the buffers. After all, what if you want to set distance factors on a per-buffer basis, so that certain buffers could be heard further away than other buffers?

To simulate that, you should use buffer sound cones. Think of the distance factor and rolloff factor as global scaling values you can use in a pinch. You can use the distance factor to avoid re-scaling all of your graphics, and you can use the rolloff factor for special effects. For example, if your player finds a "Helmet of Keen Hearing" powerup, you can pull the rolloff factor closer to zero to simulate superhuman hearing abilities.

Doppler Factor (Listener Only)

Like the distance and rolloff factors, the Doppler factor is a multiplier that scales all Doppler effects. This is useful when you want to create the sensation of speed without actually having to make a big world and put fast moving things in it. You can create a smaller world, with slower moving objects, and just scale up the Doppler factor so that they sound like they're going really fast.

Processing Mode (Buffers Only)

DirectX Audio allows you to feed it the buffer properties you've just learned about in one of two modes: normal mode or head-relative mode. You can set the mode for each buffer independently.

In normal mode, you specify position and orientation in world space. That's easy enough.

In head-relative mode, position and orientation are relative to the listener. This means that if your listener is at position (7,3,0), and you tell DirectX Audio that there's a buffer at position (-3,5,0) in head-relative mode, you're saying that the buffer is three units to the left and five units above the listener. DirectX Audio will calculate the actual world position of the buffer to be (4,8,0) (see Figure 13.3).

Click To expand
Figure 13.3: In head-relative mode, buffer positions are relative to the listener.

The mode you should use depends on your sound effect. You may find it easier to use head-relative mode for buffers that are always going to be attached to the listener (imagine those things bolted directly onto the player's head). For example, in a simple flight simulator, you could use head-relative mode for the plane's engines, provided that your player could never rotate his virtual pilot's head in the game—he could only rotate the plane itself. If the player could rotate his own head in the game, you wouldn't want to use head-relative mode for the engines.

Immediate versus Deferred Settings

Every time you change one of the variables of a buffer or the listener, DirectX Audio recalculates things and remixes the final sound output right then and there. This can lead to a huge waste of resources if you're changing a whole bunch of buffers, because every time you change one, DirectX Audio recalculates everything!

To save resources in this situation, you'd want to use deferred settings. With deferred settings, you tell DirectX Audio to hold off on recalculating until you've given it all of the new parameters— definitely the way to go if you're updating all the buffers and the listener every frame. This chapter's sample code will show you how to use deferred settings—essentially, you pass the DS3D_DEFERRED flag when you're calling the methods to set the buffer and listener parameters, and then at the very end you call CommitDeferredSettings to apply all of the changes. It's a slight added hassle, but one that's well worth it to conserve processing power!


PROGRAMMING 3D SOUND

Once you understand the 3D sound concepts, programming 3D sound using DirectX Audio isn't terribly difficult. Here's how to enhance the audio engine to support 3D sound.

C3DSoundListener

The listener is contained in a new class, C3DSoundListener. Here's the class definition:

class C3DSoundListener
{
public:
  friend class CAudioManager;
  virtual ~C3DSoundListener();
  void SetPosition(D3DVECTOR position) {
    m_Props.vPosition = position; RefreshProps();
  }
  D3DVECTOR GetPosition() { return(m_Props.vPosition); }
  void SetVelocity(D3DVECTOR Velocity) {
    m_Props.vVelocity = Velocity; RefreshProps();
  }
  D3DVECTOR GetVelocity() { return(m_Props.vVelocity); }
  void SetOrient(D3DVECTOR OrientTop, D3DVECTOR OrientFront) {
    m_Props.vOrientTop = OrientTop;
    m_Props.vOrientFront = OrientFront;
    RefreshProps();

  }
  D3DVECTOR GetOrientTop() { return(m_Props.vOrientTop); }
  D3DVECTOR GetOrientFront() { return(m_Props.vOrientFront); }
  void SetDistanceFactor(float f) {
    m_Props.flDistanceFactor = f; RefreshProps();
  }
  float GetDistanceFactor() { return(m_Props.flDistanceFactor); }
  void SetRolloffFactor(float f) {
    m_Props.flRolloffFactor = f; RefreshProps();
  }
  float GetRolloffFactor() { return(m_Props.flRolloffFactor); }
  void SetDopplerFactor(float f) {
    m_Props.flDopplerFactor = f; RefreshProps();
  }
  float GetDopplerFactor() { return(m_Props.flDopplerFactor); }
protected:
  // only accessible to friends (i.e., CAudioManager)
  C3DSoundListener();

  CAudioManager *m_Manager;
  void RefreshProps();
  DS3DLISTENER m_Props;
  IDirectSound3DListener8 *m_Listener;
};

There are a lot of accessor functions here for the members of the m_Props structure. The m_Props structure is of type DS3DLISTENER, which is a DirectX-supplied structure that contains all of the properties for a listener.

Other than the m_Props member and all of its associated accessor functions, all this C3DSoundListener class contains is a protected RefreshProps method, a pointer to the CAudioManager, and a pointer to the DirectSound listener interface, m_Listener.

Notice that each set accessor method calls RefreshProps after it sets the appropriate structure value. The RefreshProps method sends the m_Props structure to DirectX Audio by calling the SetAllParameters method of the listener interface:

void C3DSoundListener::RefreshProps()
{
  HRESULT hr = m_Listener->SetAllParameters(&m_Props, DS3D_DEFERRED);
  ThrowIfFailed(hr, "Error calling SetAllParameters");
}

The only interesting thing in this little method is the use of the DS3D_DEFERRED flag. This flag tells DirectX Audio to hold off on recalculating the mixing buffers until the CommitDeferredSettings method of the listener interface is called.

Changes to CAudioManager

Since there can only be one listener, I've structured the audio engine so that the audio manager contains and keeps track of the global listener instance. There's a new m_Listener member of CAudioManager, which is a C3DSoundListener. Other code accesses the listener class by calling the GetListener accessor function.

CAudioManager is also responsible for getting the listener interface in the first place. Once it gets this interface, it gives it to the new listener instance by setting its m_Listener member.

Here's how it looks in code—this new code is part of the Init method of CAudioManager:

// get the listener buffer
IDirectMusicAudioPath8 *defaudiopath = NULL;
IDirectSound3DListener8 *listener;
hr = GetPerformance()->GetDefaultAudioPath(&defaudiopath);
ThrowIfFailed(hr, "Can't get default audio path!");

hr = defaudiopath->GetObjectInPath(0, DMUS_PATH_PRIMARY_BUFFER,
  0, GUID_NULL, 0, IID_IDirectSound3DListener8, (LPVOID*) &listener);
ThrowIfFailed(hr, "can't get listener on default audio path!");
// tell our listener object to use this listener interface
GetListener().m_Listener = listener;
GetListener().RefreshProps();
Commit3DSoundParameters();
SAFE_RELEASE(defaudiopath);

The code just grabs the listener interface by getting an interface to the default audiopath, then calling the GetObjectInPath method of that interface. In DirectX Audio, there's only one listener interface, so it doesn't matter which audiopath you get it from.

Once it has the listener interface, the code hands it to the C3DSoundListener object, then calls the RefreshProps method of that object, so that the object has a chance to send its default values to DirectX Audio. Also, because the listener's RefreshProps method set the parameters using the deferred flag, the code immediately calls Commit3DSoundParameters to force DirectX Audio to commit the newly-sent parameters and recalculate the mix. Once all that's complete, the code cleans up its mess by releasing the audiopath interface. Note that the listener interface isn't released; the listener object now owns that interface, and will release it when the listener's destructor is called.

The Commit3DSoundParameters method is a one liner:

void CAudioManager::Commit3DSoundParameters() {
  GetListener().m_Listener->CommitDeferredSettings();
}

This method just calls the CommitDeferredSettings method of the listener interface.

C3DSoundEffect

One new class down, one to go. C3DSoundEffect represents a 3D sound buffer:

class C3DSoundEffect : public CDirectMusicSegment
{
public:
  C3DSoundEffect(CAudioManager *mgr);
  virtual ~C3DSoundEffect();
  // override segment's function to play out our own audiopath
  virtual bool Play() {
    return(CDirectMusicSegment::Play(NULL, false, m_AudioPath));
  }
  virtual bool Play(CSoundInstance *newinst, bool control) {
    return(CDirectMusicSegment::Play(newinst, control, m_AudioPath));
  }
  virtual bool PlayAsPrimary(CSoundInstance *newinst) {
    return(CDirectMusicSegment::Play(newinst, true, m_AudioPath));
  }

  // properties
  // accessor functions snipped to save space
protected:
  void RefreshProps();
  DS3DBUFFER m_Props;
  IDirectMusicAudioPath8 *m_AudioPath;
  IDirectSound3DBuffer8 *m_3DBuffer;
};

Again, most of it is just accessor functions for the m_Props structure (which for this class is of type DS3DBUFFER). However, there are some new things.

First, notice that C3DSoundEffect derives from CDirectMusicSegment, but that it overrides CDirectMusicSegment's Play methods with its own. I did this because, unlike a normal segment, a 3D sound buffer needs to explicitly provide a 3D audiopath every time it plays a sound.

This meant that I needed to go into CDirectMusicSegment and change things around so that you could pass in your own audiopath pointer, and if you did, the CSoundInstance that eventually was created wouldn't release the audiopath pointer. I'm not going to go through those changes line by line, because they're fairly self-explanatory, but you do need to know that they occurred.

This means that the C3DSoundEffect class needs to create and manage its own audiopath. Here's how the audiopath creation looks (this is taken from the C3DSoundEffect constructor):

// create our 3D audiopath
HRESULT hr = m_Manager->GetPerformance()->CreateStandardAudioPath(
  DMUS_APATH_DYNAMIC_3D, CAudioManager::NUMCHANNELS,
  TRUE, &m_AudioPath);
ThrowIfFailed(hr, "Couldn't create 3D audio path!");
// get our 3D sound buffer from that new audiopath
hr = m_AudioPath->GetObjectInPath(DMUS_PCHANNEL_ALL,
  DMUS_PATH_BUFFER, 0, GUID_NULL, 0, IID_IDirectSound3DBuffer8,
  (LPVOID*)&m_3DBuffer);
ThrowIfFailed(hr, "Couldn't get buffer from audio path!");

This code creates a standard 3D audiopath (note the use of the DMUS_APATH_DYNAMIC_3D flag), then queries it for an interface to the 3D DirectSound buffer contained within it.

C3DSoundEffect also has a RefreshProps method that's identical to the listener's, except it's operating on a DirectSound buffer instead of a listener.


THE DEEP SPACE SAMPLE PROGRAM

For this chapter, I wanted to create a sample program that demonstrated sound in an actual 3D world. Sure, I could have just provided a console app, but that doesn't really show you how to integrate 3D sound into your game. To fully understand 3D sound, you need to see the code in its native habit—working hand-in-hand with the 3D graphics code to create a convincing reality.

So, I stole from myself. Ch13p1 is based on a sample program I created for my last book, Special Effects Game Programming Using DirectX 8.0. Readers of that book will recognize it as a slightly modified version of the Ch20p3_ComplexExplosion sample program of Chapter 20.

I'm not going to spend time explaining how the 3D graphics principles of this sample program work, but to properly understand what's going on in the sample, you should know about things like skyboxes, world transform matrices, and X file meshes. By the way, if you happen to need a good book on special effects, my book… wait, what's that? You're right! You didn't BUY MY BOOK to hear a sales pitch, much less be assaulted with subliminal advertising! Okay, never mind, I'll keep going.

Run the app, and you'll see something resembling that of Figure 13.4.

Click To expand
Figure 13.4: A screenshot of this chapter's sample program.

Essentially, you're in deep space, and you can move around freely in all directions. There's a spaceship in front of you that's emanating two 3D sounds: an engine sound (engine.wav) that loops continuously, and a beep sound (beep.wav) played periodically.

Use the W, A, S, and D keys to move forward, rotate left, move backward, and rotate right, respectively. The Q and Z keys will move you up and down, the B key will toggle the ship's beeping (it gets irritating), and the G key will toggle your warp drive (allowing you to move faster and more easily hear the Doppler effect).

If you fly around in this sample program, you can hear 3D sound in all its glory. For starters, notice that as you get closer to the ship, the sound volume increases, and that it pans and appears to really be coming from the spaceship. Notice also the Doppler effects; you should hear the spaceship's beep and engine sounds change pitch as you fly toward it, past it, and away from it.

The Ch13p1_3DSound sample program is based on the D3D sample application framework, which means all of the action occurs inside a CMyD3DApplication object. The various members of this object are called at the right times by the D3D sample application framework.

Loading the Sounds

In the OneTimeSceneInit method of CMyD3DApplication, there's some code to load the wave files off disk and set up their default behavior:

m_Beep = GetAudioManager()->Load3DSound("beep.wav");
m_Engine = GetAudioManager()->Load3DSound("engine.wav");
CDirectMusicSegment *engineseg =
  dynamic_cast<CDirectMusicSegment *>(m_Engine.Get());
engineseg->SetRepeats(DMUS_SEG_REPEAT_INFINITE);
// set range on our engine and our beep
C3DSoundEffect *beep =
  dynamic_cast<C3DSoundEffect *>(m_Beep.Get());
C3DSoundEffect *engine =
  dynamic_cast<C3DSoundEffect *>(m_Engine.Get());
if (beep) {
  beep->SetMaxDistance(100);
  beep->SetMinDistance(10);
}
if (engine) {
  engine->SetMaxDistance(80);
  engine->SetMinDistance(10);
}

GetAudioManager()->Commit3DSoundParameters();

This chunk of code loads the two wave files into two 3D sound effects: m_Engine and m_Beep. It then sets some 3D sound parameters on both objects—notice that the code sets things up so that the engine sound effect goes away completely when you're 80 units away, but that the beep sticks around for another 20 units. Both sounds reach full intensity when you're within 10 units of the spaceship.

Notice also that the code casts m_Engine to a CDirectMusicSegment pointer, and then calls a new method, SetRepeats, to make the engine sound loop infinitely. SetRepeats is just a wrapper function for the IDirectMusicSegment method of the same name.

Once the parameters are set, the code tells the audio manager to commit them, and everything's ready for the game loop to be started.

Updating the Buffers and the Listener

Every frame, the positions of the buffer and the listener must be updated, along with some other properties. Here's that code, taken from the FrameMove method of CMyD3DApplication:

m_BeepTimer += m_fElapsedTime;
if (m_BeepActive && m_BeepTimer > BEEPINTERVAL) {
  m_Beep->Play();
  m_BeepTimer = 0;
}
// set beep position to spaceship position
C3DSoundEffect *beep = dynamic_cast<C3DSoundEffect *>(m_Beep.Get());
if (beep) {
  D3DVECTOR v;
  v.x = 0;
  v.y = 4;
  v.z = 0;
  beep->SetPosition(v);
}
// set listener position to camera position
C3DSoundListener &listener = GetAudioManager()->GetListener();
listener.SetPosition(m_Camera.GetPosition());

// set listener orientation
D3DXVECTOR3 lookat = D3DXVECTOR3(
  m_Camera.GetViewMatrix()._13,
  m_Camera.GetViewMatrix()._23,
  m_Camera.GetViewMatrix()._33);
D3DXVECTOR3 up = D3DXVECTOR3(
  m_Camera.GetViewMatrix()._12,
  m_Camera.GetViewMatrix()._22,
  m_Camera.GetViewMatrix()._32);
D3DXVECTOR3 nlookat, nup;
D3DXVec3Normalize(&nlookat, &lookat);
D3DXVec3Normalize(&nup, &up);
if (nup == nlookat) { nup = D3DXVECTOR3(0,1,0); }
listener.SetOrient(nup, nlookat);
// set listener velocity based on previous frame position and
// current position
m_ListenerVelocity.x =
m_Camera.GetPosition().x - m_LastListenerPosition.x;

m_ListenerVelocity.y =
  m_Camera.GetPosition().y - m_LastListenerPosition.y;
m_ListenerVelocity.z =
  m_Camera.GetPosition().z - m_LastListenerPosition.z;
listener.SetVelocity(m_ListenerVelocity);
m_LastListenerPosition = m_Camera.GetPosition();
listener.SetDopplerFactor(DS3D_MAXDOPPLERFACTOR);
listener.SetDistanceFactor(10);
GetAudioManager()->Commit3DSoundParameters();

This code's doing a lot of tasks, but none of them are terribly complex. It starts by incrementing a timer value and determining if it's time to play the beep sound. Next, it moves the beep 3D sound buffer to the spaceship's current position (which is hard-coded at (0,4,0)). It does the same thing for the listener, and then turns its attention to setting the listener's orientation.

Fortunately, the camera class contains a view matrix, so it's a simple matter of extracting the up and lookat vectors, normalizing them, and then calling SetOrient to make them the up and front vectors of our listener. Next, the code calculates the listener's velocity by looking at the current camera position, and subtracting from that the position of the camera on the last frame.

Finally, there's some fudging. The code sets the Doppler factor to maximum, and scales the distance by a factor of 10, in order to make the Doppler effects stand out (so you can hear them). You should experiment with different values for these two factors to get a feel for how to create a particular aural environment for your own game.

The code finishes by committing all of the parameters it's changed. Nothing to it!


CONCLUSION

I think, all in all, the most complicated part of implementing 3D sound is learning the concepts behind it, and the proper way to drill down through DirectMusic and interface with 3D DirectSound buffers directly. Thankfully, by learning these concepts and techniques, you spare yourself from having to write tons of code to perform the required mathematical processes to make a sound appear to exist in a real, 3D world. Not that the math required isn't fascinating—it is, and if you're interested, you can learn a bit more about it by following some of the links I've provided on your CD.

You now have a basic understanding of 3D sound. In the next chapter, you'll extend that knowledge by learning how to use some third party APIs that perform advanced 3D sound processing functions, such as obstruction and reflection.


Chapter 14: 3D Sound Using OpenAL

Download CD Content

OVERVIEW

In the last chapter, you experienced the glory of using 3D sound to create a truly believable 3D world. Granted, it was a pretty boring 3D world, with only one object, but still, you get the idea.

In this chapter, you'll learn how to accomplish that same feat, only without using DirectX. Instead, you'll use a library called OpenAL, or the Open Audio Library.

OpenAL is an API created by a consortium of people and companies to be an open, crossplatform API for 3D sound. It's the sister API to OpenGL for 3D graphics rendering, and the design and conventions used by OpenAL are the same as what you'd see in OpenGL.

Creative Labs is the company currently doing the most to support OpenAL. They, along with a now-defunct company named Loki Entertainment Software, spearheaded the OpenAL effort, and today you can find the OpenAL SDKs hosted off their site (developer.creative.com).

OpenAL is a very well-written cross-platform API, designed to do one thing—3D sound. Conceivably, you can do all of your game's audio using OpenAL, provided you're not heavily into dynamic music (or don't mind writing the code to support that on OpenAL).


DIFFERENCES BETWEEN OPENAL AND 3D SOUND IN DIRECTX AUDIO

If you haven't read the concepts from Chapter 11, you should stop reading this and go back and learn those concepts. OpenAL shares 90 percent of the concepts used by DirectX Audio's 3D sound implementation.

Sources, Buffers, and Listeners

The OpenAL functionality revolves around three main kinds of objects: sources, buffers, and listeners. Listeners in OpenAL and DirectX Audio are conceptually the same thing; in OpenAL audio, there's still only one listener, which represents the ears of the in-game player.

However, OpenAL has a new object—a source—as well as a different meaning for the term "buffer." An OpenAL buffer is the PCM sampled data itself, whereas a source describes an instance of a buffer, a position from which a certain buffer emanates (see Figure 14.1).

Click To expand
Figure 14.1: Multiple sources can reference the same buffer.

OpenAL uses these constructs so that if, for example, your game has three different helicopters, you don't need to load three copies of the helicopter WAV file into memory. You load the WAV into a buffer, and then create three sources that reference that same buffer.

From a different angle, both APIs have this capability. In OpenAL, you have multiple sources referencing a single buffer, which contains the actual sound data. In DirectX Audio, you have audiopaths; you could say that an OpenAL buffer is equivalent to a DirectMusic segment, and that an OpenAL source is equivalent to an audiopath off that segment.

It Isn't COM Anymore

The DirectMusic API is driven by COM, which means by now you're used to dealing with acquiring and releasing interfaces. OpenAL is not COM. Since it's cross platform, it relies on the triedand-true method of API interfaces—a big list of functions, enums, and structures.

OpenAL (and OpenGL too, for that matter) adheres to a strict naming convention for its functions. If you look in the API reference, you'll see several functions whose names are almost identical—except one will end in f, another will end in fv, and a third might end in i. These correspond to the type of data the function takes as arguments.

For example, there are four functions with names alListenerf, alListener3f, alListenerfv, and alListeneri (by the way, all OpenAL functions start with al). All four of these functions do the same thing—they set the properties of a listener. However, alListenerf takes one floating-point argument, alListener3f takes 3 floats as arguments (a vector), alListenerfv takes an array of floats (a vector in the C++ sense of the word), and alListeneri takes an integer. This seems weird, but it becomes second nature once you get the hang of it, and makes it more difficult to accidentally get your data types mucked up.

Buffer Properties

The available properties for a buffer are summarized in Table 14.1. As you can see, they all relate to the properties of the wave file originally loaded into the buffer.

Table 14.1: OpenAL Buffer Properties
Property | Description
AL_FREQUENCY | Frequency of the WAV file.
AL_BITS | Bits per sample of the WAV file.
AL_CHANNELS | Number of channels (1=mono, 2=stereo). Note that if this is greater than one, the buffer will not be positioned aurally when played (in other words, if you want your sound to appear to come from the right place, use mono samples).
AL_SIZE | Size (in bytes) of the buffer.
AL_DATA | Memory address data was originally loaded from (not particularly useful since you probably freed the data once it was copied into the buffer).

All of these values are read-only, set when you call alBufferData to pump a buffer full of sample data. Since all of these properties are integers, you can read them by calling the OpenAL function alGetBufferi.

Source Properties

OpenAL sports several properties for its sources (see Table 14.2). Most of these you're already familiar with from the last chapter, but there are some new ones, too.

Table 14.2: OpenAL Source Properties
Property | Description
AL_PITCH | A pitch multiplier for this source (always greater than zero). Values between zero and one will lower the pitch; values above one will raise it.
AL_GAIN | Volume of the source. Values between zero and one will lower the volume; values above one will raise it.
AL_MAX_DISTANCE | Distance at which the listener can no longer hear this source.
AL_ROLLOFF_FACTOR | Rolloff multiplier for this source. Usually one, but can be lowered to values between zero and one for longer rolloff, or raised above one for shorter rolloff.
AL_REFERENCE_DISTANCE | The distance at which the source would lose half its volume (not counting the rolloff factor).
AL_MIN_GAIN | The minimum gain (volume) of this source.
AL_MAX_GAIN | The maximum gain (volume) of this source.
AL_CONE_OUTER_GAIN | Volume when the listener is outside of the source's cone.
AL_CONE_INNER_GAIN | Volume when the listener is inside the source's cone.
AL_CONE_OUTER_ANGLE | Angle of the sound cone. The default is 360, which removes the cone altogether.
AL_POSITION | 3-float vector specifying the position of the source.
AL_VELOCITY | 3-float vector specifying the velocity of the source (used for Doppler effects).
AL_DIRECTION | 3-float vector specifying the direction the cone of the source is pointed.
AL_SOURCE_RELATIVE | If set to AL_TRUE, the position of the source is relative to the listener (like DirectX Audio's head-relative mode).
AL_LOOPING | If set to AL_TRUE, the source loops continuously.
AL_BUFFER | The ID of the buffer the source plays.
AL_SOURCE_STATE | Current state of the source (AL_INITIAL, AL_STOPPED, AL_PLAYING, or AL_PAUSED).
AL_BUFFERS_QUEUED | The number of buffers queued for playback on this source (this property is read only—you can set it indirectly by calling alSourceQueueBuffers).
AL_BUFFERS_PROCESSED | The number of queued buffers that have already been played (again, this is a read-only property).

You can set these source properties by using the OpenAL functions alSourcef, alSourcefv, alSource3f, and alSourcei. Retrieve them by calling alGetSourcef, alGetSourcefv, alGetSource3f, or alGetSourcei, depending on the data type of the property.

Tip

Be careful—OpenAL uses a left-handed coordinate system, whereas Direct3D is right-handed. Put another way, in OpenAL, negative z values descend into the screen, whereas in Direct3D, negative z values go out of the screen.

Listener Properties

The OpenAL listener also has a handful of properties, summarized in Table 14.3. You can set these properties by calling the alListener family of functions (alListenerf, alListener3f, alListenerfv, and alListeneri). Retrieve their values by calling alGetListenerf, alGetListener3f, alGetListenerfv, or alGetListeneri.

Table 14.3: OpenAL Listener Properties
Property | Description
AL_GAIN | Master gain value (this is sort of like a volume knob; raise it above one, and all sound will be amplified, set it to a value between zero and one and all sound volumes will be lowered).
AL_POSITION | 3-float vector specifying the listener's absolute position in the 3D world.
AL_VELOCITY | 3-float vector specifying the velocity of the listener for Doppler effects.
AL_ORIENTATION | A two vector, 6-float property specifying the orientation of the listener. The first three floats form the lookat vector, the last three form the up vector.


INTEGRATING OPENAL INTO THE AUDIO ENGINE

Adding OpenAL support to the audio engine is pretty straightforward. This section will guide you through the process of installing the OpenAL SDK and using it to create a handful of new AudioEngine classes.

Installing OpenAL

To begin the installation of the OpenAL SDK, go to your CD's goodies folder and double-click the OpenALSDK.exe file contained in the OpenAL goodies folder. Navigate through the wizard, installing the SDK at C:\program files\creative labs\OpenAL 1.0 SDK.

Tip

If you're using OpenAL, you should include as part of your game distribution the OpenAL redistributables, found in the dll\Redist folder of your OpenAL SDK. The files in this package will allow anyone who needs it to install the OpenAL runtime files on his system.

The SDK includes a few sample programs and some documentation, both of which I encourage you to look at later. For now though, to use OpenAL, all you need to do is make sure that the OpenAL Include and Lib folders are in the appropriate search paths for any projects that use OpenAL. The sample program for this chapter expects to find the OpenAL include files at ..\..\OpenAL 1.0 SDK\Include, and the lib files at ..\..\OpenAL 1.0 SDK\lib.

Structure of the OpenAL Library

The OpenAL SDK is divided into two different libraries. First, there's the core OpenAL library (OpenAL32.lib), and second, there's the OpenAL Utility Library (ALut.lib). Usage of the OpenAL Utility Library is optional, but most of the time you'll want to use it, since it makes working with OpenAL a bit easier.

The OpenAL library functions, however, are grouped into three different categories. Most of the functions fall into the core OpenAL category. All of the core OpenAL category functions begin with al. However, there are also some functions that fall into the OpenAL utilities category (and begin with alut), and some context functions not discussed here that fall into the OpenAL context category (and begin with alc). The context functions are useful if you're working with more than one audio device, or if you want explicit control over the single audio device you're using. Most games don't have to worry about them, instead relying on the OpenAL utility library to automatically select the best context and use that.

New OpenAL AudioEngine Classes

Supporting OpenAL in the audio engine means creating a handful of new classes. First and most important, there's COpenALManager, a new class that's the OpenAL equivalent of CAudioManager. I did things this way so that the OpenAL section of the engine would remain completely independent from the other parts that use DirectX.

Here's the definition of COpenALManager:

class COpenALManager
{
public:
  COpenALManager();
  virtual ~COpenALManager();
  virtual void Init();
  virtual void UnInit();
  COpenALBufferPtr CreateBufferAndLoadWave(std::string filename);
  COpenALSourcePtr CreateSource(COpenALBufferPtr linktobuffer);
  COpenALListener &GetListener() { return(m_Listener); }
protected:
  static COpenALListener m_Listener;
};

There's not much in it—just methods to initialize and uninitialize OpenAL, a couple methods to create buffers and sources, and an OpenAL listener accessed via the GetListener method.

The code that uses the OpenAL section of the audio engine is responsible for instantiating the actual COpenALManager class, as well as providing an accessor function to it (just like for CAudioManager).

The other new classes are COpenALBuffer, COpenALSource, and COpenALListener, all of which you'll learn about in the following sections.

Note also that COpenALBuffer and COpenALSource both have smart pointer classes (COpenALBufferPtr and COpenALSourcePtr). These smart pointers are the same as the ones you've seen before; they're derived from the CRefCountPtr template and specialized to work with COpenALBuffers and COpenALSources. These smart pointers free us from having to remember to delete buffers and sources when we no longer need them—the pointers will do it automatically.

Initializing and Uninitializing OpenAL

Now that you have OpenAL installed, and you have a brief idea of the new class structure, you can learn how to write code that uses it. Take a look at Figure 14.2, which shows an overview of using OpenAL.

Click To expand
Figure 14.2: Typical flow when using OpenAL

All code that uses OpenAL must begin by initializing the OpenAL library, and end by uninitializing it. The easiest way to do this is by calling the OpenAL utility functions alutInit and alutExit. Here's how it looks in code, in the Init and UnInit methods of COpenALManager:

The alutInit function takes two parameters (argc and argv), but currently they're not used, so you can just pass 0 for argc and NULL for argv. The alutExit function doesn't take any parameters.

void COpenALManager::Init()
{
  alutInit(0, NULL);
  alGetError(); // clear error code
}
void COpenALManager::UnInit()
{
  alutExit();
}

Notice that the code clears the error value immediately after initializing OpenAL. This brings us to our next topic, namely, how to error check in OpenAL.

Error Checking in OpenAL

There are two main ways you can detect errors when you call OpenAL functions. First, and most obvious, some functions will return values that denote error (if a function returns AL_FALSE, something bad has happened).

Second, you can use the alGetError function to figure out if the last thing you did errored out. The alGetError function returns an enumerated value, which hopefully is AL_NO_ERROR, but could be one of the values shown in Table 14.4 if something's gone wrong.

Table 14.4: OpenAL Error Codes
Return Value | Meaning
AL_INVALID_NAME | You gave a bad name as input.
AL_INVALID_ENUM | You passed an invalid integer value to a function that expected a value defined by an enumeration.
AL_INVALID_VALUE | The value you passed doesn't make any sense, or is illegal (for example, you passed a negative number when specifying the pitch of a source).
AL_INVALID_OPERATION | You called something at the wrong time, or in the wrong way.
AL_OUT_OF_MEMORY | Memory allocation failed.

Creating and Destroying Buffers

The first thing you'll usually do to set up the sound for your game is to load the WAV files you need and create your buffers. This is accomplished by calling the long-winded, but very descriptively named COpenALManager method: CreateBufferAndLoadWave. Here's how it looks:

COpenALBufferPtr COpenALManager::CreateBufferAndLoadWave(
  std::string filename)
{
  COpenALBuffer *newbuffer = new COpenALBuffer();
  unsigned int newid;
  alGenBuffers(1, &newid);
  if (alGetError() != AL_NO_ERROR) {
    Throw("Error generating AL buffer.");
  }
  newbuffer->m_ID = newid;
  newbuffer->LoadWave(filename);
  return(COpenALBufferPtr(newbuffer));
}

Here you can see the code creating a new COpenALBuffer instance, and then calling alGenBuffers. The alGenBuffers function takes two arguments: the number of buffers you want to create (one in this case) and a pointer to an array of integers (or just a single integer, if you're creating one buffer) that will receive the OpenAL IDs of the new buffer(s). Here the code creates one buffer and stores its ID in the newid variable, which it then gives to the new COpenALBuffer class. After the class has got its ID, the OpenAL manager calls the buffer's LoadWave function, which loads the wave specified into the buffer. Once that's done, the code creates and returns a new smart pointer for the new buffer.

Here's how LoadWave looks:

void COpenALBuffer::LoadWave(std::string filename)
{
  unsigned char *data = NULL;
  alutLoadWAVFile(const_cast<char *>(filename.c_str()),
    &m_Format, (void **)&data, &m_Size, &m_Freq, &m_Loop);

  // copy wave data into buffer
  alBufferData(m_ID, m_Format, data, m_Size, m_Freq);
  if (alGetError() != AL_NO_ERROR) {
    alutUnloadWAV(m_Format, data, m_Size, m_Freq);
    Throw("Error loading WAV data into buffer.");
  }
  // free original data
  alutUnloadWAV(m_Format, data, m_Size, m_Freq);
}

The real work here is done by the OpenAL utility function alutLoadWAVFile. This function takes two inputs—a filename and an address of a pointer. It allocates memory, loads the wave into that memory, sets the pointer to it, and returns the properties of the wave file. The code hands all this stuff off to the alBufferData function, which copies the wave data out of data and into the OpenAL buffer with the specified ID (m_ID). Once that's done, the original WAV data is unloaded via a call to the OpenAL Utility function alutUnloadWAV.

When the buffer is no longer needed, its associated smart pointer (COpenALBufferPtr) takes care of deleting the object. As part of the deletion process, COpenALBuffer's destructor will be called. The destructor looks like this:

COpenALBuffer::~COpenALBuffer()
{
  if (alIsBuffer(m_ID) == AL_TRUE) alDeleteBuffers(1, &m_ID);
}

Essentially, it asks OpenAL if its ID (m_ID) exists, and if it does, it tells OpenAL to delete it, by calling the alDeleteBuffers function.

Creating and Destroying Sources

Once the buffers have been created and have had WAV data put into them, it's time to create a few sources that reference those buffers. This is done by calling the CreateSource method of COpenALManager:

COpenALSourcePtr COpenALManager::CreateSource(
  COpenALBufferPtr linktobuffer)
{

  COpenALSource *newsource = new COpenALSource();
  unsigned int newid;
  alGenSources(1, &newid);

  if (alGetError() != AL_NO_ERROR) {
    Throw("Error generating AL source.");
 }
 newsource->m_ID = newid;
 newsource->LinkToBuffer(linktobuffer);
 return(COpenALSourcePtr(newsource));
}

The pattern here is the same as creating a buffer. The code makes a new COpenALSource class, calls alGenSources to create the OpenAL source, takes the ID supplied by that function and gives it to the new class, and then calls the LinkToBuffer method of the class to tell the source that it should use the given buffer. It finishes by returning a new smart pointer to the new source.

The LinkToBuffer method of COpenALSource is really just setting the AL_BUFFER property for that source:

void COpenALSource::LinkToBuffer(COpenALBufferPtr linkto)
{
  alSourcei(m_ID, AL_BUFFER, linkto->GetID());
  if (alGetError() != AL_NO_ERROR) {
    Throw("Error linking to OpenAL buffer.");
  }
  m_Buffer = linkto;
}

Notice that the COpenALSource class keeps a pointer to the buffer it's linked to (m_Buffer). This guarantees that a buffer won't be deleted until all sources that use it have also been deleted (or have been linked to another buffer). Again, the power of smart pointers makes this really easy.

Source destruction looks just like buffer destruction—it happens in the destructor and calls alIsSource to make sure the source's ID is valid before calling alDeleteSources to purge the source from OpenAL.

Playing Sources

To play a source, call the PlayFromStart method of COpenALSource. There are also additional methods to pause, play from pause, stop, and rewind sources. These use the alSourcePlay, alSourcePause, alSourceStop, and alSourceRewind OpenAL calls, and are easy enough to follow just by looking at the code.


THE DEEP SPACE SAMPLE IN OPENAL

This chapter's sample program is exactly the same as last chapter's, only it uses OpenAL instead of DirectX Audio. Conceptually, the tasks the game has to perform are very similar to what it had to do for DirectX Audio's 3D Sound. The game creates the buffers and sources inside the OneTimeSceneInit method, then sets the positions and velocities of everything from within the FrameMove method. The actual function calls are a bit different, but you should be able to follow what it's doing by consulting both this chapter and the last. You can find source code for this chapter on your CD, in the Ch14p1_3DOpenALSound folder; the code for the last chapter is in the Ch13p1_3DSound folder.


CONCLUSION

So which one's better? On a feature-by-feature basis, the two APIs are neck-to-neck. When it comes to 3D sound, you'll find that in general, if OpenAL has a particular feature, the other has something similar. It's not like you're going to be longing for a crucial capability on one API or the other.

OpenAL has a critical advantage over DirectMusic: whereas 3D sound in DirectX Audio is bound to the Windows platform, OpenAL currently exists on both Windows and Macintosh, and there's nothing stopping it from existing on Linux, UNIX, and other platforms. If you're looking to make your game cross platform, and still want 3D audio, OpenAL is a perfect choice.

Conversely, if you're already tied to the Windows platform (because, say, you're using DirectMusic's dynamic music capabilities), then there isn't a lot of impetus for you to spend time integrating OpenAL. If you've already got the DirectMusic infrastructure in place and working, you'll probably just want to stick with using 3D sound through it.


Chapter 15: Advanced Topics in 3D Sound

Download CD Content

OVERVIEW

The previous two chapters taught you how to use two different 3D sound APIs to achieve basic 3D positional effects. That's a great skill to know, but there's more to 3D sound than that.

This chapter will take a look at some advanced techniques for 3D sound. It will focus mainly on how to use environmental modeling to add some flavor to your 3D environments, but it will also touch on two advanced technologies: Creative's Environmental Audio Extensions (EAX) and Dolby technologies.


ENVIRONMENTAL MODELING

3D sound is more than just sources and listeners. The last two chapters left one very important variable completely out of the picture: the environment in which the sound is occurring. For example, using knowledge from the last two chapters, you could create some realistic footstep sounds. Attach a listener at your player's head, and one sound source on each foot, and play the sound based on the animation of the leg—no problem. But there is still an element missing. Footsteps on the marble floor of a huge library's atrium sound very different from footsteps on the marble floor of a small kitchen. The massive size of the library atrium is going to cause the sound waves to behave very differently than they would in a small kitchen.

Thankfully, there are APIs that can model these environmental effects and come up with reasonable approximations of how a given environment would change a sound. If you provide footsteps_on_marble.wav, the environmental effects processing can make it sound like a footstep on marble in a huge library or a small kitchen.

Environment Modeling Using I3DL2

Reverberation is arguably the most important property of an environment. In this section, you'll learn more about how to use reverberation to convey a particular environment, and you'll learn how to use I3DL2 to implement reverberation effects inside your game.

What Is I3DL2?

I3DL2 stands for Interactive 3D Audio, Level 2. I3DL2 was formed by the Interactive Audio Special Interest Group, an organization made up of audio professionals from all faces of the game development and multimedia industries. It originally started in 1994, and shortly thereafter formed a working group that in 1998 created the IA-SIG Interactive 3D Audio Rendering and Evaluation Guidelines (Level 1). In 1999 they enhanced the standard and released the IA-SIG Interactive 3D Audio Rendering Guidelines (Level 2), or I3DL2.

I3DL2 Terms and Vocabulary

Before you learn how to code environment modeling using I3DL2, you need to get a handle on the language of reverberation. To accurately model an environment, you'll need to understand some of the components that make up the echoes and muffles of an acoustic environment.

Picture a big room (say, a cathedral) with a sound source (say, a grunting orc) and a listener (a valiant hero) separated by some distance (see Figure 15.1).

Click To expand
Figure 15.1: Shows direct path, early reflections, and late reverb.

When that orc grunts, the valiant hero is going to hear several distinct wave types. First, he will hear the sound waves coming directly from the orc (path 1). This is called the direct path. Second, he will hear the sound waves that started in the same general direction but bounced off a wall on the way (path 2)—these are called early reflections. Finally, the valiant hero will hear the sound waves that started off in the opposite direction and bounced off many different walls. Collectively, these sound waves are called the late reverberation. Figure 15.2 shows a graph of these three distinct wave types.

Click To expand
Figure 15.2: A graph of the three distinct wave types.

Of course, this is oversimplified, but the basic model is the same. To model an environment in I3DL2, you adjust variables inside this model. For example, in I3DL2 there's one particular variable that controls how much time passes between the listener hearing the direct path of the sound, and hearing the first early reflection. To simulate a large room, you'd increase this value, because in a large room the sound waves following path 2 have to travel far before they hit a wall. To simulate a smaller room, you'd make the time difference between direct path and early reflection very small, possibly so small that the listener can't tell the difference. I3DL2 lets you adjust the delays and volumes of all three paths (direct path, early reflections, and late reverberation).

Also, when sounds are reflected, the high frequencies behave differently than the low ones. Specifically, the high frequencies fade out much faster. Also, different rooms reflect sounds with different diffusion and density values. I3DL2 lets you adjust all of these parameters to simulate a wide variety of acoustical environments.

I3DL2: Just Another Effect

In DirectX Audio, I3DL2 is implemented as an effect, through an effect interface named IDirectSoundFXI3DL2Reverb8. This means you use it the same way you use any effect—the same way you learned back in Chapter 12. This is great, because it makes using I3DL2 very simple—you attach this effect to your source sound, set its parameters, and the effect will take care of morphing your sound based on those parameters.

To integrate the I3DL2 effect into the audio engine, you need a new effect class. Here's its declaration:

class CI3DL2Effect : public CEffect
{
public:
  CI3DL2Effect() { ZeroMemory(&m_Params, sizeof(DSFXI3DL2Reverb)); ClearPreset(); }
  virtual ~CI3DL2Effect() { }
  virtual void SetAllParameters(IDirectMusicAudioPath8 *audiopath);
  virtual GUID GetGuid() { return(GUID_DSFX_STANDARD_I3DL2REVERB); }
  virtual void SetPreset(DWORD i3dl2preset);
  virtual void ClearPreset() { m_Preset = -1; }
  DSFXI3DL2Reverb m_Params;
protected:
  DWORD m_Preset;
};

Notice that this class, while similar to the other effect classes, has a few additional methods and members. There are two new SetPreset and ClearPreset calls, as well as a new protected member, m_Preset. You'll learn how all these work in the Preset Effects section, coming in a couple of pages.

I3DL2 Reverb Parameters

For now though, take a closer look at the DSFXI3DL2Reverb structure (summarized in Table 15.1). This is the heart of the I3DL2 reverb processing. The variables in this structure describe to the DirectX effect how to morph the sound. Here's what each member means in detail (note that I'm purposely describing them out of order, since some explanations depend on others):

Table 15.1: DSFXI3DL2Reverb Structure Members
Parameter Name | Description
flHFReference | Reference high frequency, in Hertz. Several upcoming parameters operate on high frequencies. This parameter tells DirectX what a high frequency is. For example, if you set this to its default 5,000Hz value, you're telling DirectX that anything above 5,000Hz should be considered a high frequency.
lRoom | Room attenuation value, in millibels (mB). This parameter influences how loud the room effect is. Remember, these are attenuations, which means that full volume is zero, and anything less than full volume is a negative number. The default value is –1,000mB (–10 decibels).
lRoomHF | Room attenuation value for high frequencies, in millibels. In real life, different rooms muffle high frequencies differently than low frequencies. Using this variable, you can tell DirectX how much to attenuate high frequencies (frequencies over the flHFReference parameter, above). For example, you could assign this value to –10,000mB, and lRoom to –1,000mB to create an environment that completely muffles high frequencies but lets low ones go through. The effect would be similar to being underwater.
flRoomRolloffFactor | The room rolloff factor tells DirectX how quickly the reverb and reflection effects fade as the listener and source get farther away. Note that this parameter only influences the reverb effects; the rolloff factor for the original sound is still controlled by the listener.
flDecayTime | How long (in seconds) the echoes persist. If you set this to a high value, it'll sound like you're in the Grand Canyon (lots of echoes!). A lower value will simulate muffled rooms without any echoes.
flDecayHFRatio | Ratio of the high frequency decay time to the low frequency decay time. If you set this to 1, high frequency and low frequency echoes will both last for the exact same amount of time. If you set this lower than 1, high frequencies will fade out before lows. For values greater than 1, high frequencies will persist longer than low ones (usually unrealistic). The default value is set to 0.83.
lReflections | Early reflection attenuation relative to lRoom, in mB. Early reflections are the first echoes one hears in a room (see Figure 15.1). This parameter tells DirectX whether the volume of the early reflection effects is the same as lRoom,or whether they're softer or louder. The default is –2,602mB, making them softer.
flReflectionsDelay | Number of seconds that elapse between the direct path sound and the early reflections. Simply put, this is how long DirectX waits after a sound is played before it starts playing echoes. The default is 0.007 seconds. To simulate larger rooms, increase this value; to simulate small spaces, decrease it.
lReverb | Attenuation of the late reverb relative to lRoom, in millibels. Similar to lReflections, this specifies whether reverb effects should be louder or softer than the lRoom attenuation. The default is 200mB, making them slightly louder.
flReverbDelay | The time between the first early reflection and the reverb, in seconds. In other words, this tells DirectX how long to wait once the first reflection has been played before playing the reverb effects.
flDiffusion | Late reverberation delay echo density, in percent. The default is 100%.
flDensity | Late reverberation delay modal density, in percent. The default is 100%.
Tip

In case you're wondering, a millibel (abbreviated mB) is one one/hundreth of a decibel. That is, 100 millibels is the same as one decibel. Several of the upcoming parameters are attenuations measured in millibels. If you specify an attenuation of –50 mB, you're specifying a volume reduction of half a decibel.

All of these parameters have a minimum, maximum, and default value predefined in the DirectX headers. These follow the convention DSFX_I3DL2REVERB_(parameter)_MIN, DSFX_I3DL2REVERB_(parameter)_MAX, and DSFX_I3DL2REVERB_(parameter)_DEFAULT. For example, the three predefined values for lRoom are DSFX_I3DL2_REVERB_ROOM_MIN, DSFX_I3DL2_REVERB_ROOM_MAX, and DSFX_I3DL2_REVERB_ROOM_DEFAULT.

Preset Effects

It takes a bit of practice to know how to adjust all of these different variables to arrive at an environment that sounds good (that is, accurately conveys the sense of space you desire). To help out, there are several preset effects you can choose to use. These cover a wide range of environments, and free you from having to twiddle with the individual variables.

Using these presets is very easy; just call the SetPreset method of IDirectSoundFXI3DL2Reverb8, and pass it the GUID of the preset you want. The full list of GUIDs is included in the DirectX documentation for the SetPreset function. The actual values for each preset live in the dsound.h header.

Creating an Environment Editor

As you can see, the numbers aren't exactly intuitive. To properly create stunning audio environments using I3DL2 reverberation, you're going to need a program that will allow you to adjust these numbers and instantly hear the effects of those adjustments. In other words, you'll need an environment editor.

Thankfully, there aren't that many variables. A simple dialog with 12 trackbars (also called sliders) should be enough for a basic editor, so that's what you'll learn how to create in this section. Figure 15.3 shows a screenshot of this editor. As you can see, it's just a bunch of trackbars, along with load and save buttons, and a couple of buttons to browse for and play a WAV.

Click To expand
Figure 15.3: A screenshot of the Ch15p2_I3DL2EnvEdit or sample program.

Go ahead and spend some time playing with the Ch15p2_I3DL2EnvEditor sample program—the sections that follow will dissect this program for you. If you haven't had extensive experience writing for the Win32 GUI, this sample program may teach you a few tricks in that arena.

Initialization

Trackbars, in Win32 programming, are easy to work with as long as you remember to call the InitCommonControls Win32 API function. If you fail to call that, you won't be able to create any window that has a trackbar in it. Trackbars, unlike other controls (buttons, edit boxes, and so forth), weren't part of the original release of the Windows GUI. They were grafted on later, hence the need to call this function.

Dealing with Trackbars

I'm not going to cover the standard mechanism of creating a modeless dialog box and writing a message loop for that—there are several good introductory Windows Programming books that cover that material, and cover it well. But I would like to spend some time showing you how the GUI functionality works.

The trackbars, like every other control, operate using windows messages. The three most important messages you can send to a trackbar are detailed in Table 15.2.

Table 15.2: Common Trackbar Messages
Track Bar Message | Description
TBM_SETRANGE | Set the minimum and maximum value of the trackbar.
TBM_SETPOS | Set the current position of the trackbar.
TBM_GETPOS | Return the current position of the trackbar. When the user moves a trackbar, it notifies its parent window by sending a WM_HSCROLL message (for trackbars aligned horizontally) or a WM_VSCROLL message (for trackbars aligned vertically).

This looks pretty easy, right? All you need to do is send each trackbar a TBM_SETRANGE and TBM_SETPOS message to initialize the dialog box. Then, you watch out for WM_HSCROLL messages, and send TBM_GETPOS to get the trackbar's new position.

There's one small catch—trackbars only take integers. For most applications, this isn't a problem, but this sample program has several values that must be floats. For example, there's the reverb delay variable, flReverbDelay, which has a minimum value of 0.0 and maximum value of 0.1.

To accommodate this, you need to scale everything up by multiplying. For example, you might choose to multiply the flReverbDelay values by 10,000, giving you a nice integer range of zero to 1,000. Everything is then fine, as long as you remember to multiply the flReverbDelay on the way into the slider, and divide the slider's position by that value after you get it using TBM_GETPOS.

Tip

Trackbars can do much more than what I've described here. For example, you can also give them tick marks, and autobuddy them to another control! For more information, check out the MSDN documentation, at Platform SDK/User Interface Services/Shell and Common Controls/Trackbar Controls.

In the Ch15p2_I3DL2EnvEditor sample program, there are two overloaded functions called RefreshSlider. The function that takes integer values does nothing special; the floating point function takes a scaling factor.

Loading and Saving Environments

The easy way to load and save something that's in a structure is to simply save that structure's memory footprint directly to disk, as a binary file. And that's exactly what this simple editor does. Here are the functions that load and save from disk:

void OnLoad(HWND hDlg)
{
  char filename[MAX_PATH] = { 0 };

  OPENFILENAME ofn = { sizeof(OPENFILENAME), hDlg, NULL,
                       "Reverb Files\0*.reverb\0All Files\0*.*\0\0",
                       NULL,
                       0, 1, filename, MAX_PATH, NULL, 0, NULL,
                       "Open Reverb File",
                       OFN_FILEMUSTEXIST|OFN_HIDEREADONLY, 0, 0,
                       TEXT(".reverb"), 0, NULL, NULL };
 // Display the OpenFileName dialog. Then, try to load the
 // specified file
if(GetOpenFileName(&ofn))
{
  int handle = open(filename, O_RDONLY | O_BINARY);
  if (handle == -1) Throw("Can't open file!");
  read(handle, &g_ReverbValues, sizeof(DSFXI3DL2Reverb));
  close(handle);

  RefreshGUI(hDlg);
 }
}
void OnSave(HWND hDlg)
{
  char filename[MAX_PATH] = { 0 };

  OPENFILENAME ofn = { sizeof(OPENFILENAME), hDlg, NULL,
                       "Reverb Files\0*.reverb\0All Files\0*.*\0\0",
                       NULL,
                       0, 1, filename, MAX_PATH, NULL, 0, NULL,
                       "Save Reverb File",
                       OFN_HIDEREADONLY, 0, 0,
                       TEXT(".reverb"), 0, NULL, NULL };
 // Display the OpenFileName dialog. Then, try to load the
 // specified file
if(GetOpenFileName(&ofn))
{
  int handle = open(filename, O_RDWR | O_BINARY | O_CREAT | O_TRUNC);
  if (handle == -1) Throw("Can't save file!");
  write(handle, &g_ReverbValues, sizeof(DSFXI3DL2Reverb));
  close(handle);
 }
}

As you can see, the bulk of the code deals with the common file open and file save dialog boxes. To use these dialogs, you need to fill up an OPENFILENAME structure. You can look up the GetOpenFileName function in MSDN to figure out what each value in the OPENFILENAME structure means.

Once you get the structure filled up, displaying the dialog is as easy as calling GetOpenFileName. If the user hits OK, the function will return true, and the preceding code will go about the business of saving or loading the file.

Editor-Wrap Up

Of course, this is just for a simple editor. An editor for a professional game would be much more complex than this—it would take into account the player's position in the game world, and would allow audio designers to set up different sections of a level, each section with its own environmental audio parameters.

Just to get your mind warmed up, here are a few things you could do to enhance the program:

  • Make it save its variables in a more standard format, such as XML.

  • Add support for presets—I envision a drop-down menu that lets you choose a preset, tweak it, and save it out as a new environment.

  • Add features that allow the user to edit DirectSound 3D listener and source properties in addition to these environment variables. Quite frequently, an adjustment to one will necessitate an adjustment to another.

I3DL2 Wrap-Up

Keep in mind that there's more to I3DL2 than just reverberation. I3DL2 also specifies an interface for occlusion and obstruction, two principles that are equally important in 3D sound (see below for their definitions). Through DirectSound property extensions, you can access this occlusion and obstruction functionality and use it to make your game sound even more realistic.

If you'd like to learn more about I3DL2, check out the IA-SIG's 3D Audio Working Group, at http://www.iasig.org.


EAX

Now that you've gotten a chance to step into the ring with I3DL2, I'm going to round out this chapter by briefly mentioning a couple of other technologies that will prove to be useful if you want to be on the cutting edge of 3D audio. The first technology I want to talk about is called EAX.

EAX stands for Environmental Audio Extensions. It's a technology developed by Creative Labs, and has gone through two versions (at the time of this writing, EAX 2.0 is the most current). To understand what EAX does, you must first understand the most important aspects of shaping a sound based on its environment:

Reverberation. As you've now learned, by using some math to model the reflections of simulated sound waves off simulated walls, the computer can create very realistic echoes, which in turn can simulate a wide variety of environments.

Occlusion and Obstruction. Occlusion refers to something that physically separates the listener from a source (such as a wall). Obstruction, on the other hand, occurs when the listener and the source are in the same room, but there's an object (say, a crate) in between them.

Source Directivity. When you orient a sound in real life, not all frequencies respond to the orientation equally. The lower frequencies don't respond nearly as much as the high frequencies do. That's why, when setting up a home theater, it doesn't really matter where you put your subwoofer, but the position and orientation of your midand high-frequency speakers are very important. Advanced 3D sound APIs let you adjust this source directivity independently for high and low frequencies.

These three principles are the core concepts behind environmental modeling. EAX implements all three of these principles.

EAX is available for both DirectSound and OpenAL, and the integration with both is very straightforward. In DirectSound, you simply grab an interface to an EAX object corresponding to a particular sound source; in OpenAL, EAX is an extension, and is easily accessed through the OpenAL extensions mechanism.

You can learn more about EAX via Creative's developer Web site at http://developer.creative.com.


DOLBY DIGITAL ENCODING

If you're a professional developer creating a professional game, you owe it to yourself to give forms of Dolby Encoding a good look, especially Dolby Digital encoding. Essentially, this encoding allows you to encode 5.1 discrete channels of audio into a single bitstream. Just as you can encode two separate channels into a stereo WAV file, you can encode Left, Right, Center, Left Surround, and Right Surround channels into a Dolby bitstream. The bitstream also supports a LFE (Low-Frequency Effects) channel that contains frequencies less than 120Hz.

For more information on Dolby Digital Encoding, check out the Dolby Web site at http://www.dolby.com.


CONCLUSION

This chapter concludes your exploration of 3D sound. By now you should have enough foundation to implement 3D sound in your own games, and possibly to take that extra leap by integrating an advanced technology such as EAX.

The future of 3D audio gets bigger and brighter each day. Already, sound cards are beginning to appear on the market that mimic in hardware the reflections of individual sound waves. Moore's Law continues to bring professional-grade audio equipment into the hands of the consumer, and each generation of sound card brings with it more power and better functionality. If you've thought that better game audio technology just meant more channels and higher sample rates, I hope this chapter has changed your mind.


Chapter 16: DirectPlay Voice

Download CD Content

This chapter's sole purpose is to show you how to use the DirectPlay Voice API. This is a little-known component of DirectPlay that facilitates real-time chat between players in a multiplayer game. It's one of those rare technologies that's relatively easy to implement, yet adds a powerful new dimension to your game.

THE DIRECTPLAY VOICE API

It's possible that the DirectPlay Voice API owes its existence to one simple fact: multiplayer games are more fun if you can hear the other player scream. Few things are more gratifying than ambushing another player and listening to him swear as he realizes you now own him. With DirectPlay Voice, you can add this level of immersion in your game with only a few lines of code.

The Basic Concept

DirectPlay Voice works by sending voice data over a DirectPlay connection. It begins by recording the player's speech. The recording trigger can be manual (for example, the user has to press a key or a joystick button to start talking), or it can be automatic (the system starts recording when it detects a sharp change in volume on the mic). In its simplest usage, when one player starts talking, DirectPlay Voice samples his voice from the microphone (at a low sampling rate), then compresses it using a specially designed voice compression algorithm. DirectPlay Voice then sends this compressed data to the other computer via the DirectPlay networking protocol. The other computer decompresses the voice data, and plays it back out the sound card. Ideally this happens fast enough so that the chat appears instantaneous, but if there's network traffic, there might be a slight lag between when one player speaks and the other player hears that speech.

DirectPlay Voice Network Topologies

What I just described is an example of peer-to-peer DirectPlay Voice communication. In this topology, every computer is responsible for sending its player's voice data to every other computer. This means that if you're playing with five other people, when you talk, your computer must send your voice data to five different computers (see Figure 16.1).

Click To expand
Figure 16.1: In a peer-to-peer environment with six players, every time you talk, your voice must be sent to the five other computers.

As you can imagine, this can clog your upstream and downstream pipes. A typical compressed voice needs about 8 kilobytes per second of bandwidth (though you can use different compression codecs with bandwidth requirements ranging from 1.2kbps to 13kbps). This means you need 40kbps (8kbps × 5) of network bandwidth whenever you speak, and you also need to be able to download at least 40kbps (in case everybody else speaks at once). If you're playing on a LAN, no problem—LANs have a minimum bandwidth of 10Mbps. But, if you're playing through a modem, or even through a broadband connection with slow upload speeds, a simultaneous 40kbps upload/download speed may be unattainable.

For these situations, DirectPlay Voice allows you to choose one of two alternative topologies—the Forwarding Server Topology or the Mixing Server Topology.

In the Forwarding Server Topology, when you speak, your computer sends your voice stream to a central voice server. The server then sends your voice to all other computers in the game (see Figure 16.2).

Click To expand
Figure 16.2: In a forwarding server topology, you send your voice data to a central server; the server takes care of sending it to everyone else.

This frees your computer from having to send multiple copies of the data—the voice server does that instead. That way, only the voice server needs to have fast upload speeds. Note, however, that the clients still need to have fast download speeds, because if five people talk at once, one client is going to have to be able to download five streams simultaneously (see Figure 16.3).

Click To expand
Figure 16.3: If everybody talks at once, even with a voice server, you'll need 40 kpbs of download bandwidth to receive all 5 streams.

Think of the Forwarding Server Topology as the "one up, many down" topology—in the worstcase scenario, each client will only upload one voice stream; however, they may be downloading many voice streams simultaneously.

The third topology—the Mixing Server Topology—is the "one up, one down" topology. In this topology, the server receives data from all the clients, and decompresses the voice streams it receives. It then sends out a specially made stream to each client containing the mix of all the voices appropriate for that client (see Figure 16.4).

Click To expand
Figure 16.4: A mixing server sends each computer a mix of all the voices it needs.

For example, if player 1 and player 2 talk simultaneously, the server will send player 2's voice stream to player 1, player 1's voice stream to player 2, and a mix of both voice streams to all other players. That way, each client only needs to download one stream. However, the server has to work significantly harder—instead of just routing voice streams as it does in the forward server topology, it must now decompress, mix, and recompress the mixed streams for each client. The Mixing Server Topology trades processing power for smaller bandwidth utilization.

Of course, the best topology to use depends on where and how the game is being played. If it's being played on a LAN, usually you'll want to just use peer-to-peer—you've got plenty of bandwidth, and peer-to-peer ensures that each computer is doing the same amount of work. If the game is being played over the Internet, the choice hinges on whether there's a dedicated server present. The mixing server takes a large chunk of processing power—large enough that for most games, it's not feasible to also run the game itself because the frame rate would be too low. If the voice server is also a game player, the forwarding server topology is the best choice. However, if the voice server is just a dedicated game server, and doesn't have to worry about pumping triangles or anything else besides networking, it can easily handle the load of mixing the voices, and the best solution is to use a mixing server topology.

Tip

Instead of just presenting these topology options to your players, you can write code that can automatically decide which topology to use, based on the other network settings your players have specified. For example, if any client is connecting to the game through a modem, your code can automatically rule out the use of the peer-to-peer topology.

First, a Crash Course in DirectPlay

DirectPlay Voice runs atop DirectPlay. This means that if you've never written code that uses DirectPlay before, you need to understand DirectPlay's basic principles before you can get chatting through DirectPlay Voice.

This isn't a book on DirectPlay, so I'm going to concentrate on the basics, and move fairly fast through this part. However, it's imperative that you understand how the core of DirectPlay works in order to properly understand how DirectPlay Voice works. So, if you're having trouble understanding things from this brief description, or you want to learn about some things that I've specifically left out (for example, lobbies or player groups), you owe it to yourself to buttress your knowledge by looking at the DirectPlay sample programs and/or picking up some additional resources more focused on DirectPlay or networking.

The Flow of Events in a DirectPlay Session

To some people, DirectPlay can be a tough nut to crack. The trick is to break it down and look at the high-level steps you need to take in order to connect and play a multiplayer game.

There are three basic steps to any multiplayer game. First, you connect to the game. Then, as you're playing, your code sends and receives messages about the state of the game—for example, the position of the units in an RTS, or the fact that a player has just fired a weapon, and so on.

The bulk of your game code's time is going to be spent sending and receiving messages. You need to send and receive your own message types, and you need to process the DirectPlay system messages (player joined, player disconnected, and so on) as well.

Tip

Different games use different methods of conveying information using the smallest possible messages. Most games sync up the state of the game world at the beginning of the game, and then only send and receive messages containing specific changes to the world (for example, the server will send a message to a client saying, "tank number 4 has just moved to position (5,16)").

Finally, there's the obligatory "uninitialize and clean up" phase. This is always important, but it's especially important here, because if you forget to do it, DirectPlay may not notice that your game has stopped, and the other people playing won't receive a notification that you've bailed.

Tip

DirectPlay, like DirectMusic, makes heavy use of GUIDs (globally unique identifiers). DirectPlay uses them to identify games, service providers, and several other things during a multiplayer game. A GUID is unique in both time and space—when you create a GUID, you have a guarantee that it's not the same as any GUID ever created, or any GUID that ever will be created.

Now that you've got the high-level overview, let's zoom in a little closer. The following sections explain, in order, the detailed steps you should go through to use DirectPlay.

Creating the DirectPlay Interface

The first order of business is to create the interface to the DirectPlay COM object. The interface you use depends on the network topology you want for your game—peer-to-peer or client server. Table 16.1 summarizes the available interfaces.

Table 16.1: Available DirectPlay Interfaces
Interface | Description
IDirectPlay8Peer | Use this interface for a peer-to-peer topology.
IDirectPlay8Client | Use this interface when connecting as a client to a clientserver based topology.
IDirectPlay8Server | Use this interface to create a new server and a new clientserver session.

The sample programs in this book use a peer-to-peer topology, so they use an IDirectPlay8Peer interface. From this point forward, I'll describe things only in terms of that interface. However, aside from the connection process, most things translate fairly easily—all three interfaces have a nearly identical set of methods.

Selecting a Service Provider

Once you've created the appropriate DirectPlay interface for your network topology, you'll need to specify a service provider. DirectPlay can operate over different kinds of communication links—TCP/IP, as well as modems and serial connections. Each communication link has a different service provider. You'll need to enumerate the list of service providers from DirectPlay, and then present that list to your user, so he can select a service provider.

You can enumerate the available server providers by calling the EnumServiceProviders method of your chosen DirectPlay interface.

Starting a Game or Connecting to One

You've enumerated the available service providers, displayed them to your player, and received his selection. In a peer-to-peer world, the next step is to ask the player whether he would like to connect to an existing game, or start a new one.

You can determine what games are out there by calling the EnumHosts method of IDirectPlay8Peer. You'll get back a list of available hosts—computers running a session of your game.

You may be wondering how DirectPlay knows whether a computer is running your game. It knows thanks to the power of GUIDs. Each DirectPlay application has its own GUID, and to DirectPlay internally, this GUID is the name of the game. When you call EnumHosts, you can imagine DirectPlay broadcasting a question to all computers in your local realm: "Hey, if you're hosting a game with this GUID, please identify yourself." It then collects the responses and forwards them to your code. Think of each host responding with a GUID that identifies itself, as if saying, "Hello, this is host (GUID), and I'm hosting the game you're interested in."

You in turn present this list to your user, who chooses a host. You then pass the GUID of the host he selected back to DirectPlay via the Connect method of IDirectPlay8Peer, and wait for DirectPlay to return success or failure. If it returns success, you're connected, and you can start sending and receiving messages!

If your user chooses to host a new game instead of connecting to an existing one, the process is easier. All you do is call the Host method of IDirectPlay8Peer, giving it a DPN_APPLICATION_DESC structure containing information about your new game (for example, the name of this particular instance of the game, the password if it's private, the maximum number of players, and so on).

If the Host method returns success, you're ready to start sending and receiving messages.

Receiving Messages

Most of the data you'll send back and forth during a typical multiplayer game will be in your own format. However, there are canned messages that DirectPlay sends you and that you need to respond to. The full list of message types isn't listed here (it's in the DirectPlay docs), but Table 16.2 lists the basic set of messages you'll need to watch for.

Table 16.2: Common DirectPlay Messages
Message | Name Description
DPN_MSGID_CREATE_PLAYER | DirectPlay sends your code this message when a new player joins your game. When you receive this message, you should query for the player's name and other data by calling the GetPeerInfo method of the IDirectPlay8Peer interface, then use that information to initialize your game-specific player data.
DPN_MSGID_DESTROY_PLAYER | DirectPlay sends your code this message when a player disconnects from your game. Your job is to make sure that player is no longer referenced by your game code.
DPN_MSGID_RECEIVE | DirectPlay sends your code this message when it receives some game-specific data from another computer. Inside this message is a pointer to the raw bytes of the data received; your job is to parse those bytes into whatever format you've created, then process that data.
DPN_MSGID_SEND_COMPLETE | DirectPlay sends your code this message when the data that you've told it to send has actually gone out over the wire. In DirectPlay, data sends are usually asynchronous, which means once you kick them off, they happen in the background, while your game is doing other things.

These four messages represent the most common messages you're likely to see in a DirectPlay session.

When you initialize your IDirectPlay8Peer interface, one of the parameters you must provide is a pointer to a callback function that DirectPlay will call whenever a message comes in. In this respect, programming DirectPlay is a bit like programming Windows dialog boxes. Your callback function receives, as arguments, the ID of the message, along with a pointer to a message buffer.

Your job in this callback function is to look at the message and decide what to do. Usually, you accomplish this by using a switch statement, as shown in the following code:

HRESULT WINAPI OnDirectPlayMessage(
 PVOID pvUserContext, DWORD dwMessageId, PVOID pMsgBuffer )
{
 switch( dwMessageId ) {
  case DPN_MSGID_CREATE_PLAYER:
 {
  // a create player message has been received!
  HRESULT hr;
  PDPNMSG_CREATE_PLAYER pCreatePlayerMsg;
  pCreatePlayerMsg =
      reinterpret_cast<PDPNMSG_CREATE_PLAYER>(pMsgBuffer);
  DWORD dwSize = 0;
  DPN_PLAYER_INFO* pdpPlayerInfo = NULL;

  // do additional things here...
  }
  break;
  // other cases snipped
 }
return(S_OK);
}

If your callback function correctly interprets the message, you should return S_OK. You can give other return values in response to particular messages as detailed in the DirectX documentation.

Sending Messages

To send some data to the other computers in a multiplayer game, you use the SendTo method of IDirectPlay8Peer. It's very easy to figure out—it takes, as arguments, the ID of the player you want to send to (or the predefined DPNID_ALL_PLAYERS_GROUP for all players), a DPN_BUFFER_DESC structure containing a pointer to the data, and the number of bytes to send. It also takes some flags and other arguments you can use to fine-tune the behavior. For example, you can specify the DPNSEND_GUARANTEED flag to make DirectPlay guarantee that your data gets to the intended recipient.

Threads and Synchronization

One of the most difficult aspects of programming using DirectPlay is that DirectPlay is a multithreaded component. One of the most critical to understand caveats of this is that DirectPlay can call your receive callback function simultaneously from several different threads.

This means that you need to make sure your callback function is thread safe. Make sure that if you're changing global data (for example, anything that isn't declared on the stack, or is declared globally), those changes are wrapped in a critical section or some other thread synchronization primitive.

Disconnecting

When your player decides to exit your game, you need to make sure you call the Close method of IDirectPlay8Peer, so that DirectPlay has a chance to let everybody else know that this player is leaving. You also need to call Close for the lobby interface, IDirectPlay8LobbiedApplication. Once you've closed both interfaces, you can release them as you would any other COM interface, namely, by calling their Release methods or by using the SAFE_RELEASE macro.

Using CNetConnectWizard

The previous three steps can take a fair amount of code. Microsoft realized this, and included as part of the DirectX SDK, a couple of C++ classes designed to make it very easy to get through the banalities of setting up DirectPlay and connecting to a game.

This class is called CNetConnectWizard, and it's available inside the Samples directory of your DirectX installation. It does an excellent job of walking the user through selecting a service provider and joining an existing game, or starting a new game. The only drawback to it is that it does this through a series of regular dialog boxes. Your game may want to present options to your players using a custom GUI, with cool-looking buttons and scrollbars. There's no easy way to do this with the wizard, so in those situations you'll have to write your own code. But if you don't mind seeing regular dialog boxes during your game, you can use CNetConnectWizard to save you some time.

Creating and Initializing

CNetConnectWizard is usually created dynamically, like in the following code, taken from Ch16p1_VoiceChat:

CNetConnectWizard* g_pNetConnectWizard = NULL;  // Connection wizard
g_pNetConnectWizard = new CNetConnectWizard( hInstance, NULL,
 "Ch16p1_VoiceChat", &g_GameGUID );]
g_pNetConnectWizard->Init( g_pDPPeer, g_pDPLobbiedApp );

Here you can see the code creating a new CNetConnectWizard and calling its Init method. The Init method takes two interfaces—one IDirectPlay8Peer and one IDirectPlay8LobbiedApplication.

I've specifically avoided talking about lobbies in general and the IDirectPlay8LobbiedApplication in particular; lobbies are an advanced feature of DirectPlay you need only concern yourself with if you want your game to be launchable from a game-browsing program. The lobby is a mechanism to pass connection and game settings from one program to another. You can read more about it in the DirectX documentation if you're interested, but all you need to know for this chapter is that the CNetConnectWizard needs a lobbied application interface.

Hooking Your Receive Callback Function

The CNetConnectWizard can only work its magic if it can process DirectPlay messages. This means that your DirectPlay receive callback must call the MessageHandler method of CNetConnectWizard, passing it the same arguments DirectPlay supplied to your callback.

The IDirectPlay8LobbiedApplication interface requires a callback function, just like the IDirectPlay8Peer interface, and the CNetConnectWizard must hook this callback in the same way. Both hooks are illustrated in the following code, taken from Ch16p1_VoiceChat:

 HRESULT WINAPI OnDirectPlayMessage(
  PVOID pvUserContext, DWORD dwMessageId, PVOID pMsgBuffer )
 {
 switch( dwMessageId ) {
   case DPN_MSGID_CREATE_PLAYER:
     // stripped out
     break;
   case DPN_MSGID_DESTROY_PLAYER:
    // stripped out
    break;
    // other messages
 }

   // give the wizard a chance to react to the message
   return g_pNetConnectWizard->MessageHandler(
     pvUserContext, dwMessageId, pMsgBuffer );
 }
 HRESULT WINAPI OnDirectPlayLobbyMessage(
   PVOID pvUserContext, DWORD dwMessageId, PVOID pMsgBuffer )
 {
    return g_pNetConnectWizard->LobbyMessageHandler(
      pvUserContext, dwMessageId, pMsgBuffer );
 }

The IDirectPlay8LobbiedApplication callback function, OnDirectPlayLobbyMessage, doesn't do any processing at all—it just hands what it got directly to the wizard. The main IDirectPlay8Peer callback, however, watches for specific types of messages, processes them, and then gives the wizard a chance to do the same and return a value.

Setting Default Values and Displaying the Wizard

After you've created the CNetConnectWizard and have hooked it up to your IDirectPlay8Peer and IDirectPlay8LobbiedApplication receive callback functions, you're ready to set its default values, and tell it to display its dialogs and query the user for his choices.

The following code shows how this is done:

// set defaults
g_pNetConnectWizard->SetPlayerName( "Anonymous" );
g_pNetConnectWizard->SetSessionName( "New Game" );
g_pNetConnectWizard->SetPreferredProvider( "DirectPlay8 TCP/IP Service Provider" );
// display the connection wizard
hr = g_pNetConnectWizard->DoConnectWizard( FALSE );
if (FAILED(hr)) throw(hr, "Connection wizard failed.");
if( hr == NCW_S_QUIT ) { return(0); }

As you can see, the code puts default values into the wizard by calling the SetPlayerName, SetSessionName, and SetPreferredProvider methods. These default values will be displayed in the dialog boxes presented to the user (see Figures 16.5 and Figures 16.6).

Click To expand
Figure 16.5: CNetConnectWizard's connect dialog.

Figure 16.6: CNetConnectWizard's new game dialog.

The DoConnectWizard is where all the fun occurs. When you call this method, the wizard dialogs are displayed, and the user inputs his choices. The DoConnectWizard method returns an HRESULT; if this failed, something unexpected has gone wrong. If the HRESULT is NCW_S_QUIT, this means the user chose to quit or close the wizard before it was finished. Usually, this means that the user wants to quit the multiplayer game altogether, so you need to back out and return him to your game's title screen, or exit, in the case of the Ch16p1_VoiceChat sample program.

If the user does get all the way through the wizard, the CNetConnectWizard class will automatically call either the Host or Connect methods of IDirectPlay8Peer. In other words, it takes care of everything for you!

If you want to save the values the user selected so that they're the defaults the next time the wizard is launched, you can get them by calling the GetPlayerName, GetSessionName, and GetPreferredProvider methods. Then, just store them in the registry or in a preferences file somewhere, and read them back in before you display the wizard the next time around. Also, you can call the IsHostPlayer method to determine whether the user ended up hosting a new game.

Shutting Down the Wizard

When you're all done with the wizard, cleaning it up is as easy as calling its Shutdown method, and then deleting the instance of the object itself (via a call to delete or by using the SAFE_DELETE macro).

Programming DirectPlay Voice

Whew! Now that you've had the whirlwind tour of DirectPlay, you can start on the focus of the chapter—using DirectPlay Voice to implement real-time voice chat in your games.

DirectPlay Voice, in many ways, is a mirror image of DirectPlay. There is a message callback function for DirectPlay Voice messages, as well as separate interfaces for voice clients (IDirectPlayVoiceClient) and servers (IDirectPlayVoiceServer). There's also a tiny interface called IDirectPlayVoiceTest, with just one method: CheckAudioSetup. This method runs DirectPlay Voice's audio setup wizard so that users can adjust their playback and recording volumes, and specify which hardware (mic input) to record (see Figure 16.7).

Click To expand
Figure 16.7: The sound hardware test wizard.

The following sections detail how to program DirectPlay Voice.

Initializing DirectPlay Voice

DirectPlay Voice has no peer-to-peer interface; there are only IDirectPlayVoiceServer and IDirectPlayVoiceClient. This means that before you can initialize a DirectPlay Voice interface, you have to figure out whether you're a voice server or a client. There's no requirement that the DirectPlay server must be a DirectPlay Voice server, and there's nothing stopping a DirectPlay client from being a DirectPlay Voice server. However, the easiest way to decide is to simply mirror your DirectPlay role—if you're a host in a peer-to-peer session, or you're a DirectPlay server, you're a voice server; otherwise, you're a voice client.

Voice Messages

When you create your DirectPlay Voice interface, you specify a callback function to receive DirectPlay Voice messages. Table 16.3 shows the most common voice messages you'll receive.

Table 16.3: Common DirectPlay Voice Messages
DirectPlay Voice Message ID | Description
DVMSGID_CREATEVOICEPLAYER | An existing DirectPlay player has just activated DirectPlay Voice and has become a DirectPlay Voice player. Inside the message is the ID of the player who upgraded to voice. You need to use this ID to set up any additional player parameters in your game to use DirectPlay Voice.
DVMSGID_DELETEVOICEPLAYER | An existing DirectPlay player has stopped using DirectPlay Voice. Note that this doesn't mean he is no longer playing; he may still be a normal DirectPlay player.
DVMSGID_PLAYERVOICESTART | DirectPlay Voice sends you this message when it starts playing an incoming voice stream.
DVMSGID_PLAYERVOICESTOP | DirectPlay Voice sends you this message when it stops playing an incoming voice stream.
DVMSGID_RECORDSTART | DirectPlay Voice sends you this message when it starts recording a voice stream from the local player.
DVMSGID_RECORDSTOP | DirectPlay Voice sends you this message when it stops recording a voice stream.
DVMSGID_LOSTFOCUS | DirectPlay Voice sends this message when the local player has lost the ability to record audio. Usually it means the player has switched to another application and your application no longer has free reign over the audio capture hardware.
DVMSGID_GAINFOCUS | DirectPlay Voice lets you know when the player can once again send audio by giving you this message. Usually it means that the player has task-switched back into your game and you once again have control over the audio capture hardware.

Testing Your Audio Setup

To get the most out of real-time chat, it's very important that your player properly calibrate his mic and speakers. You can launch the setup dialog at any time by creating a IDirectPlayVoiceTest interface and calling its sole CheckAudioSetup method.

By passing arguments to the CheckAudioSetup method, you can specify the device for playback as well as for capture.

Half versus Full Duplex Sound Cards

Voice capture hardware falls into two main categories—those cards that support full-duplex audio and those that do not. Full duplex simply means that you can record and playback at the same time. Half duplex means that you can't record when you're playing any sound, and vice-versa.

Unfortunately, DirectPlay Voice doesn't allow voice recording from half-duplex hardware. Any users with half-duplex sound cards will be able to hear other player's voices, but won't be able to talk themselves.

The vast majority of the sound cards on the market today are full duplex, so it's usually not an issue, but you can determine whether the user is running on a fullor half-duplex sound card by using the GetSoundDeviceConfig method of either the client or server DirectPlay Voice interfaces. This method returns a structure full of all sorts of information about the hardware, including whether it's full or half duplex.

Using CNetVoice

CNetVoice, similar to CnetConnectionWizard, is a helper class written by Microsoft and distributed as part of the DirectX SDK samples. It's designed to hide all the intricacies of dealing with DirectPlay Voice, and it does a very good job of that.

To use CNetVoice, all you need to do is instantiate it, then supply some default options and call its Init method, as shown by the following code:

g_pNetVoice = new CNetVoice(
  OnVoiceClientMessage, OnVoiceServerMessage );
// Set default DirectPlayVoice setup options
DVCLIENTCONFIG config;
ZeroMemory( &config, sizeof(config) );
config.dwSize = sizeof(config);
config.dwFlags = DVCLIENTCONFIG_AUTOVOICEACTIVATED |
   DVCLIENTCONFIG_AUTORECORDVOLUME;
config.lPlaybackVolume = DVPLAYBACKVOLUME_DEFAULT;
config.dwBufferQuality = DVBUFFERQUALITY_DEFAULT;
config.dwBufferAggressiveness = DVBUFFERAGGRESSIVENESS_DEFAULT;
config.dwThreshold = DVTHRESHOLD_UNUSED;
config.lRecordVolume = DVRECORDVOLUME_LAST;
config.dwNotifyPeriod = 0;
GUID codecguid = DPVCTGUID_DEFAULT;
// initialize voice class
g_pNetVoice->Init( hDlg, g_pNetConnectWizard->IsHostPlayer(),
 TRUE, g_pDPPeer, DVSESSIONTYPE_PEER, &codecguid, &config );

As you can see, you supply your two callback function names to the CNetVoice constructor. Also, note that most of the members of the DVCLIENTCONFIG structure can simply be set to their default values.

Once this has been done, all that remains is to flesh out the client and server message handlers. After that, just remember to delete your CNetVoice object, and you're all set!

Reference Counted Player Objects

In the Ch16p1_VoiceChat sample program, you'll notice that a player's voice settings and regular DirectPlay settings are integrated into a single object: CPlayer. This is a typical thing to do—most games prefer having one class or structure that represents a player, including core DirectPlay properties, like his name, as well as DirectPlay voice properties (like full duplex/half duplex).

The only thing you need to watch out for with this is that you don't accidentally delete a player structure when, in fact, that player is still playing the game, but they've just stopped using DirectPlay Voice.

The Ch16p1_VoiceChat sample program guards against this by using reference counts. When it receives a new player message, the sample program creates a new player object and sets its reference count to one. When it receives a DirectPlay Voice "create voice player" message, it looks up the correct player structure (stored in an STL map of DPNIDs to CPlayer objects), and increments its reference count. So, by the time everything is created, all of the player objects have a reference count of two—one reference for core DirectPlay and one for DirectPlay Voice.

When it comes time to delete the object, the code starts by decrementing the object's reference count. Only if the reference count falls to zero does the object actually get deleted and removed from the map.

If you do things like this, it doesn't matter which "delete player" message you get first. The object only gets deleted when both DirectPlay and DirectPlay Voice have received the delete message.

Voices as 3D Sound Buffers

One of the cool features of DirectPlay Voice is that it lets you tie the incoming voice streams to 3D sound sources. That way, if another player is standing to the left of you when he speaks, his voice will appear to come from your left.

Tip

Note that integration of DirectPlay Voice with 3D sound sources can only occur if you're not using a mixing server topology. In a mixing server topology, all of the streams are mixed together before they reach your computer, and the server that did the mixing has no idea where the players are located.

You can set this up by calling the Create3DsoundBuffer method of the client DirectPlay Voice interface (IDirectPlayVoiceClient). This method lets you tie a 3D sound source to a particular player ID. From there, you just need to position this buffer where the player is actually standing, and DirectX Audio will do the rest!


CONCLUSION

In this chapter, I presented an introduction to DirectPlay and DirectPlay Voice. You should realize that there are many other useful features of DirectPlay that can make your game even better. I encourage you to explore these features on your own, and to try them out by writing your own DirectPlay-enabled applications.


Chapter 17: Audio Visualization

Download CD Content

OVERVIEW

Yes, I know what you're thinking—what is a chapter on visualization doing in a book about game audio programming? After all, audio visualization—whether it be through a simple wave, spectrum analysis, or a decorative, music-driven effect popular in today's MP3 players—is not something one normally associates with games, or game programming.

However, audio visualization ties together myriad fundamental concepts of game audio. The same techniques at the core of audio visualization are also at the core of several advanced audio techniques, including audio compression, audio filtering, and so on. A proper understanding of audio visualization, and specifically the Discrete Fourier Transform, is essential to becoming a game audio guru.

Plus, it's a fascinating trip through biology, mathematics, breaking dishware, and the search for extra-terrestrial intelligence. When you consider how few subjects you can say that about, you'll understand why I just couldn't finish this book without spending some time on it.

In this chapter, you're going to learn how to write a spectrum analyzer. Essentially, a spectrum analyzer is a bar graph that tells you which frequencies are in a given sound, and how loud each of those frequencies is. Figure 17-1 is a screenshot of this chapter's sample program, a spectrum analyzer—you've probably seen a similar "frequency bar graph" in your favorite MP3 player.

Click To expand
Figure 17.1: A screenshot of the ch17p1_visuals sample programe.


THE DISCRETE FOURIER TRANSFORM

The core of spectrum analysis is simple. You have a sequence of numbers representing your sampled sound (or, more precisely, representing the amplitude of your sound at several different points in time). You want to generate another set of numbers, which represent the amplitudes of the various frequencies contained in your original sample. Once you have these amplitudes, you can choose to display them any way you like. You might draw a bar graph, with each bar representinga different frequency, and the height of the bar representing its amplitude (like most popular MP3 players do). Or, you might plot a 2D graph, with frequency on the vertical axis and time on the horizontal axis, using color to represent amplitude. To reiterate: once you get the array of frequency amplitudes, you're home free—from there it's just a matter of GUI programming.

Tip

In mathematical terms, when you're writing a spectrum analyzer, what you're really doing is converting your sample from the time domain to the frequency domain. In other words, you're changing the x-axis of your graph. Instead of the x-axis being time, as it usually is for sampled sounds, you're changing it to be frequency.

One way to get the frequency amplitude array is to use a Discrete Fourier Transform, or DFT. The bulk of this chapter will focus on the theory behind the DFT.

Resonance

I'm sure that you're familiar with the concept of resonance: the opera singer can shatter a wine glass by singing at its resonant frequency. Essentially, the vibrations caused by the opera signer's voice cause the wine glass to vibrate stronger and stronger, until eventually the vibration forces are too much for the glass to bear, and it shatters.

What you may not understand is exactly why the forces build up to shatter the glass. To understand that, imagine that you come across a large, heavy silver orb suspended from the ceiling by a sturdy rope. Intuitively, you know that tiny applications of force, when applied at precisely the right time, can make the orb swing higher and higher. It doesn't matter how much the orb weighs, all you have to do is give it a tiny push right when it needs it, and you'll eventually make it swing.

Now, imagine that instead of a silver orb, the thing being pushed at exactly the right time is a molecule inside the wine glass, and the thing doing the pushing is not a pair of human hands, but rather, waves of compressed air, also known as sound waves, traveling through the air. If each sound wave arrived at just the right time, they could "swing" the molecule further and further away from its resting place. Further, these sound waves would all be hitting multiple molecules in the glass, making all the molecules swing in different directions, until eventually the glass just shatters.

How You Hear

It's this property of resonance that allows you to hear. Your ear is like a biological spectrum analyzer. Inside your ear is an apparatus called the cochlea, shaped like a shell. Inside the cochlea is a membrane called the basilar membrane. The basilar membrane's base is narrow and stiff, which makes it resonate with high frequency sounds. The other end of the membrane, the apex, is wide and flexible, making its resonant frequency low. In between, the different parts of the basilar membrane have different resonant frequencies. The membrane is covered with nerve endings, and when a particular frequency causes a particular portion of the membrane to resonate, that resonation causes a nerve to send a message to your brain, and you "hear" that frequency.

Resonance Expressed Mathematically

Sinusoid waves are interesting because the sum of their parts is always zero—in other words, the waves spend the exact same amount of time below zero as they do above zero. This means that in general, if you sample a wave's amplitude at regular intervals, and then add those samples up, you're going to arrive at an answer that's close to zero. The positive and negative samples will, in general, cancel each other out.

However, something interesting happens if you sample the wave at its frequency. No matter when you start sampling, you will eventually end up with a number that diverges from zero. If you happen to start sampling when the wave's above zero, the wave will be in that exact same spot the next time you sample it, causing your sum to grow bigger and bigger over time. Similarly, if you happen to start sampling when the wave's below zero, your sum will move further and further negative with every sample. This is resonance, expressed mathematically.

This means that if you want to figure out whether a given wave has, contained within it, a wave of a particular frequency, all you need to do is multiply points of one wave with points of the other, and add up the results of all those multiplications. This could be expressed in code like this:

float sinAmp = 0.0f; // this will contain the final amplitude value
for (int k = 0; k < numsamples; k++) {
  float x = 2.0f * M_PI * (float)freq * (float)k /(float)numsamples;
  sinAmp += sample[k] * sin(x);
}

This is fictional code—it doesn't appear in the Ch17p1_Visuals sample program. Assume that numsamples contains the total number of samples in our input wave, and that sample is an array of floating point numbers, between –1 and 1. The code loops through the entire array of samples, multiplying each sample by sin(x), then adding that result onto sinAmp. When the loop finishes, sinAmp contains the amplitude of the wave (of frequency freq) present in the source.

To calculate x, the input to the sin function, the code multiplies the frequency in question (freq) by 2[.pi] (2 times the value of pi, or approximately 6.28). The sin function expects radian input, which is a special form of angular measurement that ranges from 0 to 2 pi radians, instead of from 0-360 degrees. Finally, the code multiplies that by k (our loop counter) over the total number of samples (numsamples). This is what "iterates over" the sine wave the code is generating—when k is zero, x is zero. When k is numsamples, k divided by numsamples is 1, and x becomes 2 pi times freq. It's this mechanism that gives you the point on the imaginary wave of frequency freq that you then multiply against the corresponding point on the input wave.

Once you've multiplied each point on the source wave by the corresponding point on the imaginary freq wave, and added all those together, you (somewhat magically) arrive at the amplitude of the freq wave contained within your source wave. In essence, you've simulated mathematically the process of resonation!

When Sine Alone Won't Cut It

It's important to realize that when "resonating" your input wave against a sine wave of a given frequency, the final amplitude you get out will vary depending on the phase of the waves. The phase is a measurement of how much the wave is shifted left or right on the graph.

Imagine that you have two source waves, both of which are a sine wave at 440Hz and amplitude 5. Wave 1 has a phase of zero, and wave 2 has a phase of pi/2 radians. If you "resonate" both of these waves, the two numbers you get for your final amplitudes will be different. Wave 1's amplitude will be higher than wave 2's. When wave 1 is at its peak of 5 or –5, the resonating wave will be at its peak of 1 or –1, whereas with wave 2, when it's at its peak, the resonating wave will be zero. In other words, the resonating wave "misses out" on the peaks of wave 2, simply because it's shifted (or, in more technical terms, has a different phase). When the input wave is at its maximum, the resonating wave is at zero, and when the resonating wave is at its maximum, the input wave is at zero. This leads to a final amplitude value that is lower than it should be.

To solve this problem, you use a cosine wave. A cosine wave is exactly like the sine wave, only it starts at one, whereas a sine wave starts at zero (see Figure 7.2). In other words, a cosine wave is a sine wave with a different phase.

Click To expand
Figure 17.2: A cosine wave is a sine wave with a different phase.
Tip

In fact, it can be proven mathematically that any wave containing a single frequency at a single amplitude, regardless of phase, can be represented by the sum of a sine wave and a cosine wave. There's not enough room to go into the mathematical proof, but I encourage you to explore it on your own using some of the links I've provided on your CD.

If you resonate an input wave against both sine and cosine waves, you're guaranteed to get the same final amplitude number, regardless of the phase of ("amount of shift in") your input wave. This is because the sine and cosine waves cover for each other— when one is at zero, the other is at its maximum, and so you're guaranteed to never "miss" the peaks of your input wave.

Using both sine waves and cosine waves gives us code that looks like the following:

float sinamp = 0.0f;
float cosamp = 0.0f;
for (int k = 0; k < numsamples; k++) {
  float x = 2.0f * M_PI * (float)freq * (float)k /(float)numsamples;
  sinamp -= sample[k] * sin(x);
  cosamp += sample[k] * cos(x);
}
float amp = sqrt(sinamp*sinamp + cosamp*cosamp);

The preceding code is similar to what you saw a few sections ago, only instead of using just a sine wave, it uses both a sine and cosine wave (I subtract from the sinamp variable to make the sine wave start below the zero line, as the cosine wave does). Also, there's a new line that calculates amp as the square root of sinamp squared plus cosamp squared.

This is really just a distance calculation in disguise. Suppose you want to find the distance from the origin of a particular point, say, (3,4). To find the distance from the origin to this point, you square both components, then add the squares together and take the square root. 3 squared is 9, 4 squared is 16, 9 plus 16 is 25, so the final distance is the square root of 25, or simply 5. If you plot a 2D graph with sinamp on one axis and cosamp on the other, you can calculate the final amplification as the distance to the point formed by (cosamp, sinamp). If this isn't immediately clear to you, there is a ton of backstage mathematical ponderings as to why this is true—I encourage you to explore the links I've included on your CD, as the road to understanding this is truly a fascinating one, filled with imaginary numbers and several other forms of weirdness.

And now, at long last, you have the single most important piece of knowledge needed to implement a spectrum analyzer: you know how to figure out "how much" of a certain frequency is contained in a wave file. Now that you know this, all you need to do is perform the same operation using several different freq values, and you'll arrive at an output array suitable for graphical display.

The Nyquist Number

The only question left is, "which frequencies should I use?" To answer that, you must first come to grips with the Nyquist number. The Nyquist number, in this context, represents the highest frequency that can be properly captured using a certain sampling rate.

For example, consider a source wave with a frequency of 10Hz, sampled 20Hz. I've kept this number small to illustrate this concept—as you know, waves are usually captured at sample rates of 22,050Hz or 44,100Hz, but for this example, let's throw quality out the window and sample at 20Hz.

Sampling a 10Hz wave using a sample rate of 20Hz doesn't yield a great sample, but it does properly capture the original frequency of 10Hz. Now, imagine that you try to sample a wave of 30Hz at a sampling rate of 20Hz. The set of samples you get are completely bizarre, because the wave is undulating too fast for you to see it. This is called aliasing, and it's the audio equivalent to why fast-moving helicopter blades and tires appear to occasionally be moving backwards. Of course, they aren't really moving backwards—your brain's visual sampling rate is simply too slow to capture the true motion, so it appears as if they're moving backwards.

The Nyquist number is the highest frequency you can capture with a given sampling rate and not get any bad aliasing effects. It is always exactly half the sampling rate. Therefore, at 44,100Hz, the highest frequency you can capture is 22,050Hz.

Discrete Fourier Transforms

Now, at last, you have all the pieces to the puzzle. You know how to figure out if a wave contains a given frequency, and now, thanks to the Nyquist number, you know the highest frequency you need to look for. All that remains is to wrap the code you're already familiar with inside another for loop:

for (int bin = 0; bin <= numsamples/2; bin++) {
  float cosAmp = 0.0f;
  float sinAmp = 0.0f;
  for (int k = 0; k < numsamples; k++) {
    float x = 2.0f * M_PI * (float)bin * (float)k / (float)numsamples;
    sinAmp += m_Input[k] * sin(x);
    cosAmp += m_Input[k] * cos(x);
  }
  m_AmpOutput[bin] = sqrt(sinAmp*sinAmp + cosAmp*cosAmp);
}

The freq variable is now bin, a for loop running from zero to the Nyquist frequency, and the final amplitude is now an array of amplitudes for each frequency.

Animating Spectrum Analyzers

The preceding section of code will give you the amplitudes of various frequencies contained in the entire input wave. To implement an animating spectrum analyzer, all you need to do is calculate the frequencies for a small portion of the input wave, starting at the sample that's currently coming out the speakers, and ending some number of bytes from that point (512 is a good number). You'll learn about this in more detail in the next sections.


THE SAMPLE PROGRAM

Congratulations, you've made it through all of the theory surrounding the spectrum analyzer. Now it's time to roll up your sleeves and actually implement one! The following sections will walk you through the code contained in the Ch17p1_Visuals sample program.

Figure 17-1 shows a screenshot of the Ch17p1_Visuals sample program. As you can see, it has a very minimal GUI, consisting of two buttons, browse and play, above a large rectangle where the spectrum analyzer is displayed.

Loading the Wave

This sample program, like the one from Chapter 2, uses DirectSound, the low-level DirectX Audio interface. I chose to use DirectSound mainly because in DirectSound, it's significantly easier to figure out what byte of a sample is coming through the speakers at any given instant than in DirectMusic. It's also a symmetric end to the book—so as we started with the low-level API, we return in the end to the low-level API.

Unfortunately, using DirectSound means you must load the wave file yourself—you can't just load it into a DirectMusic segment. To help out a little, you can use the code contained in CWAVFile to actually read the wave, like so:

CWAVFile wavfile;
wavfile.Load(filename);
CreateSecondaryBuffer(&g_pWAVBuffer, wavfile);
unsigned char *ptr1 = NULL;
unsigned long len1 = 0;
g_pWAVBuffer-&-Lock(0, 0, (void **)&ptr1, &len1, NULL, NULL,
DSBLOCK_ENTIREBUFFER);
memcpy(ptr1, wavfile.GetData(), wavfile.m_DataLen);
g_pWAVBuffer->Unlock(ptr1, len1, NULL, NULL);

This chunk of code was taken from Ch17p1_Visual's LoadSound function. Here you can see the code loading the WAV into wavfile, then locking a secondary DirectSound buffer, filling in its contents, and finally unlocking it (consult chapter 2 if you need a refresher on how to lock and unlock a DirectSound buffer).

CDiscreteFourierTransform

Ch17p1_Visuals relies on a new class: CDiscreteFourierTransform. This new class is designed to contain all of the data and methods needed for performing Discrete Fourier Transforms. Here's the declaration:

class CDiscreteFourierTransform
{
public:
  CDiscreteFourierTransform(int bufsize, int samplerate, int
 bitspersample, int channels);
  virtual ~CDiscreteFourierTransform();
  void LoadInputData(unsigned char *data);
  void PerformSlowFourierTransform();
  float GetAmp(int ndx) { return(ndx > m_BufSize ? 0.0 :
 m_AmpOutput[ndx]); }
  float GetFreq(int ndx) { return(ndx > m_BufSize ? 0.0 :
 m_FreqOutput[ndx]); }
  int m_SampleRate;
  int m_NumChannels;
  int m_BitsPerSample;

protected:
  int m_BufSize;
  float *m_Input;
  float *m_FreqOutput;
  float *m_AmpOutput;
};

Nothing terribly complex here—the main members are the three float pointers m_Input, m_FreqOutput, and m_AmpOutput. These are actually dynamically allocated arrays of floating point numbers—their size depends on the buffer size (m_BufSize). The m_Input array is used to store the array of input samples. The m_AmpOutput array contains the amplitudes of each frequency—the heart of the spectrum analyzer. The m_FreqOutput tells the client code the frequency of a given index. It's useful if you want to put labels on your spectrum analyzer.

Constructing the Arrays

The floating point arrays are constructed when the object is, as shown in the following code:

CDiscreteFourierTransform::CDiscreteFourierTransform(
  int bufsize, int samplerate, int bitspersample, int channels) :
  m_BufSize(bufsize), m_SampleRate(samplerate),
  m_BitsPerSample(bitspersample), m_NumChannels(channels)
{
  // make space for buffers
  m_Input = new float[bufsize];
  m_FreqOutput = new float[(bufsize/2)+1];
  m_AmpOutput = new float[(bufsize/2)+1];
  // we know already what the frequencies of each bin will be
  for (int bin=0; bin < m_BufSize/2; bin++) {
    m_FreqOutput[bin] = (float)bin * m_SampleRate / m_BufSize;
  }
}

Also notice that the code takes this opportunity to create the frequency array, m_FreqOutput. Since this array remains constant, this is the best time to calculate its values.

Loading the Values

When the client code wants to update the spectrum analyzer, it begins by loading a new set of values into CDiscreteFourierTransform. These new values are the snippet of the wave, starting at the sample that's currently playing and extending for BUFFERSIZE samples (a constant integer set to 512).

Before the DFT can be calculated, the CDiscreteFourierTransform class must convert the array of samples from 0-255 (or 0-65535 for 16 bit samples) into floating point values from –1.0 to 1.0. This is accomplished as follows:

void CDiscreteFourierTransform::LoadInputData(unsigned char *data)
{
  // convert each data point from 0-255 to floating point -1..1
  if (m_BitsPerSample == 8) {
    for (int q=0; q < m_BufSize; q++) {
      float d = data[q*m_NumChannels];
      m_Input[q] = (d-127.0f)/127.0f;
      if (m_Input[q] < -1.0f) m_Input[q] = -1.0f;
      if (m_Input[q] > 1.0f) m_Input[q] = 1.0f;>
    }
 }
 else if (m_BitsPerSample == 16) {
   short int *sidata = reinterpret_cast_short int *>(data);
   for (int q=0; q < m_BufSize; q++) {
     float d = sidata[q*m_NumChannels];
     m_Input[q] = (d-32767.0f)/32767.0f;
     if (m_Input[q] < -1.0f) m_Input[q] = -1.0f;
     if (m_Input[q] > 1.0f) m_Input[q] = 1.0f;
     }
 }
 else {
   // not supported
   for (int q=0; q < m_BufSize; q++) { m_Input[q] = 0.0f; }
 }
}

As you can see, there are some pointer tricks at play here. The code treats the incoming data variable as either an array of 8-bit values, or an array of short ints (16-bit values), depending on the m_BitsPerSample value. Also note that it uses m_NumChannels, so that in stereo samples, it uses only the first channel.

Calculating the DFT

Finally, the moment you've been waiting for. Once the arrays are set up and the m_Input array is loaded with floating point values, the client code calls the PerformSlowFourierTransform method to calculate the DFT and populate the m_AmpOutput array:

void CDiscreteFourierTransform::PerformSlowFourierTransform()
{
  for (int bin = 0; bin <= m_BufSize/2; bin++) {
    float cosAmp = 0.0f;
    float sinAmp = 0.0f;
    for (int k = 0; k < m_BufSize; k++) {
      float x = 2.0f * M_PI * (float)bin * (float)k / (float)m_BufSize;
      sinAmp += m_Input[k] * sin(x);
      cosAmp += m_Input[k] * cos(x);
    }
    m_AmpOutput[bin] = sqrt(sinAmp*sinAmp + cosAmp*cosAmp);
  }
}

At this point, there should be no surprises in that code. It's based on the theory you learned at the beginning of the chapter.

Rendering the Graph

Finally, here's the code that uses CDiscreteFourierTransform's m_AmpOutput array to actually render the spectrum analyzer:

void RenderSpectrum(HDC hdc, HWND hDlg)
{
  HWND framewin = GetDlgItem(hDlg, IDC_FRAME);

  RECT framerect;
  int framewidth,frameheight;
  GetWindowRect(framewin, &framerect);
  ScreenToClient(hDlg, (POINT *)&framerect.left);
  ScreenToClient(hDlg, (POINT *)&framerect.right);
  framewidth = framerect.right - framerect.left;
  frameheight = framerect.bottom - framerect.top;
  // draw each bar
  int n=0;
  for (float x=framerect.left; x < framerect.right;
    x += framewidth / ((float)BUFFERSIZE/2.0f), n++) {
    RECT barrect;
    barrect.left = x;
    barrect.right = x + (framewidth / ((float)BUFFERSIZE/2.0f));
    barrect.bottom = framerect.bottom;

    barrect.top = framerect.bottom -
      (g_FT->GetAmp(n) / 50.0f * (float)frameheight);
    if (barrect.top < framerect.top) { barrect.top = framerect.top; }
    FillRect(hdc, &barrect, (HBRUSH)GetStockObject(GRAY_BRUSH));

    // fill in the top portion black
    if (barrect.top > framerect.top) {
      barrect.bottom = barrect.top;
      barrect.top = framerect.top;
      FillRect(hdc, &barrect, (HBRUSH)GetStockObject(BLACK_BRUSH));
    }
  }
}

This is bare-bones Win32 API GDI code, so it's a little involved, but it's not doing anything complex. It simply loops from the left side of the output rectangle to the right, calculating the height of each bar and drawing two rectangles, one gray (the bottom of the bar), and one black (the top of the bar). In this manner, some of the flickering is reduced, since we're not clearing the whole rectangle and then immediately drawing bars on top of it.

If you're so inclined, I encourage you to make this look better. A common effect is to make a redto-green gradient on the bars, making the spectrum analyzer a little more colorful.


FAST FOURIER TRANSFORMS

There's one big problem with the code I've presented in this chapter's sample program: it's too slow! I've tried to make the code as easy to follow as possible, but that means that it's nowhere near optimized, and is therefore not suitable for anything other than teaching concepts. Even a small BUFFERSIZE value of 1024 will strain all but the latest hardware!

Tip

If you run the popular SETI@Home screensaver, you may have noticed that it spends a lot of its time calculating "FFTs." These FFTs are, in fact, Fast Fourier Transforms—the screensaver is using the Fast Fourier Transform to deduce the amplitude of frequencies in the source input signal.

To get any real mileage out of the DFT, it must be sped up. Fortunately, in 1965, Cooley and Turkey invented an amazing algorithm that speeds up the DFT by several orders of magnitude. This algorithm is called the Fast Fourier Transform, or FFT. The FFT does the same thing as a Discrete Fourier Transform, only it does it, well, wicked fast.

I don't have the space to cover it in detail here, but I've provided several links on your CD that will point you to explanations and implementations of the FFT. Your mission, should you choose to accept it, is to add an FFT method to CDiscreteFourierTransform, making it truly ready for use in real-time applications.


CONCLUSION

I hope this chapter has given you some of the core concepts you need to implement a cool-looking spectrum analyzer. Also, and more poetically, I hope that this chapter has given you an appreciation for the elegant simplicity of sound waves. To me, it's amazing that everything we hear— from opera to explosions to human voices—is because different frequencies of waves are causing different sections of our cochlea to resonate.

There's much more to the Discrete Fourier Transform than I've covered here. In fact, the DFT is one of those things that can be explained from many different angles, some of which use some highly advanced math. If you're the kind of person who enjoys jaunts down roads filled with imaginary numbers and integrals, I encourage you to explore in more mathematical detail the nature of the DFT and the FFT. I've provided some links on your CD to get you started.


Conclusion

Quite simply—this is the end. I hope that you've reached this point with a solid understanding of the basics of game audio programming, and that you've found fresh inspiration to go out and apply what you've learned in your own games.

The audio engine that you've worked on through the course of this book is ready and anxious for you to take it and apply it to your own projects. You can grab the final audio engine, contained on your CD as "FinalAudioEngine," or you can grab a snapshot of the engine from any of the chapters if you only want features up to and including that chapter.

I encourage you to explore the set of links I've provided on your CD. These links—several for each chapter or each major topic—are the perfect resources with which to increase your understanding of game audio and enable you to add even more cool features to the audio engine. This engine is yours—take it, change it, enhance it, and apply it to any project you'd like. Have fun!


Appendix A: Creating Dynamic Music

OVERVIEW

Take off your programmer hat, and put on your musician hat. This appendix walks you through the creation of all the parts needed for a simple dynamic musical score.

This tutorial concentrates on the concepts of creating dynamic music, not the mechanics of making the DirectMusic Producer GUI work. I provide the occasional screenshot that you can validate your progress against, but if you're having issues with the DirectMusic Producer GUI, you should consult the online help and/or the tutorial that Microsoft provides with the software.

Also, this tutorial keeps things fairly simple, musically. In real life, you'll want your scores to be more interesting than what's presented here; in this appendix, I provide only the bare minimum needed to clearly illustrate dynamic music's features. This means that at the end of this tutorial, you'll have a piece of dynamic music containing only two or three parts. Of course, once you read through this appendix, you can always go back and flesh out the music!

You'll start by learning how to install DirectMusic Producer, and how to create static (unchanging) music using it. You'll then learn how to add elements that change your static song into a dynamic song.


INSTALLING DIRECTMUSIC PRODUCER

First thing's first—you'll find the setup program for DirectMusic Producer on your CD, as part of the DirectX SDK. I'm not going to walk you through setup—it's nothing special. Consult the DirectMusic Producer documentation if you run into trouble.

Setting Up Your MIDI Keyboard

Tip

If you're planning on doing any serious music composition, and you don't have a MIDI keyboard, stop right now and go get one. Trying to use DirectMusic with just a mouse will infuriate you, trust me. Even a small one-octave keyboard, which you can get for around $120, is worth it.

But, if you absolutely can't afford a MIDI keyboard, there is an application on your CD, in the Goodies folder, that will turn your regular QWERTY keyboard into a MIDI keyboard.

To set up your MIDI keyboard in DirectMusic Producer, click on the toolbar button that looks like a MIDI port (see Figure A.1).

Click To expand
Figure A.1: Click on the toolbar button that looks like a MIDI port to adjust your MIDI configuration.

The most important setting in the dialog is the MIDI input drop-down. You shouldn't need to change the other settings (echo, latency, and synthesizer), but if you're curious, DirectMusic Producer help will tell you what they're for.

You might think that you should be able to play notes on your keyboard and hear them now, but because of how DirectMusic Producer works, you can't just yet. To test your MIDI keyboard input, perform the following steps (see Figure A.2):

  1. Click the New toolbar button and select Style. DirectMusic will prompt you for the name of a project—the name doesn't matter, but it's "MidiTest" for the purposes of this example.

  2. You should now have a tree of objects on the left of your screen. Navigate to the Patterns folder, open it, and double-click Pattern1.

  3. In the right pane of the screen, there is a blue square with "1 (1): 1" on it. Double-click this and a keyboard should appear down the left-hand side.

  4. Click on a key of the keyboard. You should hear the corresponding note, and the whole section should turn light yellow. You should now be able to play notes on your keyboard and hear them.

  5. Choose Close Project from the File menu, and don't save your changes.

  6. Navigate to where the project was created in step 1, and delete it (unless you changed things, it should be in My Documents\DMUSProducer).

Click To expand
Figure A.1: A test project that you can use to see if your MIDI input is working.

If Your MIDI Keyboard Isn't Working

Here are some things to check if you're new to MIDI and it appears that your keyboard isn't talking to DirectMusic:

  • Make sure that the computer's MIDI Out is connected to your keyboard's MIDI In, and vice-versa. Do not connect In to In or Out to Out. This is by far the most common problem.

  • Ensure that DirectMusic is using the right MIDI input as explained above.

  • Ensure that the sound output device is also correct (in that same dialog), and that nothing is accidentally muted (hey, it happens!).

  • Check to make sure your MIDI port is installed correctly by going to your Device Manager, in Control Panel\System.

  • Make sure that there's nothing special you need to do to your keyboard to make it talk MIDI. Some keyboards require you to enter MIDI mode before they'll send data out the MIDI port.

  • In general, setting up MIDI under Windows is not a difficult task. I've never encountered a MIDI problem caused by anything low level: IRQs, old/buggy drivers, bad cables, and so on. Most of my MIDI problems have been caused by not telling my music software to listen for MIDI, or by not telling my keyboard to send MIDI.


CREATING NORMAL MUSIC

Now that you've got DirectMusic Producer installed and your MIDI keyboard hooked up, it's time to start creating a song! My process (which you are free to follow) goes like this:

  1. Choose instruments that will be used in the music.

  2. Create the first band (in other words, set up default panning and volume for the instruments).

  3. Create the patterns of music (this is usually the biggest step).

  4. String the patterns together in the segment.

  5. Tweak everything as needed.

  6. Save and enjoy.

Tip

If you don't want to walk through all of the steps in this section, I've included the project file for this appendix on your CD.

Remember, this section is strictly about creating normal music. You'll walk through all of these steps at their simplest levels (no dynamic features); then, in the later parts of this appendix, you'll revisit what you created in this section and change it from static music into dynamic music.

Creating the Objects

Every journey begins with a single footstep, and every piece of dynamic music begins with a new DirectMusic project. Click New from the file pull-down menu, and select Project. Next, create a new style using the same dialog. Notice how when you created a new style, you automatically got a new band and a new pattern as well.

Tip

The list of objects in the new dialog is multiselectable, so in the future, you can create a new project and a new style in one fell swoop by holding down control and clicking on both Style and Project.

This example is small, so you can get away with not renaming the objects DirectMusic creates into names more descriptive than Band1, Pattern1, and so on. However, when creating your music for real, you should get in the habit of assigning descriptive names to your patterns.

Icons, Extensions, and File Formats

Before you go any further, you should learn how to recognize the objects you've just created. If you're staring in bewilderment at all the icons in DirectMusic's tree view, this section's for you. Table A.1 provides the icons, file extensions, and short descriptions of the objects available in DirectMusic Producer.

Table A.1: The Objects of DirectMusic Producer
Icon | Object | Normal | Runtime
 | Project—The top-level object | PRO | N/A
 | DLS Collection—Custom instruments and samples (drumloops, and so on) live in DLS Collections | DLP | DLS
 | Instrument in a DLS Collection | Embedded in DLS | Embedded in DLS
 | WAV file in a DLS collection | WAV | Embedded in DLS
 | Style—A collection of Patterns, Motifs, and Bands | STP | STY
 | Band—List of instruments used in a pattern | BNP | BND
 | Segment—Tells DirectMusic which style (and pattern within that style) to play | SGP | SGT
 | Audiopath—for example, stereo, mono, stereo with reverb, and so on | AUP | AUD
 | Motif—a musical sound effect triggered by the game (and guaranteed in Style to be in key!) | Embedded | Embedded
 | Pattern—A chunk of music ranging between 1 and several measures in in Style length | Embedded | Embedded
 | Script—Tells DirectMusic what to do to play a sound effect | SPP | SPT
 | WAV File—Used for sound effects or sampled music (for example, guitar riffs) | WVP | WAV
 | Chordmap—Used for dynamic music composition | CDP | CDM
 | Container—Contains custom objects | COP | CON
 | Tool Graph—Applies tools and effects to certain PChannels | TGP | TGR

The objects of DirectMusic can be saved to disk in two different ways: regularly or in runtime format. As you're creating your music, you save it normally, but when you're finished and you want to hear your music in your application, you should save in runtime format. The runtime formats don't have editing data in them, so they're a little smaller than their regularly saved counterparts.

Creating a DLS Collection

For this song, you'll use a custom drumloop sample to add some flair. To do this, you need a DLS collection; if all you need are the instruments offered by the General MIDI sound set, you won't need to create a DLS collection.

Create the DLS collection by clicking the New button and selecting DLS collection. Next, look at it in your tree view—you'll see it's broken into two folders: instruments and waves.

The waves folder contains the raw WAV samples used to create instruments. This means that your first step is to add a WAV sample to the waves folder. Do this by right-clicking on the folder and selecting Insert Wave. Navigate to and select the wave file you want to insert (in this case, [image] DrumLoop.wav, on your CD). Next, double-click the DrumBeat icon that you just created, and the WAV file will open up in the pane to your right.

DirectMusic Producer provides some basic WAV editing functions, but for anything beyond simple manipulation tasks, you'll find that it's easier to load your WAV into another application (for example, GoldWave or Sound Forge).

Play the sound, just to make sure it loaded correctly. There's a couple of green play buttons in your toolbar, near the center of your screen. The one with the bar plays from the beginning, whereas the regular-looking one plays from the cursor position. You don't necessarily have to be looking at something to play it; you can use the combo box to the left of the play button to select any object in your project.

Now that you have a WAV file loaded, you can create an instrument. Note that the instrument auto-creates when you create it from a WAV, so look for it in the tree of objects. DirectMusic conforms to the DLS specification, and the DLS specification has many options for how to turn a WAV file into an instrument. According to the specification, different musical ranges can be assigned different WAV files—for example, you could have one WAV file to represent the lower octaves of your instrument, and another to represent the upper octaves. You could even go so far as to have a different WAV for each note of the instrument!

For this example, since you're using a drumloop, you don't care about notes. You want the drumloop WAV to play at its correct frequency, whenever you tell it to. So, set both the high and low note ranges to C5, using the note range edit boxes on the main wav editing page. C5 (a.k.a. the C of the fifth octave) is the note that corresponds to the default frequency of the sample; that is, DirectMusic assumes (unless you tell it otherwise) that the WAV file is a WAV file of something playing a C5 note. If you play one note lower (B4), DirectMusic pulls the pitch of the sample down to that note; if you play a C6, it pulls it up an octave. If you play a C5, DirectMusic doesn't change the pitch of the sample.

Next, turn your attention to the group of controls under the Instrument heading (see Figure A.3). Check the Drums checkbox. This tells DirectMusic that this instrument is actually a drum kit. A drum kit is a special instrument that's a collection of drums—each note is a different type of drum. For example, C5 might be a bass drum, the next note up (C#5) might be a snare drum, and D5 might be a hi-hat. You can create dozens of different drums and assign them all to one instrument.

Click To expand
Figure A.3: The controls to set an instrument's ID, and whether or not it's a drum set.

This makes it very easy to input drum tracks. If each drum were its own instrument, you'd have to have different parts for all of them, which means you'd have to first record the bass drum, then the snare drum, then the hi-hats—everything separate. However, with a drum kit, you can record all the drums at once by hitting different keys on your keyboard. It's still not the same as playing real drums, but it's much easier than laying each drum down separately.

Immediately to the left of the drums checkbox are three numbers that jointly specify the instrument's ID. MSB stands for Most Significant Byte; LSB is Least Significant Byte. My advice is, unless you have more than 127 custom instruments, just leave the MSB and patch numbers set to zero and use the LSB as the instrument ID. A good strategy is to group like instruments together—for example, start your strings as LSB 10, your woodwinds at LSB 20, and so on.

For this example, the instrument ID really doesn't matter, so I just picked MSB 0, LSB 1, patch 0. The shorthand for this is (0, 1, 0).

Tip

For more information on MSB, LSB, and patch numbers, see the DirectMusic documentation, at Playback and Recording\Patch Numbers and Bank Select.

The next step to creating this custom drumloop is to make sure that your articulation is set up correctly. The term articulation refers to the note's relative volume over time. Different instruments have different articulations; a note played by a piano, for example, gets loud very quickly, then slowly fades off. A note played by a different instrument may start soft and get louder over time. The bleeps of an analog synthesizer are instant-on, instant-off, like the flip of a switch—they have no fade out period.

If you were creating a real instrument here, you'd want to adjust the attack, decay, sustain, and release (ADSR) for the volume of the instrument. This is a MIDI thing; attack refers to how quickly the note reaches its maximum volume, delay speaks to how quickly it fades from that maximum volume, sustain represents the volume the note "holds," and release specifies how quickly the note fades out completely (see Figure A.4). Note that there are also four more ADSR values for the instrument's pitch, although these values are rarely changed.

Click To expand
Figure A.4: Every instrument in a DLS1 collection has points for attack, sustain, decay, and release (ADSR).
Tip

At the time of this writing, there are two versions of the DLS standard: DLS level 1 and DLS level 2. The older DLS level 1 specification supports the attack, decay, sustain, and release variables for instruments, but DLS level 2 allows even more precise tweaking, by providing Delay and Hold variables as well (see Figure A.5). If you want to use these, uncheck the DLS1 checkbox, but realize that now your instrument set will only work if the software synthesizer supports DLS2 (DirectMusic, starting with DirectX 8.0, does). For more information on the differences between DLS1 and DLS2, check out the DirectMusic Producer online help, at Playback and Recording\DLS Designer\DLS Level 1 and Level 2.
Click To expand
Figure A.5: DLS level 2 instruments have delay, attack, hold, decay, sustain, and release points.

Since you're creating a drumloop sample, you don't want anything fancy, so set the attack, decay, and release values at 0.001, and the sustain value at 100%. This tells DirectMusic that the note comes on instantly, stays at full volume, then stops itself instantly—exactly what a drumloop should do.

One last thing before you're done creating your custom instrument—uncheck the Note Overlap Allowed checkbox. This tells DirectMusic that if this instrument is playing, and another note for it is encountered, DirectMusic should stop the first note from playing, instead of playing both notes on top of each other. In other words, the instrument has only 1 level of polyphony—only one instance of the sound can be playing at any time. This is great for drum samples and drum loops; it's not so great for regular instruments.

Go ahead and press the C5 key on your keyboard a few times, and notice how the sample starts as soon as the key is down, and stops as soon as you lift your finger off the key. If this whole ADSR business is new to you, play around with the variables a little and see how they influence what happens when you hit that C5 key.

Congratulations, that's it—you've just created the custom DLS instrument for this example song. There are options in this dialog box that I didn't talk about, because they're not usually used and/or they're not relevant to this example. To fill in the blanks, consult the DirectMusic Producer documentation, at Playback and Recording\DLS Designer.

Tip

An easy assumption to make would be that you have to set up custom DLS collections if you want sound effect segments you can play on demand. You could create a DLS collection, put in your sample, and then just create a segment that played a style containing only one pattern, and in that one pattern there would be only one note—a C5 note of the sound effect you want to play.

This would work, but an easier way is to use wave-based segments (that is, segments whose content is a wave file, not notes). DirectMusic supports this, and it's the official and usually best way to integrate sound effects into your project. Creating wave-based segments is covered in Chapter 12.

Setting Up the Band

Now that you've created the drumloop as a custom DLS instrument, it's time to turn your attention to the style object, and specifically, to the instruments you'd like to use in the patterns of that style.

First, the definition: A band is a subset of the General MIDI sound set and any DLS collections you have; a band also tells DirectMusic the default volume, panning, octave, and transposition values for the instruments of the style. Every instrument in a band exists in either the GM sound set or in a custom DLS collection, but not every GM sound set and DLS sound is necessarily in a band. Or, in programmer terms, a band is an added level of indirection for choosing an instrument.

Thankfully, setting up a band is less complicated than turning a drumloop WAV file into a DLS instrument. To set up your band, double-click on Band1, inside the Bands folder of your style. You should see the Band Editor, as shown in Figure A.6.

Click To expand
Figure A.6: The band editor of DirectMusic Producer.

First, assign your custom drum sample to PChannel 10. Why channel 10? The MIDI spec says that drum kits must be on channel 10. You told DirectMusic that your drumloop was a drum kit when you checked the drums checkbox in the DLS editor; therefore, DirectMusic will only let you assign that instrument to channel 10.

Double-click on PChannel 10 in the left list box (or right-click it and select properties, or press F11—it's all the same). Check the instrument checkbox, then click the big button underneath it. From the pop-up menu that appears, choose the bottom entry: Other DLS. A dialog box will appear in which you can choose a DLS collection and an instrument within that collection to assign to that performance channel. If you've followed the tutorial correctly so far, you should see your (0,1,0) instrument in the list. Click OK to assign this instrument to PChannel 10. Close the properties window and notice that the list box has changed.

You also need to set up the two other (real) instruments you'll use in this example music. The music will have two parts—harmony and melody. The harmony will use PChannel 1, and the melody will live on PChannel 2. Since you'll be inputting arpeggios for the harmony, I'd suggest using a piano; for the melody, I chose the "Voice Oohs" GM sound.

The graph to the right allows you to adjust the relative volume and panning for the instruments in your band.

Tip

By default, only 16 PChannels are listed in the band editor. You can add more (or delete some of them) by rightclicking inside the list.

Volume runs vertically, panning horizontally. If you wanted to make an instrument play at maximum volume and center pan, put it on the top, centered. However, you generally don't want to do that just yet. I prefer to adjust panning and relative volumes after I have a pattern put together, so for now, move all of the instruments to the exact center of the graph; you can come back and spread them out later once you have something you can listen to.

That should do it for the band!

Creating Your First Pattern

You've created your DLS instruments and assembled your band—now you can create some music. In this section, you'll create one four-measure section of the example song.

The Pattern Editor

Double-click Pattern1, inside the Patterns folder of your style, and the Pattern Editor will appear. Get comfortable with this editor; it's where you'll be spending the bulk of your time. Here's an explanation of everything in this editor, starting at the top and working down.

At the very top is your time signature track. The heavy lines are bars (measures), and the dark blue lines are beats within a measure. The red line is the play cursor. You'd think that to change the time signature, you'd right-click here, but instead, you have to right-click on the style node itself, and select properties.

Below the timeline is the chord track. This tells DirectMusic which chord goes with each of the measures in your song. You'll learn more about the chord track in a few pages, when you get to the section on Advanced Dynamic Music.

Next are the instrument parts. Right now, there's only one instrument part. Notice the maximize and minimize buttons for each part, on the right—go ahead and maximize a part, and you'll be treated to the piano roll display for that part, allowing you to edit notes and variations. At the top are 32 buttons arranged in a 16x2 grid. These are the variation buttons; you'll learn more about them when you get to the Dynamic Music section.

Tip

Note that you can stretch the main windows vertically by hovering your mouse over their vertical boundaries (watch for the icon to change) and dragging.

To the left of the variation buttons are the vertical zoom in and zoom out buttons; their horizontal counterparts are located to the right of the horizontal scroll bar at the bottom of your window. Table A.2 describes the hotkeys for zooming.

Table A.2: Zooming Hotkeys
Hotkey | Result
A | Vertical zoom in
Z | Vertical zoom out
I | Horizontal zoom in
O | Horizontal zoom out

To the left of the vertical zoom buttons is a button that allows you to change the instrument for this particular part. Note that what you're really doing here is changing the band; selecting a new instrument here is equivalent to going to the band editor and selecting a new instrument for that PChannel. Next to the instrument name is a little band button that acts as a shortcut to the band editor.

Now you get to the piano roll display. This is the core of the pattern editor. Time runs horizontal, pitch vertical, and notes are represented as bars. The louder the note, the taller the bar, and the longer the note, the wider the bar. You can click a piano key with your mouse and hear that note; you can also insert a note at the current cursor position (red line) by hitting the insert key. You can select and manipulate notes using common selection methods and functions (cut, copy, paste, and so on).

Below the piano roll is the variation switch point track. This track tells DirectMusic when it's safe to switch variations in mid-pattern. You'll learn more about it in the Advanced Dynamic Music section.

The final, lowest track is the CC, or Control Curve track. This is where control commands, such as pitch bend and modulation, live. You'll learn how to set up a control curve in this section.

Entering Some Notes

Enough chat—now it's time to make some music. You'll start by lengthening Pattern1 inside the Patterns folder of your style. Do this by opening the properties for the pattern and upping the length of the pattern. I like to over-lengthen the patterns at first, and adjust them to their proper lengths after I've recorded data into them. Super-long patterns allow you to record several takes at once, without having to hit stop, rewind, and record again. Then, once you have several takes recorded, you pick the one that you like best, delete everything else, and move your good take to the start of the pattern (either through cut and paste or by selecting the notes and dragging them all at once).

To enter notes, maximize the first part and highlight the piano roll by clicking anywhere inside it (it will turn light yellow). Next, move your cursor to the beginning of the song by clicking, or by pressing the home key. Finally, click the record button to start recording. Alternatively, you can also create notes by clicking where you want the note, and hitting the insert key—it's tedious, but it's one way of inputting music if you don't have a MIDI keyboard.

Summon your mad keyboarding skills (you do have mad keyboarding skills, right?) and input the eight measures shown in Figure A.7.

Click To expand
Figure A.7: Eight-measure background piano part for Pattern1.
Tip

As you're recording, if you'd like the computer to keep time for you, click on the little metronome toolbar button.

The musical theorists out there will instantly recognize the chord progression in Figure A.7 as a simple I—VI—IV—V progression in the key of C. This means that the pattern starts in the key of C major, moves to A minor, F major, and then to G major. Keep this in mind; it will become important in later sections.

Quantizing Notes

After you input your notes, you'll notice that (unless you're a very, very good musician) you hit certain notes a little early, and others a little late. You should correct these so that everything is lined up exactly on the beat or grid that it's supposed to be on. If you want to preserve the "live human performance" feel, you can add random delays using the note properties window shown in Figure A.8 (after you've corrected the notes).

Click To expand
Figure A.8: The note properties window.

To begin lining up your notes, right-click in the piano roll and select Snap To None from the context menu that appears. This will allow you to move the notes around freely. You can move a note graphically by grabbing its middle (watch for your mouse cursor to change), or numerically using the note properties window. Don't worry about getting the notes lined up perfectly—just get them pretty close.

Once you've done that, you can have the computer take over. This is like washing dishes—you get them sort of clean and the machine gets them really clean. The computer can make your notes start at exactly the right time through a procedure called quantization. Quantization is just a fancy word for snapping notes to the nearest beat you specify. You can choose to quantize based on quarter notes, eighth notes, and a wide variety of other things. For this example, you'll want to quantize on eighth notes, so go to the Edit pull-down menu, select Quantize, adjust the parameters as needed, and hit OK.

It may take a couple of tries to get the note timings exactly right. DirectMusic Producer may snap some notes incorrectly during quantization, so you should play your work in progress after you've finished lining up the notes to make sure it sounds correct.

Adding Additional Parts

What you just entered was just the background chords; now it's time to add the melody. First, jump back over to your band editor and add Voice Oohs (or your favorite melody-carrying instrument) to PChannel 2. Then, jump back into the pattern editor and right-click on the piano roll. Click Add Pattern Parts, near the bottom of the context menu, to add a new part for PChannel 2.

Tip

Alternatively, you could have recorded both melody and chords at once into the piano track. Then, once you had both recorded, you could cut the melody out and paste it into the second part.

Now you can highlight part 2's piano roll and input the melody in Figure A.9. After that, quantize the notes as explained in the previous section.

Click To expand
Figure A.9: The melody part for Pattern1.

Finishing Up Pattern Input

Now that you've seen the process in detail, gain some experience by practicing. Create another pattern and input the two parts and eight measures shown in Figure A.10.

Click To expand
Figure A.10: Background and melody parts for Pattern2.

You should now have a full 16 measures of music.

Tip

If you already have a favorite piece of software that can record in MIDI, you can use that to record your playing and just import MIDI files into DirectMusic Producer. In DirectMusic Producer, select Import File into Project off the File menu, and go from there.

Stringing Patterns Together Inside the Segment

The final step to creating the basic song is to string these five patterns together. To do that, you'll need a segment—go ahead and create a new one, 16 measures long.

DirectMusic Producer will automatically prompt you to add tracks; by default, a segment has only a time signature track. You want to add a style track and a groove track.

The style track tells DirectMusic what styles to play, and when to play them. In this example, the style track is very straightforward—you want it to play the style you created in the previous section, for the entire length of the segment. Position your play cursor at the beginning of the segment (hit the home key) and press the insert key to insert a new entry into the style track. The style properties dialog will appear, allowing you to select a style. You should see your style listed in the drop-down list; select it, and set it to take effect at measure 1. Also note that when you insert your entry, a new band track appears, preset to the band in your style.

Now DirectMusic knows that at measure 1 of your segment, it should use some particular pattern inside the style you specified. But how does it know which pattern? If you play the segment at this point, you'll hear one of your patterns at random. That's great for certain situations, but what if you want your patterns to be played in a certain order?

That's where the groove track comes in. The groove track provides two vital parameters—first, it sets the groove level, which DirectMusic Producer uses to "rule out" patterns in the style (patterns with a groove range outside the current groove level cannot be played). The second vital parameter tells DirectMusic what to do when more than one pattern can be played. By default, DirectMusic plays one pattern at random, but you can change this to have it play all of the valid patterns in order—exactly what you need for this demo song.

Right-click on the groove track and select Properties. From the Properties window that appears, select Sequential from First from the Repeat Pattern drop-down menu. Now, play your song—you should hear all of your patterns played in the correct order.

Looping the Segment

One last thing—you need to make the eighteen measures of your segment loop infinitely. Rightclick on the segment in the tree and select properties. From the segment properties window, select the loop tab, then click the Infinite checkbox. As you can see, there's other advanced looping parameters you can use, but most of the time a simple infinite loop will get the job done just fine.

Saving as Runtime Format

At this point you've done enough to be able to use this (short) piece of music in your game. But, to do so, you need to save it in runtime format. This is as easy as selecting Runtime Save All from the File menu.

Tip

You can embed copyright and other miscellaneous information into your music by using the info tab found in most of the property windows for your objects. The info tab allows you to specify a subject, author, copyright, and version, along with any additional text you'd like.

When you do this, DirectMusic Producer saves each file in your project in runtime format. (much quicker than compiling source code!) It puts the runtime files in a RuntimeFiles directory inside your project's home directory, usually inside My Documents\DMUSProducer.


DYNAMIC MUSIC

Now that you have mastered the essentials of recording patterns, creating bands, assembling styles, and unifying everything under a segment track, you can go back to the example song and start adding some dynamic elements. This section will walk you through some basic things you can do to make your song sound different each time it is played.

Adding Variations

The easiest way to get your music sounding dynamic is to add some variations to the parts of your patterns. A variation is simply an alternate track that plays instead of a main track.

The secret to adding variations to your parts is the array of 32 buttons located immediately above the piano roll. Each part can have up to 32 variations, so there are two rows of 16 buttons. Each button conveys three crucial pieces of information: whether the variation is active, whether it's enabled, and whether it contains notes.

First, the easy one—if a variation contains notes, the color of its number will be black. If there are no notes in that variation, the number will be white. Next, if a button is pushed, the variation is active (meaning that its notes are displayed in the piano roll, and you can edit them).

Finally, a button's background color tells you whether its enabled or disabled. A light background means enabled; a dark background means disabled. Being enabled or disabled is different than being active or inactive. An enabled variation is one that can be randomly chosen for playback; a disabled variation is one that DirectMusic can never select.

By default, all variations for a part are active and enabled. The next time you create a pattern, notice that all the variation button numbers are white, meaning that no notes have been entered. As soon as you input your first note, they all turn black.

That's right—little did you know that you were editing all 32 variations simultaneously! This is done so that you can record your base melody into all 32 variations at once, and then tweak each variation individually. So, go ahead and do that now—to start, click the variation 2 button (variation 1 will be the original, unchanged melody). It pops up, meaning that any notes you input now will go to all variations except variation 2. But wait! That's not what you want! To invert all the buttons, double-click on button 2. Now all of them except button 2 should be up (inactive).

Now you can go crazy and start adding subtle variations to the part. You might want to invert an arpeggio, add a frill, syncopate a little—whatever suits your mood. When you've got variation two all set, you can move on to variation three, and so on. Usually two or three variations get the job done quite nicely. It's very rare to use all 32!

When you're done, play your pattern back. Notice that DirectMusic chooses variations at random. If you want it to do something else (for example, play all the variations in order), you can give it new orders via the part properties window (see Figure A.11). To get to part properties, right-click anywhere in the piano roll, except on a note, and select properties. In the variations group, there's a Play drop-down box that lets you choose one of several different behaviors (these behaviors are akin to what you can choose for the groove-level behavior—see the DirectMusic Producer docs for a complete explanation).

Click To expand
Figure A.11: The part properties window.

Creating an Embellishment

An embellishment is a special kind of pattern, usually an intro, fill, break, or end pattern. In this section, you'll create an introduction pattern and set it up to play at the beginning of the segment.

Start by creating the introduction pattern itself—make a new pattern, then go to its properties and press the intro button. This tells DirectMusic that this pattern is an intro pattern. Now, just create a cool introductory pattern (like the one in Figure A.12), and name it "Intro."

Click To expand
Figure A.12: The intro embellishment pattern.

Next, do the same thing to create a break pattern. Input the measures shown in Figure A.13, mark the pattern as a break, and rename it "Break."

Click To expand
Figure A.13: The break embellishment pattern.

Now you need to set up your segment to play your intro and break embellishments. Right-click the icon of your segment in the tree view, and select properties. Put in a new length of 23 bars, then go to the loop tab and set your loop start to bar 4, and your loop end to bar 22. Make sure the Infinite checkbox is checked, so your segment will loop forever.

Okay, now the segment's long enough that you can input your intro and break measures. Rightclick on the beginning entry in your groove track and select properties. In the properties window, highlight the Intro entry in the embellishment list box. Set your groove level to 1. This tells DirectMusic that it can play any pattern marked as an intro that has a groove level of 1.

Now, go to the properties for Pattern1, and set its groove range to 1-1. This makes it so it's only possible for DirectMusic to play Pattern1 when the groove level is set to 1. Do the same thing for Pattern2, setting its groove range to 2-2.

Go back to your segment editing window. Since your intro is three measures long, position your red vertical line cursor at the beginning of measure four, and press the insert key. Now, make sure none of the rows in the embellishment list box are selected, and make sure your groove level is still at 1. Also, make sure that your repeat pattern drop-down box contains Sequential from First. This tells DirectMusic that at measure 4 it should start playing any pattern that's not an embellishment and that has a groove level of 1 (Pattern1 in this case).

Next, go to measure 12 and insert another entry in your groove track that changes the groove level to 2. This is what will force DirectMusic to play Pattern2—Pattern2 is the only pattern with a groove level of 2.

Finally, go to measure 20 and insert another groove track change. This time, put your groove level back at 1 and select Break from the embellishment list box.

If you play your segment at this point, you should hear your Intro pattern, followed by Pattern1 and Pattern2, then your break pattern. After your break pattern, DirectMusic should loop and play Pattern1 again (notice that the loop range excludes the intro measures, since you only want those to play once).

That's all there is to it! Creating an embellishment is as easy as creating a pattern and pushing a button!

Adding Motifs

Dynamic music is cool not only because it's ever-changing, but also because it can respond to events in the game. There are two main types of responses dynamic music can provide. First, if something major happens, you can completely change the sound of the song. For example, if your player gets ambushed by a huge group of enemies, the dynamic music can quickly raise its intensity; when the player defeats them all, it can slowly decrease intensity, back to its idle state.

That's one type of behavior, but you're not going to learn about that one until the section on Creating Alternative Segments. The second thing dynamic music can do is swell momentarily or play a quick jingle when something interesting happens. For example, if your player picks up the Raccoon Helmet powerup, you might want the music to play a few notes from the Theme to the Raccoon Kingdom (bear with me here, I'm making this up as I go). You're not really changing the music so much as you are playing a sound effect over the music; a sound effect that just so happens to be made out of notes.

Traditionally, it's been hard to create musical sound effects that fit with the music because it's been difficult to determine what's going on musically when an event is triggered. If your player picks up the Raccoon Helmet when the background music is in the key of A, and the Theme from the Raccoon Kingdom is in the key of B, you absolutely don't want to mix the two! What you want is for your code to take the Raccoon Theme and transpose it down a step, so that it plays in A, in harmony with the A background music.

Chords for Composition and Chord Tracks

To have DirectMusic do this, you need to tell it a bit more about your music than just your notes. DirectMusic can't automatically infer what key your music is in, so you have to tell it. Take a peek at the track immediately above the first part piano roll in your pattern editor. This is called the "Chords for Composition" track. It tells DirectMusic what key your music is in at that particular moment. You can think of this track as the DirectMusic equivalent to the key signature in traditional music notation.

However, just because a currently playing pattern was written in the key of C doesn't mean it's currently playing in C. You can have DirectMusic transpose your patterns for you, by using a Chord Track inside the segment. For example, you could create a pattern and set its Chords for Composition track to be CM7 (C Major, 7th octave) for the entire pattern, and then create a Chord Track in your segment that's set to DM7 (D Major, 7th octave). When DirectMusic sees that DM7 in your segment chord track, it will think to itself, "hmm, this pattern was written in CM7, and they want it in DM7 now, so I'll transpose it upward two semitones." In more mathematical terms, you can imagine that DirectMusic is "subtracting" 1 key from another, to get the key delta, which it then applies to your pattern.

For this reason, if you want your music to be played exactly as you wrote it, you need to make sure that your pattern's Chords for Composition and your segment's chord track are equal. This will ensure that the delta between composition chord and currently playing chord is zero, and DirectMusic won't transpose your song.

Inputting a Segment

That's the theory. Now you'll put it to use. For starters, create a new motif (right-click on the Motifs folder and select New) and input the motif shown in Figure A.14. As you can see, it's nothing complex—just a few notes, played quickly enough so that they sound more like a sound effect than a measure of music.

Click To expand
Figure A.14: A simple motif pattern.

While you're at it, add a chord track to your segment by right-clicking and selecting Add Tracks. You should see the new chord track appear in your segment editor, defaulted to 2CM7.

Tip

Motifs are just specially-marked patterns, so they can have multiple parts and variations just like normal patterns. Since you now know how to add variations, feel free to create some variations for your motif—it'll make your music slightly more dynamic. Just don't overdo it!

Setting Up the Chords

Think back to how DirectMusic decides to transpose your patterns—it looks at the difference between the Chord for Composition and the current segment's chord. The same thing applies to motifs. When your code tells DirectMusic to play a particular motif, DirectMusic looks at the difference between the motif's Chords for Composition and the current value of the segment's chord track. If your motif is in EM7, and the segment is currently playing GM7, then DirectMusic will transpose your motif up three semitones, so that it plays in GM7, in harmony with the segment.

But, at this point in the tutorial, you haven't modified any chords for composition tracks, or segment chord tracks. So, all chord tracks are set to their default value, CM7. So, no transposition occurs, and all is right with the world. If you don't want to use motifs, leaving everything as CM7 will work fine. But, if you want motifs that transpose themselves into the correct key automatically, you need to manipulate your motif's chords for composition track, and your segment's chord track.

The motif you input in the last section is, in fact, based on CM7. So there's no need to change the motif's chords for composition track. Also, this motif sounds great played over the top of Pattern1. If you don't believe me, start pattern 1 playing, and then use the secondary playback toolbar (shown in Figure A.15) to start your motif playing.

Click To expand
Figure A.15: The secondary playback toolbar.
Tip

Also, while you're experimenting, notice that regardless of when you hit the play button, DirectMusic always starts your motif on a beat boundary. Of course, that's configurable, but in most cases, lining things up on a beat boundary works great.

The problem is when you play your motif over Pattern2. Pattern2 follows an F—G—F—G progression, and your C-based motif doesn't quite fit. Sure, it's not as bad as playing a B chord over an F, but it's not great. Suppose you want your motif to play in the same key of F or G that's currently playing. That is, you want it to play in F during measures 1, 3, 5, and 7, and in G during measures 2, 4, 6, and 8 (following the chords of Pattern2).

To set this up, go to Pattern2 and focus your attention on the Chords for Composition track. You should see 2CM7 listed as the chord for measure 1. Right-click on that and select Properties. This will bring you to the chord properties window, shown in Figure A.16. In this window, you need to do two things.

Click To expand
Figure A.16: The chord properties window.

First, you need to push the base key and octave down from 2C to 1F. Click the bottom spin arrow seven times to move your base key down to 1F. Notice as you do that the red keys shift down with you. These red keys are the notes of the chord the motif is based in. You need to make sure that the four red notes of each channel (1-4) form an F major chord (F-A-C-F). Click a piano key to toggle it.

Tip

When changing the red piano key notes, start at the bottom channel, channel 1. As you change the red notes on channel 1, DirectMusic will automatically change and synchronize channels 2, 3, and 4. Normally, you'll want all four channels to have the same red keys, so this is a useful shortcut.

The text box that currently contains M7 is just a place for you to note, in musical terms, the chord you're inputting using the red piano keys. DirectMusic does not look at the contents of the text box; it's strictly for human musician use.

There are many other fancy things you can do with the chord properties box, but for the purposes of this tutorial, this is all you need to do. Close the properties window, and move your cursor (vertical red line) to the start of the next measure. With the Chords for Composition track highlighted (in light yellow), press the Insert key on your keyboard. Now, follow the same process in the chord properties window to input 1GM7 as the Chord for Composition of this measure. Continue doing this for each measure, alternating between 1FM7 and 1GM7, for all of the measure of Pattern2 and Pattern2HighGroove.

Just to keep things straight, you might want to enter M instead of M7 into each value, and make sure the red notes form a major chord, and not a major seventh chord. Go through all of the patterns and the segment and use the chord properties window to change all of the default 2CM7 values to 2CM.

Wait! You're not done yet! If you play your segment now, you'll notice that DirectMusic is now transposing Pattern2 and Pattern2HighGroove, and it sounds horrible. The reason DirectMusic is transposing is because you've changed Pattern2's Chords for Composition, but you haven't changed the segment's chord track. You need to go to the segment and add values to the chord track that correspond to the chords of the pattern playing underneath. This means you should enter a 2CM value at measure 1 and 20, a 1FM at measures 12, 14, 16, and 18, and a 1GM at measures 13, 15, 17, and 19. If you do this, the segment's chord will always equal the pattern's chord, and DirectMusic won't transpose anything.

Now for the final test—play your motif any time your segment is playing, and notice how DirectMusic automatically transposes it to F or G based on the segment track. Cool, isn't it?

Of course, you can make chords, patterns, and motifs that are much more complicated (and interesting!) than what I've illustrated here. DirectMusic can transpose between practically any two chords. For example, it will happily transpose major to minor—all you have to do is set up the red piano keys correctly in the chord property window. So, experiment!

And Last, a Word about Chord Paths

Before we wrap up this section about chords, I wanted to mention one other powerful feature of DirectMusic—you don't have to feed it the chords for each measure by hand. You can let it choose chords on its own!

When used properly, and with patterns that support it, this can lead to incredibly dynamic music. Essentially, all you do is set up a "chord map," telling DirectMusic the chord progressions it's allowed to make throughout the segment. DirectMusic will automatically pick a path through this chord map, and you'll get truly dynamic music.

For more information on how to do this, consult the DirectMusic Producer documentation, and read the tutorial provided with DirectMusic Producer.

Creating Alternative Patterns

One last thing to do before you're done, and that is to create alternative patterns for a higher groove level. That way, your game can control the intensity of the music. Your code can control a groove-level modifier, which by default is set to zero. This groove-level modifier and the current groove level (set by the segment) are added together to determine the final groove level (and therefore the patterns that can be played).

You've already created two patterns, with groove levels 1 and 2. You'll now create another two patterns at groove levels 11 and 12. These two new patterns (call them Pattern1HighGroove and Pattern2HighGroove) will contain the same music as in Pattern1 and Pattern2, but that music will be more intense. Well, not that much more intense—all it will have is a drum track, but you get the idea: when something happens in your game, your code can set the groove modifier to 10 to play the "high groove," more intense patterns.

There's not much to adding a drum track. Begin by copying pattern1 (use the clipboard) and naming it Pattern1HighGroove. Insert a new part, and make its instrument be your custom drumloop. Then, for that part, just create one big note, played at C5, and lasting for four measures. At measure four, create another four-measure-long note, so your drumloop plays for the last four measures as well.

Do this same thing with Pattern2, naming the copy Pattern2HighGroove. Don't forget to go into the pattern properties and change the patterns' groove levels to 11 and 12!

Save as Runtime (Again)

And now, you're done (Whew!). Save your dynamic music in runtime format, and place it in the same directory as the source code to Chapter 11. That's it!


CONCLUSION

There's a lot that goes into creating a good piece of dynamic music. You learned the basics—variations, motifs, alternate segments, and chord mappings, to name a few concepts—but there's still several lesser-used concepts that this appendix didn't cover.

If you're interested in really becoming a dynamic music guru, you'll need to spend a lot of time creating dynamic scores and tweaking them, until you've got a grasp on all of the features of DirectMusic Producer. DirectMusic Producer is a complex piece of software, so take it slow and experiment often.

Also, keep in mind that it's harder to write dynamic music than it is to write normal music, in the same way that it's harder to write a (good) interactive story than a normal one—you have to fill in more paths. It's easy to get carried away, and spend too much time and effort for too little results. If you've got a section in your game that requires a thirty second piece of music, and the player is only going to go to that section once, don't waste your time putting in tons of variations and alternative segments. Build the thickness of the dynamic music to match how often the player is going to hear that music.


Appendix B: Tips and Tricks for Creating Game Audio

If you're playing both the role of programmer and sound designer on your game, this appendix is for you. I've collected a grab-bag of useful tricks for creating high-quality sound effects and music for your game, and I am presenting them here in the hopes that they'll get you going or give you an idea you may not have had before.

MUSIC

Game music is often the most overlooked aspect of game development. Game audio, in general, takes a back seat to a game's visuals, and within game audio, music often takes a back seat to sound effects and speech. Don't let this happen on your game! Music is one of the greatest tools you have for conveying emotion in your game. A game with below-average graphics but sweet music often comes out ahead of a game with stunning visuals but lackluster music (or no music altogether).

In this section, you'll learn a few tricks to get high-quality music into your game.

Being Musical without Being a Musician

You don't have to be a professional musician to create in-game music that sounds great. In the graphics world, even if you're not artistically inclined, there are a wide range of software programs that can enable you to create good and sometimes even amazing graphics. Programs like Bryce, Photoshop, Kai's Power Tools, Vue d' Esprit, can sometimes rescue your game from the "programmer art" pit (notice that I keep saying "sometimes"). Similarly, there are a wide variety of programs that can help you create high-quality music quickly and easily. I've included links and demo versions of some of these programs on your CD.

Getting Hit Songs in Your Game

Nowadays, the hip thing to do is include hit songs in your game. Before you crack open your MP3 collection and start building your game's soundtrack, though, you need to realize that doing this legally costs money, and often a lot of money (as in, money that's measured in the thousands, and sometimes ten-thousands).

Of course, the technical requirements for getting hit songs into your game are easy. Just find an MP3, and you're good to go. However, if you redistribute that MP3 without the permission of the people that created the song, you're breaking copyright law and could face fines of hundreds of thousands of dollars. Even if you don't plan on making money for your game, it's illegal for you to redistribute somebody else's song without their permission.

The legal way to get hit songs into your game is to contact the owner of that song and talk to them about what you plan on doing. Be prepared with specific facts about what you want—for example, do you want the whole song or just a snippet?

Recruiting Local Talent

Maybe you know a friend who's in a band or trying to form one, or maybe there's a local band whose sound you think would be a perfect fit for your game. If that's the case, consider striking up a deal with them! It'll free you from having to actually write and record the music, and it'll give them some much-needed exposure and something on which to focus their creative energy. Having someone other than yourself counting on the release of your game also helps when you've been working on a bug for days on end, and you feel like just pitching the whole project and starting something new.

There's no hard and fast rule for what the band should get in return for their contribution for the project. It could range from a six-pack of their favorite hopps-derived beverage, to thousands of dollars, a prominent "Music By" credit at the start of the game, and a cut of the game's profits. You'll have to navigate the negotiation on your own; just be realistic about your credentials, and realistic about the band's. For example, if you've never released a game before, promising the band a cut of the game's royalties isn't really worth much. For some bands, that might be enough; for others, they might want to see some cash up front as well.

Once you've recruited your band, your next step is to describe to them in detail what you're looking for. For every song you want, explain to them the emotional state you want to create, the environment, the length, whether the song will need loop points so your code can play it infinitely, and so on. Keep in mind, that in general, most game music doesn't have lyrics. It's more akin to music in film and television, which usually just sets a mood. However, there are some situations where music with words works well—follow your gut, and pay attention to how it's done in movies.

All in all, working with a friend who's in a band can be a rewarding experience for both of you. You'll draw from each other creatively (he or she may have some great ideas for your game!), and you'll both find that having someone else on the project makes it harder to abandon.


SOUND EFFECTS

Sure, you can play a WAV file in response to a user action, but what WAV file should you play? That's where this section comes in. Here, you'll learn some techniques for getting high-quality sound effects into your game.

Determining When Sound Effects Should Occur

One of the things I see most often in independent game development is games with what I like to call "lopsided sound effects." Lopsided sound effects occur when your game has all sorts of sound effects for things that really don't matter, but is missing key sound effects that do. For example, consider a fantasy action game with a sword-wielding hero and several angry imps. Believe it or not, I've seen some amateur games that have several different imp grunt, imp hurt, and imp die sound effects, but no sword swinging, player hurt, or sword hit sounds!

You can avoid the horror of lopsided sound effects by following some basic sound effects principles. For starters, realize that sound effects are first and foremost a way to communicate to the player that something has happened. They feed information back to the user aurally. Once they fulfill that role, they can work on their second goal, which is to create emotion and immersion in a game.

A good rule is to make sure there are sound effects for everything your player can do in the game. Another good rule is to say that a person standing where they can't see the monitor can tell whether the player is actually doing things in the game.

Sound Effects Collections

So, you've determined what sound effects you need in your game and when they should occur. You stare at your list, wondering how in the world you're going to create all of these sound effects.

That's when it hits you—you don't have to! Just as there are many CD-ROMs full of clip art for use in printed publications and Web sites, there are tons of "sound effect clip-art collections" you can buy and use, royalty free, in your own applications. Most of these collections provide highquality stereo WAVs that you can downsample and convert to mono for use in your game.

You'll find that the cost of these collections varies wildly. Professional collections, designed for professional movies or broadcasts, can get fairly expensive; however, some sound effects collections can be picked up for the cost of a video game (or even the cost of a game from the bargain bin!).

As an added bonus, some of these sound effect collections also include royalty-free snippets of music, perfect for a game-over screen or options screen where all you need is a quick little tune.

Included in the links on your CD are several links to companies that sell sound effects collections.

Making Your Own Sound Effects

You've looked at the sound effects collections and just can't seem to find the right sound effect for that squeaky gate or snoring goblin. Here are some tips for recording your own sound effects.

Minimize background noise. This is a given, but if you can, move your microphone away from the computer and its noisy fans, or consider using a portable recorder. Also, try to pick a room with good acoustics. Sometimes bathrooms work well because their tiled walls bounce sound waves better.

Remember that you can manipulate your sound after you record it. There's no rule that says you can't lower the pitch of someone snoring to create a good ogre snore effect! Play around with the manipulation features of your sound software—try reversing the sound, changing its pitch, applying a flange or filter, and so on.

Think backwards. Don't say, "Okay, I need a squeaky gate sound effect, so I'm going to go find an object that sounds like a squeaky gate." Instead, find some random object, close your eyes, and think "does this sound like a squeaky gate, or maybe could be made into a squeaky gate?" Your ingame visual connotations will do a lot to create the sense of realism of a sound effect. Sure, it might be a cat meowing in real life, but when your game plays it in response to your player opening the squeaky gate, it'll sound a lot more like a squeaky gate. As long as the sound effect isn't immediately recognizable on its own, you'll be fine.

Solicit the help and input of others. Going on "sound effects hunts" can sometimes be a lot of fun—the guise of "finding sound effects" can be a great excuse for you to do fun stuff you wouldn't ordinarily do. Sometimes the only way to get a good "walking through snow" effect is to go up to the mountains and record someone walking through snow. Enlist some friends and go exploring!

Downloading Sound Effects from the Web

In a word: don't. Downloading sound effects may seem like an easy way to do things, but it's also very risky. The vast majority of WAV files you can get from the Web for free have been sampled from movies, television shows, or other video games. This means that using them is just as illegal as using a hit song without paying for it. Even if you modify the sound superficially, you still don't have a legal right to redistribute that sound without permission from its creator.

Also, there's no way to tell whether a sound effect that came from the Web is actually free. Sure, the Web site says it's free, but if it sounds suspiciously like Homer's "D'oh!" from The Simpsons, then it probably is. The last thing you want after you finally release your game is a letter from the FOX Legal Team wanting to know why their character's voice is in your game!


SPEECH

Speech is a big part of the audio in today's games. In the beginning, games could get away with bad voice acting because any voice acting was considered cutting edge, and because it was too new to be an integral part of the player's gaming experience. However, those days are now gone, and today, having bad voice acting is no better an option than having no voice acting.

This section will take a look at some ways to get high-quality speech integrated into your game.

Hiring Voice Actors and Actresses

There are many professional, as well as aspiring, voice actors and actresses out there ready and willing to provide voices for your game's characters. Usually these talents are relatively affordable—a few hundred dollars—though, like everything else, you can pay a lot of money for a highprofile actor or actress.

Working with voice actors and actresses is technically very easy, because virtually all of them have their own professional recording studios and enough technical know-how to send sound files back and forth through e-mail. Most of the time, you just send your actor or actress a script containing the exact lines you'd like him or her to say. Explaining the emotional state of the character at that point can also help (if it isn't obvious just by reading the script).

If you don't want to pay for contracting voice actors or actresses, consider enlisting the help of a few friends. Talk to some people you know, give them their scripts, set up your recording equipment, and invite everyone over to speak their lines, hang out, have dinner, or whatever. Many people really enjoy voice acting, especially when they know that their likeness isn't going to go along with their voice. Also, try it yourself! Once you get past the weirdness of hearing your own voice recorded, it's actually quite fun to vocalize a range of expressions and try to add life to your characters.

Localization Issues

One of the most difficult things to deal with when it comes to in-game speech is the issue of localization. If you want your game to be easily accessible to non-English speaking players, you will need to deal with the issue of localization.

Localization can quickly increase the number of lines for your voice actors. It may also require that you hire voice actors that can speak different languages.

Think carefully up-front about what languages your game will be in, and whether those languages will have their own set of voice files. If dialog isn't that critical to the gameplay in your game, you might be able to keep English voice files coming out the speakers, but have text captions in the correct language displayed on screen.

Lip-Syncing

One of the most difficult aspects of in-game speech is syncing the motion of the game characters' mouths to the audio that's coming out of the speakers. Even today, most games don't do this well enough for their players to actually believe that the characters are talking. However, technology is progressing, and there are already some programs on the market specifically designed to deal with the problem of lip-syncing audio to characters in games or computer renderings.

To properly lip-sync, you need to break down your source sound into distinct phonemes. Phonemes are to speech what bits are to data; in other words, phonemes are the atomic units from which all speech is created. Each phoneme can be mapped to a specific shape of a character's mouth, allowing you to easily animate the character once you have broken down the source audio file into a set of phonemes. There are programs that help you do this; I've included links to some of them on your CD.


CONCLUSION

I hope that you've found this short section of tips and tricks useful. Creating good game audio, like everything else, gets easier the more you practice it, so my final tip is simply "don't give up!" A little bit of patience and a good critical ear can go a long way toward helping you create the best possible music for your game.


Index

A

ACid, 31
ActiveX, ModPlug supporting, 17
actors, hiring, 334
ADC (analog-to-digital converter), 2-3
ADPCM compression format, 7
AIFF files, DirectShow playing, 124
aliasing, 289
Amiga, MOD music format, 8-9
articulation in DirectMusic Producer, 307
AssembleErrorInfoString, 200-201
asynchronous command execution, 153-154
attack, decay, sustain, and release (ADSR) of instrument, 167
in DirectMusic Producer, 307-309
attenuation, 93
AU files, DirectShow playing, 124
audio compression algorithms, 6
audio engines, 16
audio scripting support to, 196-205
BASS, 16
CError object, 52-53
CNotificationHandler class, 113-114
concurrent sounds, playing, 83-85
CWAVEFile, 70-73
error handling, 47-51
exceptions in, 51-54
I3DL2, integrating, 253
list of, 16-18
MIDI files with, 102-103
MikMod, using, 160-162
MP3 support, adding, 127-128
multithreading and, 115-116
notifications into, 112-114
Ogg Vorbis, integrating, 135-137
OpenAL, integration of, 240-247
sound effects, adding, 209-210
audio format chunk, 68-69
audio manager, 55. See also CAudioManager
with MIDI files, 104-105
audio path editor, 206
audio paths. See also DirectMusic default audio path, 88
independent audio paths, 89
performance channels and, 168
Play method for, 90-91
pool of, 88
querying volume of, 92
segment specific volume, 91-93
setting the volume, 92
shared audio path, 89
for volume, 88-92
audio scripting, 188-205. See also CAudioScript class
Add Track(s), 192
audio engines, adding support to, 196-205
authorizing simple script, 194-195
basics of, 190-191
developer's and, 196-205
initializing scripts, 199-201
interface and, 190
loading, 199-201
Reference Runtime folder, 194
routines
calling, 204
enumerating, 201-202
segments, creating, 192-194
sound designers and, 191-195
string values, variables as, 203
subroutines
calling, 204
enumerating, 201-202
naming, 195
syntax, editing for, 195
Time Signature Track, 192
variables
enumerating, 201-202
getting values for, 203
setting values for, 202-203
Wave track
adding, 192
properties, 193
audio tags for MP3, 119, 122
audio visualization, 283-296. See also Discrete Fourier Transform (DFT); spectrum analyzers
AudioVB
scripts, 191
syntax in, 195
auto-resetting events, 115


Index_B

Team LiB

Index

B

bands, 168
choosing instruments from, 172
in DirectMusic, 171-172, 174, 179-180
in DirectMusic Producer, 309-311
basilar membrane, 286
beat
latency and, 110
MIDI, detection with, 109-112
notifications for, 110-114
Beethoven's Ninth Symphony, 5
BeOS, MikMod on, 17
bits per sample. See sample quality
block align field for WAV files, 69
Break embellishment, 171
Bryce, 330
buffers. See also OpenAL; PCMs; 3D sound
DirectSound buffers, 23-24
byte system, 2
in WAV files, 69


Index_C

Team LiB

Index

C

C/C++. See also exception handling
BASS supporting, 16
FMOD supporting, 17
ModPlug supporting, 17
Cakewalk Pro Audio, 101
callback function
in CNetConnectWizard, 275-276
in DirectPlay, 272-273
in DirectPlay Voice, 277-278
callouts, 78
CAudioManager, 128
building, 55-59
class declaration for, 55
Init method for, 56-57
LoadSound function for, 58-59
for MikMod support, 161-162
for 3D sound, 227-228
two-phase creation, 56
UnInit function, 58
CAudioScript class, 196
AssembleErrorInfoString, 200-201
creating, 198-199
DMUS_SCRIPT_ERRORINFO object, 200-201
Init function with, 198-199
m_Script, 198-199
pointer, developing, 196-197
CC (Control Curve) track, 168
CCDDeviceInfo class, 141
CCDPlayer class, 140-141
GetAvailableDevice method, 143-145
CD audio, 13-14, 140-141
CCDPlayer::GetAvailableDevice method, 143-145
change detection, 151-152
current play position, determining, 149-150
drive letters for peripherals, 143-145
ejecting CDs, 150-151
MCI_NOTIFY flag, 153
number of tracks, determining, 152
pausing play, 148-149
Play method for, 142-143
playing tracks, 147
resuming play, 148-149
status commands, 151-152
stopping, 147-148
synchronous vs. asynchronous command execution, 153-154
volume, controlling, 153
Windows MCI for, 141-143
CD IDs, 145
CD player
initializing, 154
sample program, 153-155
CD-R/CD-RW, 14, 143
CDDB service, 155
CDirectMusicSegment class, 103
CDirectShowMusic class, 127
CDiscreteFourierTransform class, 291-292
CDs. See also CD audio
links to, 298
sampled at 44100 Hertz, 4
74 minutes on, 5
universal product code (UPC) for, 145
CDynamicMusic class, 180-183
CEchoEffect class, 211
CEffect class, 211
CError object, 52-53
channels
MIDI channels, 101
number of, 34
in WAV files, 69
chords. See DirectMusic Ch2p1_ToneGenerator workspace, 28
chunks
data chunk, 70
main chunk, 68
in RIFF files, 66-68
class IDs, 57
cleaning up code, 40
CLSID, 57
CMIDIMusic class, 103
pointer, 105-106
for tempo, 106-107
CMixer class, 97-98
CMP3AudioTag class, 127
CNetConnectWizard, 274-277
creating, 275
default values, setting, 276-277
displaying, 276-277
DoConnectWizard, 276-277
initializing, 275
receive callback function, 275-276
shutting down, 277
CNetVoice, 280-281
CNotificationHandler class, 113-114
CoCreateInstance function, 57
codecs, 7
Vorbis codec, 131
columns in tracked music, 177
compiler for DirectX Audio, 26-29
compressed, sampled music, 15
compressed files, DirectShow and, 124-128
concurrent sounds
audio engines playing, 83-85
playing, 82-83
configuration sets in DevStudio, 26
Cooley and Turkey, 295
cooperative level, setting, 31
COpenALBuffer instance, 245
COpenALManager, 242-243
COpenALSource class, 246-247
cosine waves, 39
resonance and, 287-289
CPlayer object for DirectPlay Voice, 281
CreateSecondaryBuffer function, 34
CreateSoundBuffer
for primary buffer, 32
for secondary buffer, 33
Creative Labs, 236
Web site, 261
CSound object, 60
adjustments to, 85-86
CAudioManager and, 59
Play method, 60
adjustments, 86
PlaySegmentEx with, 83
SetVolume method, 92
virtual methods, 103
CSoundEffect class, 103
CSoundInstance class, 83-85
AddEffect method, 212
creating, 84-85
effect parameters, applying, 215
enhancements to, 212-214
Init method for, 84-85
RemoveAllEffects method, 213-214
sound effects, adding, 209-210
UnInit method for, 84-85
CSoundPtr class, 196
C3DSoundEffect class, 228-229
C3DSoundListener class, 225-229
CTrackedMusic, 160-161
CTrackMinSecFrame, 141
CTrackMinSecFrame class, 146
CWAVFile, 70-73
loading
sounds from file, 75
WAV files into file, 72-73
overloaded assignment operator, 71
saving, 73
cycle time, 189


Index_D

Team LiB

Index

D

data chunk, 70
decibels, 253
Deep Space sample program, 229-233
loading sounds in, 231
in OpenAL, 248
updating buffers and listener in, 232-233
deleting primary buffer, 32
Delphi
BASS supporting, 16
ModPlug supporting, 17
DevStudio, 26
callouts, 78
custom resources, storing, 78
with Ogg Vorbis libraries, 133
resource files, creating, 76-77
direct path, 251-252
DirectGraphics, 45
DirectMusic, 8, 13, 22. See also DirectMusic Producer; MIDI keyboard
alternative patterns, creating, 326-327
bands in, 168, 171-172, 174, 179-180
beat detection, 110-112
CC (Control Curve) track, 168
chords, 322-326
chord paths, 326
segments, inputting, 323
setting up, 324-326
Chords for Composition track, 325-326
complexity of, 44-45
control segments, 183
creating segments, 46
defined, 23
DLS collections, 179
embellishments, 170-171
creating, 319-320
General MIDI instrument set, 179
grooves and groove levels, 177-179
changing groove levels, 178-179, 184-185
performance groove modifier, 184-185
range in, 174-175
IsSecondary flag, 195
Loader, 45-46
motifs, 170
adding, 321-322
beat boundary, starting on, 324
multiple parts for, 323
playing, 185-188
multithreading, 114-115
notes in, 167-168, 176-177
overview of interfaces, 45
parts, 177
defined, 168
patterns, 169-170, 177
alternative patterns, creating, 326-327
performance channels, 168-169
performance groove modifier, 184-185
Performance object, 46-47
playing
motifs, 185-188
segments, 183-184
runtime format, saving in, 317, 327
segments, 173-174
for chords, 323
control segments, 183
creating, 46
playing, 183-184
style tracks, 168, 173-174
tempo tracks, 174
tracked music compared, 175-176
tracks, 168
use of term, 15
Var Switch Point track, 168
variations in patterns, 170, 318
volume controls, 44
DirectMusic Producer, 101
additional parts, adding, 314-315
alternative patterns, creating, 326-327
articulation, setting up, 307
attack, decay, sustain, and release (ADSR) of instrument in, 307-309
AudioVP scripts with, 191
band, setting up, 309-311
band editor, 309-310
chord track, 311
creating songs with, 302-303
DLS collection, creating, 305-309
harmony, setting up for, 310
icons in, 303
installing, 300-302
Instrument heading controls, 306-307
loading WAV files, 305-306
looping segments, 317
LSB numbers, 307
melody, setting up for, 310
metronome toolbar button, 313
MSB numbers, 307
Note Overlap Allowed checkbox, 308
note range edit boxes, 306
objects
creating, 303
list of, 304-305
Pattern Editor, using, 311-315
patterns
alternative patterns, creating, 326-327
creating, 311-315
finishing input, 315-316
notes, entering, 312-313
quantizing notes, 313-314
stringing patterns inside segment, 316-317
PChannel 10, 310
quantizing notes, 313-314
runtime format, saving as, 317, 327
segments
looping, 317
stringing patterns inside, 316-317
sound effects with, 205-207
variations, adding, 318-319
volume, setting the, 311
zooming hotkeys in, 312
directories for audio data, 78-79
DirectPlay. See also DirectPlay Voice
callback function in, 272-273
common messages in, 273
connecting to game, 271-272
description of, 269-274
disconnecting from game, 274
GUIDs in, 270-271
interface, creating, 270
lobbies in, 275
peer-to-peer topology, 271
receiving messages, 272-273
sending messages, 273-274
service provider, selecting, 270-271
session, flow of events in, 269-270
starting a game, 271-272
threads in, 274
DirectPlay Voice, 263-282
basic concept of, 264
calibrating mic and speakers, 278
callback function in, 277-278
CheckAudioSetup method, 278
CNetVoice, 280-281
common voice messages, 278-279
CPlayer object, 281
Forwarding Server Topology, 265-268
full-duplex sound cards for, 280
half-duplex sound cards, 280
initializing, 278
Mixing Server Topology with, 265, 268, 281
network topologies, 264-269
in peer-to-peer communications, 264-265
programming, 277-281
reference counted player objects, 281
sound cards for, 280
3D sound buffers, voices as, 281-282
triggers in, 264
DirectShow
filters in, 125
interfaces, 125
messages from, 126-127
MP3 files, playing, 124-128
streaming MP3s with, 125-126
DirectSound, 22
buffer, 209
defined, 23
EAX for, 261
initializing, 30-32
playing sounds with, 61-62
primary buffers, 23-24
secondary buffers, 23-24
streaming wave files in, 44
DirectX 3 interface, BASS supporting, 16
DirectX AppWizard, 29
DirectX Audio. See also 3D sound; ToneGenerator
basics of, 22-26
compiler, setting up, 26-29
configuration paths for, 26-27
definitions of terms, 23
DLS1 with, 13
playing waves, sample programs for, 62
streaming, 25-26
Discrete Fourier Transform (DFT), 285. See also spectrum analyzers
arrays, constructing, 292-293
calculating, 293-294
CDiscreteFourierTransform class, 291-292
graph, rendering, 294-295
Nyquist number, 289-290
distance factor, 223
DLLs, 57
Ogg Vorbis and, 131
DLS1, 11-13, 308
DLS2, 13, 308
DLS collections, 166-167
bands, 171
with DirectMusic Producer, 305-309
sample lists as, 179
DMUS_SCRIPT_ERRORINFO object, 200-201
DoConnectWizard, 276-277
Dolby
Digital encoding, 261
Web site, 261
Doppler, Christian, 219
Doppler effect, 219-220, 223
in Deep Space sample program, 230
Download method, 60
downloading sounds, 60
automatic downloading, 61
sound effects, 333-334
drive letters for CD peripherals, 143-145
DSBLOCK_FROMWRITECURSOR flag, 37
DSBPLAY_LOOPING flag, 40
DSBUFFERDESC structure, 34
DSFXI3DL2Reverb structure, 253-255
DS3Listener, 226
duration of note, 167
dwBufferBytes, 34
DWORDs, 146-147
DXGetErrorString8 function, 34
dynamic music, 165-186. See also DirectMusic; DirectMusic Producer
DLS collections, 166-167
elements of, 166
notes in, 167-168
playing, 183-184
programming, 180-183
segments, creating, 46
tutorial for creating, 299-327
dynamic_cast keyword, 106


Index_E

Team LiB

Index

E

early reflections, 251-252
EAX, 260-261
BASS supporting, 16
elements of, 260-261
FMOD supporting, 17
echoes
effect class, 210-215
I3DL2, setting persistence in, 254
Effect Properties dialog, 207
effects. See sound effects
ejecting CDs, 150-151
embellishments. See DirectMusic
End embellishment, 171
end time of note, 167
environment editor. See I3DL2
environmental modeling, 249-261
equalizers, 95
error handling
in audio engine, 51-54
audio engines, 47-51
in CAudioScript class, 200-201
in CD player, 155
DXGetErrorString8 function, 34
message box for, 54
in OpenAL, 244-245
exception handling, 47
in audio engine, 51-54
catching errors, rule for, 48
rules for, 48-51
timing for, 49-51
unexpected events, catching, 48-49
extensions in DirectMusic Producer, 303


Index_F

Team LiB

Index

F

FAILED macro, 34
Fast Fourier Transforms (FFTs), 295-296
52nd Street (Joel), 14
Fill embellishment, 171
filters in DirectShow, 125
flags
DSBLOCK_FROMWRITECURSOR flag, 37
DSBPLAY_LOOPING flag, 40
IsSecondary flag, 195
MCI_NOTIFY flag, 153
m_Paused flag, 148-149
WAVE_FORMAT_PCM flag, 34
FM synthesis, 8-9, 158
obsolescence of, 15
FMOD, 17
format chunk, 68-70
Forwarding Server Topology, 265-268
FOURCC (four-character code), 66-68 44100 Hertz, 4
frames in MP3, 119
Fraunhofer IIS-A, 6
frequency modulation synthesis. See FM synthesis
full-duplex sound cards for DirectPlay Voice, 280
function pointers, 112-113
Future Crew, 10


Index_G

Team LiB

Index

G

Game Programming Gems 1, 53
GameCube, FMOD on, 17
General MIDI instrument set, 167, 179
General MIDI Level 1 Sound Set (GM sound set), 11
categories for, 12
genre codes for MP3, 120-121
GetConsoleWindowHandle function, 30
GetGuid method, 211
GetNotificationPMsg, 110-111
GetObject method, 74-75
global configuration, 26-27
global volume, 93
Groove Level Properties dialog, 179
grooves and groove levels. See DirectMusic
GUIDs
in DirectPlay, 270-271
GetGuid method, 211
for I3DL2 presets, 256
m_Effects vector and, 214
for notifications, 110-111
for sound effects, 207-208


Index_H

Team LiB

Index

H

half-duplex sound cards for DirectPlay Voice, 280
Harvey, David, 59
help
MikMod help, 160
Ogg Vorbis documentation, 132
Hertz (Hz), 3
sampled at 44100 Hertz, 4
history of game audio, 8-11
hit songs, including, 330-331


Index_I

Team LiB

Index

I

icons in DirectMusic Producer, 303
IDirectPlay8LobbiedApplication, 275-276
IDirectPlay8Peer, 275
IDirectPlayVoiceClient, 278
IDirectPlayVoiceServer, 278
IDirectSound8, 33
IGraphBuilder interface, 125-126
IMediaControl interface, 125-126
IMediaEvent interface, 125-126
Impulse Tracker screenshot, 158
Init method
for CAudioManager, 56-57
for CAudioScript class, 198-199
for CNetConnectWizard, 275
for CSoundInstance, 84-85
in OpenAL, 243-244
InitAudio method, 57
initializing. See also Init method; UnInit method
audio scripts, 199-201
CNetConnectWizard, 275
DirectPlay Voice, 278
DirectSound, 30-32
installing
DirectMusic Producer, 300-302
MikMod, 159-160
Ogg Vorbis libraries, 132-133
OpenAL, 241
instruments
DLS collections, 166-167
General MIDI instrument set, 167
Interactive Audio Special Interest Group, 250-251
interaural intensity difference, 218
interaural time difference, 218
Internet
sound effects, downloading, 333-334
WAV files downloaded from, 73
Intro embellishment, 171
Inversion Group overrides, 168
ISA bus, 24
IsPlaying method, 61
for CSoundInstance, 85
IsSecondary flag, 195
IT, BASS supporting, 16
I3DL2, 250-256
audio engines, integration into, 253
description of, 250-251
direct path, 251-252
DSFXI3DL2Reverb structure, 253-255
early reflections, 251-252
as effect, 252-253
enhancing program, 258-259
environment editor
creating, 256-260
loading environments, 258-259
saving environments, 258-259
trackbars in, 256-258
late reverberation, 251-252, 255
loading environments, 258-259
obstruction, interface for, 260
occlusion, interface for, 260
preset effects in, 256
reverb parameters, 253-255
saving environments, 258-259
sound wave types, 251-252
terms in, 251-252
vocabulary of, 251-252


Index_J

Team LiB

Index

J

Java, MikMod on, 17
Joel, Billy, 14


Index_K

Team LiB

Index

K

Kai's Power Tools, 330
keyboard for dynamic music. See MIDI keyboards
KiloHertz (KHz), 3


Index_L

Team LiB

Index

L

languages. See also C/C++
BASS supporting, 16
LANS, DirectPlay Voice on, 265, 269
late reverberation, 251-252, 255
libraries. See also OpenAL
into audio engines, 73
MCI library, 141
MikMod library, using, 160
Ogg Vorbis libraries, 130-132
winmm.lib library with PlaySound, 62
licensing issues with MP3, 15
Lim, Jeffrey, 158
Linux, Ogg Vorbis on, 130
lip synching, 335
listeners. See OpenAL; 3D sound Loader, 45-46
directories for, 78-79
MIDI files, loading, 103
LoadGameResources
exception handling in, 49-51
loading. See also CWAVFile; WAV files
audio scripts, 199-201
DirectMusic Producer, WAV files in, 305-306
with I3DL2 environment editor, 258-259
MIDI files, 103-106
OpenAL, WAV files in, 245-246
spectrum analyzers loading WAV files, 290-291
LoadMIDI routine, 104
LoadMotifs method, 181
LoadOggVorbis method, 135-136
LoadScript method, 199-200
LoadSegmentFromDisk function, 104
LoadSegmentFromMemory function, 104
LoadSegmentFromResource function, 104
LoadSound function
for CAudioManager, 58-59
memory, loading WAV files from, 74
for MIDI files, 104-105
LoadSoundFile, exception handling in, 49-50
LoadWaveFile, exception handling in, 49-50
lobbies in DirectPlay, 275
local musicians, recruiting, 331
Lock method, 35-37
locking buffers, 35-36
Loki Entertainment Software, 236
looping sounds, 86-87
DirectMusic Producer segments, 317
patterns, 170
SetLoopPoints method, 87
SetRepeats method for, 87
sound instances and, 83-84
lopsided sound effects, 332
lossy compression algorithm, 6
loudness, perception of, 218
LPDIRECTSOUNDBUFFER, 35


Index_M

Team LiB

Index

M

MacOS, MikMod on, 17
macros in Windows MCI, 146-147
main function, exception handling in, 49, 51
MASM, BASS supporting, 16
master tempo, 108-109
MCI. See Windows MCI
MCI library, 141
MCI_MAKE_TIMSF, 146-147
MCI_NOTIFY flag, 153
mciSendCommand API call, 141-142
m_Effects vector, 214
memory and WAV files, 73-75
messages box for error handling, 54
meters, 95
metronome toolbar button, DirectMusic Producer, 313
microphones for stereo sound, 5
MIDI, 10-11
audio engines, MIDI files with, 102-103
audio manager with, 104-105
beat detection with, 109-112
channels, 101
description of, 100-102
DirectShow playing, 124
General MIDI instrument set, 167
loading files, 103-106
low-level MIDI input/output routines, 116
Note On message, 100-101
playing files, 103-106
RTTI (run-timetype information), 105-106
tempo with, 103, 106-109
Windows MCI playing, 115-116
MIDI keyboards, 300-302
troubleshooting for, 302
Mikkers, Jean-Paul, 17
MikMod, 17
in audio engine, 160-162
CAudioManager changes for, 161-162
CTrackedMusic, 160-161
help, 160
installing, 159-160
library, 160
shutting down, 161
for tracked music, 159-162
Miles, John, 17
Miles Sound System, 17-18
millibels (mBs), defined, 253
Miod Vallat, 17
mixer volumes, 94-98
architecture of API, 95-96
changing values, 96-97
CMixer class, 97-98
freeware mixer classes, 98
Mixing Server Topology, 265, 268, 281
mixing sound, 7-8
mixing stations, 5
MMA (MIDI Manufacturer's Association), 13
mmioFOURCC function, 73
MOD music format, 8-9
BASS supporting, 16
ModPlug, 17
monaural sound (mono), 5
motifs. See DirectMusic
MP3
audio engine, adding support to, 127-128
audio tags, 119, 122
BASS supporting, 16
description of, 118
DirectShow, playing with, 124-128
file format, 119-122
frames in, 119
genre codes, 120-121
hit songs, including, 330-331
licensing issues, 15
size of, 118
use of term, 15
WAV files into, 6-7, 123-124
m_Paused flag, 148-149
MPEG Layer 3 algorithm (MP3). See MP3
m_Props structure, 226
m_Script, 198-199
MSDN, 62
muffling, 218
multithreading, 114-115
in DirectPlay, 274
music
hit songs, including, 330-331
local musicians, recruiting, 331
software for, 330
muting controls in CMixer class, 97-98


Index_N

Team LiB

Index

N

NoiseTracker, 10
notes
in DirectMusic, 167-168
in dynamic music, 167-168
properties of, 167
tracked music rows and, 176-177
velocities of, 177
notifications
in audio engine, 112-114
for beat detection, 110-114
GUIDs for, 110-111
Nyquist number, 289-290


Index_O

Team LiB

Index

O

objects in DirectMusic Producer. See DirectMusic Producer
obstruction
EAX and, 260
FMOD supporting, 17
I3DL2 and, 260
occlusion
EAX and, 260
FMOD supporting, 17
I3DL2 and, 260
Ogg library, 131
Ogg Vorbis, 7, 15, 129-137
into audio engine, 135-137
BASS supporting, 16
closing files, 135
folders for, 132
help documentation, 132
installing libraries, 132-133
libraries, 130-132
opening files, 134
ov_read function, 134-135
structure of, 131-132
using Vorbis, 133-137
Vorbisenc library, 131
vorbisfile API, using, 134-135
Vorbisfile library, 132
Oktalyzer, 10
open-source software, 130
OpenAL
audio engine, integration into, 240-247
buffers, 236-237
creating and destroying, 245-246
properties, 238
context functions, 242
COpenALBuffer instance, 245
COpenALManager, 242-243
COpenALSource class, 246-247
core OpenAL functions, 242
Deep Space sample program in, 248
EAX for, 261
error detection in, 244-245
initializing/uninitializing, 243-244
installing, 241
left-handed coordinate system, 240
listeners, 236-237
properties, 241-242
loading WAV files, 245-246
naming conventions, 237-238
new audio engine classes, 242-243
playing sources, 247
redistributables, inclusion of, 241
sources, 236-237
creating and destroying, 246-247
playing sources, 247
properties, 238-240
structure of library, 242
utilities functions, 242
order lists
in DirectMusic, 177-178
in tracked music, 175


Index_P-Q

Team LiB

Index

P-Q

panning
controls, 95
preserving, 97
parts in DirectMusic, 168, 177
Pattern Editor, DirectMusic Producer, 311-315
patterns. See also DirectMusic; DirectMusic Producer
in DirectMusic, 177
grooves and groove levels, 174-175
in tracked music, 175, 177
Pause method, 148-149
pausing CD players, 148-149
pbMemData byte pointer, 74
PCI bus, 24
PCMs
audio format chunk, 68-69
MP3 files and, 124
Ogg Vorbis files into, 134-137
WAVE_FORMAT_PCM flag, 34
per-project configurations, 26
changing, 27
performance channels
audio paths and, 168
in DirectMusic, 168-169
sharing of, 168
performance events
flowchart for, 112
notifications, 110-112
Performance object, 46-47
Philips, CD prototype by, 14
phonemes, 335
Photoshop, 330
pitch of note, 167
Play method, 39-40. See also CSound object
for audio paths, 90-91
for CD player, 142-143
playing, 4. See also CD audio; DirectMusic
CD audio tracks, 147
concurrently playing sounds, 82-83
MIDI files, 103-106
OpenAL sources, 247
secondary buffers, 39
playmodes, 168
PlaySegment call, 60
IsPlaying method with, 61
PlaySegmentEx, 82-83
PlaySound, winmm.lib library with, 62
PMsg
for beat, 110
for tempo, 107
pointers
CAudioScript pointer, 196-197
CMIDIMusic class, 105-106
CTrackedMusic pointers, 161
function pointers, 112-113
pbMemData byte pointer, 74
smart pointers, 59
polymorphism, 104
pos, 38
primary buffers
deleting, 32
in DirectSound, 23-24
setting, 31-32
programming
cycle time in, 189
DirectPlay Voice, 277-281
dynamic music, 180-183
3D sound, 225-229
ProTracker, 10
PS2, FMOD on, 17
pulse code modulation (PCM). See PCMs


Index_R

Team LiB

Index

R

Rabin, Steve, 53
RAD Game Tools, 17
Redbook audio, 13-15
RefCountPtr.h, 59
reference high frequency, 254
resonance, 285
hearing and, 285-286
mathematical expression of, 286-289
sine waves and, 286-289
resources
creating resource files, 76-77
loading WAV files from, 75-78
management of, 59
restoring buffers, 39
Resume method, 148-149
resuming CD play, 148-149
reverberation. See I3DL2
EAX and, 260
RIFFs, 66
chunks, 66-68
format chunk, 68-70
main chunk, 68
MP3 files and, 124
RLE (run-length encoding), 6
rolloff factor, 222-223
in I3DL2, 254
room attenuation values, 254
routines. See audio scripting
rows
in DirectMusic, 176-177
in tracked music, 175
RTTI (run-timetype information), 105-106
runtime format
DirectMusic Producer, saving in, 317, 327
sound effects at, 207-216


Index_S

Team LiB

Index

S

SAFE_RELEASE, 32, 40
sample, defined, 3
sample lists
in DirectMusic, 179
in tracked music, 176
sample parameters
in DirectMusic, 179-180
in tracked music, 176
sample quality, 2-5
defined, 4
math for, 34-35
for WAV files, 69
sample rates, 2-5
common audio sampling rates, 32
defined, 3
Nyquist number, 289
in WAV files, 69
samples
with DLS1, 13
mixing sound, 7-8
saving
CWAVFile methods, 73
I3DL2 environments, 258-259
sawtooth waves, 29
Schmidt, Brian, 24
ScreamTracker, 10
BASS supporting, 16
screenshot, 158
scripting. See audio scripting
search directory, using, 78-79
secondary buffers creating, 33-35
in DirectSound, 23-24
filling, 35-36
locking, 35-36
playing, 39-40
releasing, 40
restoring, 39
unlocking, 35-36
wrapping, 36-37
segments. See also DirectMusic; Producer
audio scripting for, 192-194
concurrently playing sounds, 82-83
downloading, 60
IsPlaying method, 61
level of game and, 174
PlaySegment call, 60
transitions, 174
unloading sounds, 61
sequencer software, 101
service providers
CNetConnectWizard for, 274-277
for DirectPlay, 270-271
SetAllParameters method, 211, 215
for listener interface, 226
SetCooperativeLevel, 30
SetFormat method, 32
SetFX method, 207-209
m_Effects vector and, 214
SETI@Home screensaver, 295
SetLength method for looping sounds, 87
SetLoopPoints method, 87
SetRepeats method, 87
SetSearchDirectory method, 78-79
SetStartPoint method for looping sounds, 87
SetVolume method, 92
sine waves, 38
Nyquist number and, 289
and resonance, 286-289
skyboxes, 229
smart pointers, 59
SND files, DirectShow playing, 124
software for music, 330
Sony
CD player by, 14
Redbook audio, 14
sound cards, 3
for DirectPlay Voice, 280
"Sound Cards, Voice Management, and Driver Models" (Schmidt), 24
sound cones, 221-222
sound designers and audio scripting, 191-195
sound effects, 205. See also audio scripting
audio engine, adding to, 209-210
CMIDIMusic class for, 103
collections of, 332-333
concurrently playing sounds, 82-83
with DirectMusic Producer, 205-207
downloading from Web, 333-334
echo effect class, 210
effects classes, 211
I3DL2 and, 252-253
lopsided sound effects, 332
parameters for effects, applying, 215
at runtime, 207-216
speech, recording, 334-335
tips for recording, 333
use of, 216
sound instances, 83-85. See also CSoundInstance class
Sound Recorder playing MP3, 123-124
SourceForge, 155
sources. See also OpenAL
EAX and directivity of, 260-261
Special Effects Game Programming Using DirectX 8.0, 229
spectrum analyzers, 284-285
arrays, constructing, 292
calculating DFT, 293-294
CDiscreteFourierTransform class, 291-292
graph, rendering, 294-295
loading WAV files, 290-291
sample program for, 290-295
values, loading, 292-293
speech
actors, hiring, 334
lip synching, 335
localization issues, 334-335
recording, 334-335
square waves, 29
"Squeezing More Out of Assert" (Rabin), 53
Standard Buffer checkbox, 206
start time of note, 167
starting bytes, 36
StarTrekker, 10
stereo sound, 5
streaming, 25-26
in DirectSound, 44
DSBLOCK_FROMWRITECURSOR flag, 37
MP3s, 125-126
style tracks in DirectMusic, 168, 173-174
subroutines. See audio scripting
synchronous command execution, 153-154
syncing and latency, 110


Index_T

Team LiB

Index

T

Tammilehto, Sami, 158
task switching, 39
TCP/IP for DirectPlay, 270
tempo
in DirectMusic, 174
getting the, 108
master tempo, 108-109
MIDI and, 103, 106-109
performance tempo settings, 106-108
scaling factor for, 108-109
track tempo settings, 106-108
threads in DirectPlay, 274
3D Audio Working Group Web site, 260
3D sound, 217-233. See also I3DL2; OpenAL
BASS supporting, 16
buffers, 219
C3DSoundEffect class, 228-229
in Deep Space sample program, 232-233
head-relative mode, 224
minimum/maximum distances, 220-221
position of, 220
processing mode, 224
properties of, 220-224
sound cones, 221-222
CAudioManager changes for, 227-228
C3DSoundEffect class, 228-229
C3DSoundListener class, 225-229
Deep Space sample program, 229-233
deferred settings, 225
distance factor, 223
Doppler effect, 219-220, 223
DS3Listener, 226
EAX, 260-261
environmental modeling, 249-261
head-relative mode, 224
immediate settings, 225
listeners, 219
in Deep Space sample program, 232-233
distance factor, 223
Doppler effect, 223
orientation of, 220
position of, 220
properties of, 220-224
rolloff factor, 222-223
minimum/maximum distances for buffers, 220-221
m_Props structure, 226
orientation of listeners, 220
position of listeners and buffers, 220
principles of, 218
processing mode, 224
programming, 225-229
rolloff factor, 222-223
simulating sound waves with, 219
sound cones, 221-222
sound instances and, 83
ThrowCError define, 53-54
ThrowIfFailed define, 53-54
time formats in MCI, 146-147
TMSF format, 147
TMT Pascal, BASS supporting, 16
ToneGenerator
defined, 29-30
task switching, 39
trackbars
in environment editor, 256-258
functions of, 258
tracked music, 10-12, 157-162
columns, 177
description of, 158
DirectMusic compared, 175-176
MikMod for, 159-162
order list in, 175
patterns in, 175
rows in, 175
sample list in, 176
sample parameters in, 176
tracks in, 175
tracks, 101
CC (Control Curve) track, 168
in DirectMusic, 168
in tracked music, 175
Var Switch Point track, 168


Index_U

Team LiB

Index

U

UnInit method
of CAudioManager, 58
for CSoundInstance, 84-85
in OpenAL, 243-244
UNIX, MikMod on, 17
unloading sounds, 61
USB, 143


Index_V

Team LiB

Index

V

Var Switch Point track, 168
variables. See audio scripting
variations in DirectMusic patterns, 170, 318-319
vectors
of CTrackedMusic pointers, 161
for sound effects, 214
velocity of note, 167
Visual Basic
audio scripting with, 191
BASS supporting, 16
FMOD supporting, 17
ModPlug supporting, 17
Visual Studio version 7, 26
voice actors, hiring, 334
volume. See also audio paths; mixer volumes
attenuation, 93
audio paths for, 88-91
CD audio volume, 153
CMixer class, 97-98
in DirectMusic Producer, 311
global volume, 93
mixer volumes, 94-98
other volume knobs, 92
ranges for, 93
Vorbisenc library, 131
Vorbisfile library, 132
Vue d'Esprit, 330


Index_W

Team LiB

Index

W

WAV files. See also CWAVFile; DirectMusic Producer; OpenAL
audio format chunk, 68-69
BASS supporting, 16
bits per sample data, 69
block align field, 69
byte rate for, 69
channels, number of, 69
chunks, 66-68
codecs in, 7
common format IDs, 69
data chunk, 70
DirectShow playing, 124
format, 66-68
format chunk, 68-70
GetObject method and, 74-75
Loader, using, 45-46
main chunk, 68
memory, loading from, 73-75
into MP3, 6-7, 123-124
resources, loading from, 75-78
RIFFs, 66
sample rates, 69
size of, 118
sound effect collections, 332
spectrum analyzers, loading for, 290-291
wave height, 38
Wave Table synthesis, 9
Wave track. See audio scripting
Waveform Audio functions, 62
WAVEFORMATEX structure, 32
WAVE_FORMAT_PCM flag, 34
Winamp, 31
window caption, 30
Windows MCI. See also CD audio
for CD audio, 141-143
change detection, 151-152
DWORDs, 146-147
ejecting CDs, 150-151
flags, placing, 150
MCI_MAKE_TIMSF, 146-147
MCI_NOTIFY flag, 153
MIDI on, 115-116
pausing play, 148-149
playing sounds with, 62-63
resuming play, 148-149
status commands, 151-152
synchronous vs. asynchronous command
execution, 153-154
time formats, 146
winmm.lib library with PlaySound, 62
Win32 API, 62
WMA files, 15
BASS supporting, 16
DirectShow playing, 124
playing, 128
WM_DEVICECHANGE message, 152
world transform matrices, 229
wrapping the buffer, 36-37
WYSIWYG game editors, 189


Index_X-Y

Team LiB

Index

X-Y

X file meshes, 229
Xbox, FMOD on, 17
Xiphophorus (xiph.org), 130
XM, BASS supporting, 16


Index_Z

Team LiB

Index

Z

ZIP files
in Ogg Vorbis, 132
WAV files stored as, 73
zooming hotkeys in DirectMusic Producer, 312


List of Figures

Chapter 1: The Lay of the Land

Figure 1.1: How your computer recotds audio.
Figure 1.2: How your computer plays back audio.
Figure 1.3: The algorithmbehind the simplest form of RLE compression
Figure 1.4: Simple additive sound mixing
Figure 1.5: A comparison between FM synth and Wave Table synth.

Chapter 2: Introduction to DirectAudio

Figure 2.1: DirectX Audio mixes secondary buffers into the primary buffer, then out to your speakers.
Figure 2.2: Streaming a big sound file requires loading pieces of it just as they're needed.
Figure 2.3: The global include path settings.
Figure 2.4: A sine wave.
Figure 2.5: When you lock something, you might get two pointers back.
Figure 2.6: Use the fractional part of pos to determine where in the wave you are.

Chapter 3: Wave Audio Playback

Figure 3.1: Overview of the DirectMusic interfoces used for playing sound effects.
Figure 3.2: What happens when you fail to catch an exception.
Figure 3.3: Hypothetical call stack for a game, showing where an error might be thrown and caught.
Figure 3.4: The message box presented by our errorhandling code.

Chapter 4: Loading WAV Files

Figure 4.1: This figure illustrates the file format of a typical uncompressed WAV.
Figure 4.2: The FOURCC character code for RIFF—note the character swapping.
Figure 4.3: After inserting an RC file into your project, you get a new ResourceView tab.
Figure 4.4: Importing a WAV file into your application's resources.
Figure 4.5: Uncheck the external file checkbox to embed your WAV directly in the resource script of your program.

Chapter 5: Control Freak

Figure 5.1: DirectMusic uses several different parameters to control looping of a segment.
Figure 5.2: Sounds that share audio paths also share volume knobs.
Figure 5.3: Windows provides a GUI to the mixer volumes.
Figure 5.4: The mixer API architecture is divided into several different pseudo-objects.

Chapter 6: MIDI Playback

Figure 6.1: A professional MIDI setup might have several different pieces of hardware, all assigned to different MIDI channels.
Figure 6.2: The new class layout for the audio engine.
Figure 6.3: A flowchart of the basic process for using DirectMusic performance events.

Chapter 7: MP3 and WMA Playback

Figure 7.1: What's contained in the four bytes of a frame header.
Figure 7.2: Illustrates the format of the 128-byte long audio tag.
Figure 7.3: The Windows Recorder in all its glory.
Figure 7.4: The Convert Now dialog box.

Chapter 8: Ogg Vorbis Playback

Figure 8.1: Flowchart for using the vorbisfile API.

Chapter 9: CD Audio Playback

Figure 9.1: The process for using the Media Control Interface (MCI).
Figure 9.2: The effects of sending an MCI_SET_DOOR_OPEN command.
Figure 9.3: Synchronous versus asynchronous command execution.
Figure 9.4: Screenshot of this sample's chapter program in action.

Chapter 10: Tracked Music Playback

Figure 10.1: A screenshot of Impulse Tracker, a popular tracked music editor written by Jeffrey Lim.
Figure 10.2: A screenshot of ScreamTracker, another popular tracked music editor, written by Sami Tammilehto.

Chapter 11: Dynamic Music

Figure 11.1: Illustrates the relationship between parts, PChannels, and audio paths in DirectMusic.
Figure 11.2: You choose the instruments in your band from your DLS collections and the GM sound set.
Figure 11.3: The style and groove level tracks tell DirectMusic what pattern to play next.
Figure 11.4: A sequence of notes in a tracker and in sheet music form.
Figure 11.5: The Groove Level Properties dialog.

Chapter 12: Scripts and Effects

Figure 12.1: Clicking the narrow button on the left will enable/disable the entire variation set.
Figure 12.2: The wave properties window.
Figure 12.3: The wave track properties.
Figure 12.4: The final BigCat segment,with both variations active.
Figure 12.5: Adding references to an audio script.
Figure 12.6: The error dialog generated by AssembleErrorInfoString.
Figure 12.7: the audio path editor.
Figure 12.8: Ucheck the Use standard Buffer check - box to add custom effect to your audio path.
Figure 12.9: The Effect Properties dialog for the echo effect.

Chapter 13: 3D Sound Using DirectX Audio

Figure 13.1: The minimum and maximum distances tell DirectX Audio how far away a sound can be heard.
Figure 13.2: The various buffer properties that influence the volume of a sound.
Figure 13.3: In head-relative mode, buffer positions are relative to the listener.
Figure 13.4: A screenshot of this chapter's sample program.

Chapter 14: 3D Sound Using OpenAL

Figure 14.1: Multiple sources can reference the same buffer.
Figure 14.2: Typical flow when using OpenAL

Chapter 15: Advanced Topics in 3D Sound

Figure 15.1: Shows direct path, early reflections, and late reverb.
Figure 15.2: A graph of the three distinct wave types.
Figure 15.3: A screenshot of the Ch15p2_I3DL2EnvEdit or sample program.

Chapter 16: DirectPlay Voice

Figure 16.1: In a peer-to-peer environment with six players, every time you talk, your voice must be sent to the five other computers.
Figure 16.2: In a forwarding server topology, you send your voice data to a central server; the server takes care of sending it to everyone else.
Figure 16.3: If everybody talks at once, even with a voice server, you'll need 40 kpbs of download bandwidth to receive all 5 streams.
Figure 16.4: A mixing server sends each computer a mix of all the voices it needs.
Figure 16.5: CNetConnectWizard's connect dialog.
Figure 16.6: CNetConnectWizard's new game dialog.
Figure 16.7: The sound hardware test wizard.

Chapter 17: Audio Visualization

Figure 17.1: A screenshot of the ch17p1_visuals sample programe.
Figure 17.2: A cosine wave is a sine wave with a different phase.

Appendix A: Creating Dynamic Music

Figure A.1: Click on the toolbar button that looks like a MIDI port to adjust your MIDI configuration.
Figure A.1: A test project that you can use to see if your MIDI input is working.
Figure A.3: The controls to set an instrument's ID, and whether or not it's a drum set.
Figure A.4: Every instrument in a DLS1 collection has points for attack, sustain, decay, and release (ADSR).
Figure A.5: DLS level 2 instruments have delay, attack, hold, decay, sustain, and release points.
Figure A.6: The band editor of DirectMusic Producer.
Figure A.7: Eight-measure background piano part for Pattern1.
Figure A.8: The note properties window.
Figure A.9: The melody part for Pattern1.
Figure A.10: Background and melody parts for Pattern2.
Figure A.11: The part properties window.
Figure A.12: The intro embellishment pattern.
Figure A.13: The break embellishment pattern.
Figure A.14: A simple motif pattern.
Figure A.15: The secondary playback toolbar.
Figure A.16: The chord properties window.


List of Tables

Chapter 1: The Lay of the Land

Table 1.1: General MIDI Sound Set Categories.

Chapter 2: Introduction to DirectAudio

Table 2.1: Definitions of DirectX Audio Terms.
Table 2.2: Common Audio Sampling Rates.

Chapter 4: Loading WAV Files

Table 4.1: Common Wave Format IDs

Chapter 6: MIDI Playback

Table 6.1: GUIDs for Notifications

Chapter 7: MP3 and WMA Playback

Table 7.1: Genre Codes

Chapter 14: 3D Sound Using OpenAL

Table 14.1: OpenAL Buffer Properties
Table 14.2: OpenAL Source Properties
Table 14.3: OpenAL Listener Properties
Table 14.4: OpenAL Error Codes

Chapter 15: Advanced Topics in 3D Sound

Table 15.1: DSFXI3DL2Reverb Structure Members
Table 15.2: Common Trackbar Messages

Chapter 16: DirectPlay Voice

Table 16.1: Available DirectPlay Interfaces
Table 16.2: Common DirectPlay Messages
Table 16.3: Common DirectPlay Voice Messages

Appendix A: Creating Dynamic Music

Table A.1: The Objects of DirectMusic Producer
Table A.2: Zooming Hotkeys


List of Sidebars

Chapter 1: The Lay of the Land

Why 44100 Hertz?
The Story behind CDs
Redbook Audio: Squeezed Out?

Chapter 5: Control Freak

Why Such a Funny Volume Range?

Chapter 8: Ogg Vorbis Playback

Ogg Vorbis: Funny Name, Serious Codec

Chapter 12: Scripts and Effects

Sidebar: Cycle Time


CD Content

Team LiB
Previous Section

CD Content CD Content

Following are select files from this book's Companion CD-ROM. These files are for your personal use, are governed by the Books24x7 Membership Agreement, and are copyright protected by the publisher, author, and/or other third parties. Unauthorized use, reproduction, or distribution is strictly prohibited.

Click on the link(s) below to download the files to your computer:

File | Description | Size
All CD Content | Beginning Game Audio Programming | 20,607,114
Chapter 2: | Introduction to DirectAudio | 55,760
Chapter 3: | Wave Audio Playback | 143,837
Chapter 4: | Loading WAV Files | 638,564
Chapter 5: | Control Freak | 1,535,203
Chapter 6: | MIDI Playback | 125,792
Chapter 7: | MP3 and WMA Playback | 1,620,897
Chapter 8: | Ogg Vorbis Playback | 1,065,051
Chapter 9: | CD Audio Playback | 35,816
Chapter 10: | Tracked Music Playback | 494,090
Chapter 11: | Dynamic Music | 1,265,307
Chapter 12: | Scripts and Effects | 4,268,139
Chapter 13: | 3D Sound Using DirectX Audio | 1,295,100
Chapter 14: | 3D Sound Using OpenAL | 1,194,280
Chapter 15: | Advanced Topics in 3D Sound | 827,200
Chapter 16: | DirectPlay Voice | 66,759
Chapter 17: | Audio Visualization | 1,341,431
A - creating dynamic music |  | 3,333,541
A - creating dynamic music-dynamicmusic |  | 985,888
A - creating dynamic music-normalmusic |  | 217,138

Team LiB
Previous Section