David Yackley
Microsoft Corporation
December 1999
Summary: This white paper will give you a conceptual understanding of the DirectMusic API, how DirectMusic overcomes the restrictions of the General MIDI standard and provides a quality method to distribute music over the Internet, and how DirectMusic will help you make interactive music for the future. (6 printed pages)
The Microsoft® DirectX® suite of APIs is changing the idea that Microsoft Windows® is limited in multimedia capabilities. In fact, DirectX is making Windows the optimum gaming platform.
Until recently, the DirectX APIs were primarily focused on enhancing graphics, animation, and sound on the Windows platform. Now, however, development is growing in the field of music and how it is used in multimedia applications such as games and Internet applications. The DirectMusic® API is the result of Microsoft’s own development efforts.
DirectMusic opens up unexplored avenues in the musical landscape and could change our perception of music itself. Building on the foundation of MIDI, DirectMusic economically provides consistent performance, custom samples, variability, and interactivity to the musical palette of the Windows platform. This article takes a look at the implications of this groundbreaking technology.
Music plays a crucial role in our day-to-day emotional lives. At work or play, music and emotion are powerful and inseparable partners. But sound does not simply affect the emotions, it alters perception as well. For example, studies have shown that audio quality influences our perception of graphics quality. This means that poor audio quality can make your users believe that the graphics in your multimedia application are of low quality. Conversely, good quality audio will make your users think better of the graphics in your applications.
The cinema has long exploited the power of music to influence emotion and shape viewer perceptions. In horror films, for example, the use of music may not be realistic, but it can draw you into terrifying and unseen worlds. Music can convey time and place, and reveal a character’s motives, thoughts, and fears. Often, the combination of picture and sound can create a reality that is greater than the sum of its parts.
Advertising also makes use of the power of music to move and manipulate. A broadcast commercial does not necessarily convey what you need to know about a product; it tells you what you are supposed to feel about that product. With a few short notes, music can make a listener feel good about a product or company. With a few short notes, music can create a company image. Hum the tune, get the feeling, buy the product.
Musical trends have often followed trends in the visual arts: Ravel’s Impressionist soundscapes originated from the work of Monet, the visual dissonance of Kandinsky found musical expression in Berg. This trend has generally been consistent not only from an artistic standpoint, but from a technological one as well. Before “talkies” came into the picture, silent movies existed for some time (although they were rarely completely silent due to the efforts of piano players, organists, and theatre orchestras in movie houses). A quick sampling of the online world will confirm that this artistic and technological lag is just as pronounced today. Audio is often considered the rogue child of multimedia, the last one to be called to the table and the first to be excused.
Pop quiz: What does the term “computer graphics” bring to mind? For a lot of people, “computer graphics” are “cool,” “eye candy,” and probably “state of the art.” You might think about an animation in the game Myst, or Steven Spielberg’s hyperactive Jurassic dinosaurs.
Now think about the term “computer music.” If you’re like most people, this term might bring to mind nothing more than the “beeps” and “boops” of decades gone by. Nevertheless, many impressive musical achievements have been accomplished with the aid of computers. Perhaps the primary problem, then, is one of perception. We need to teach people the importance of sound and music in multimedia.
The DirectX APIs give us the chance to deliver on the promise of multimedia; audio is an integral part of this promise. We are clearly at a crossroads.
Technology has always played a defining role in the evolution of art; it has played the same role with music. Try to imagine what today’s music would sound like without the pioneering work of a Cristofori, an Adolph Sax, or a Les Paul! Certainly the most exciting recent technological developments have found expression in much of today’s music.
The refinement and wide distribution of synthesizer technology through the 1960s and 1970s introduced new sounds, textures, and means of manipulating sound. The effect was far-reaching, from the Beatles to Switched-on Bach. As synthesizers improved in sonic quality, their deployment became even more versatile. Today it is common in most musical styles to mix synthesized sounds with acoustic ones.
The establishment of the MIDI protocol in 1982 was a major development. MIDI made it possible for electronic musical instruments to communicate with other instruments and, more significantly, with computers. Independent composers began to create home studios where they could control the entire recording process. MIDI enabled musicians to create full electronic orchestrations, it facilitated the notation of musical scores, and it provided an economic means to deliver electronic music. MIDI contributed significantly to the “sound” of the 1980s, from sequenced industrial mixes to lush electronic soundtracks. Its contribution was marred by an imprecise timbral definition that can occur when MIDI files are played back on random sound cards. It was this weakness that caused many to say that “MIDI sounds bad.” MIDI was also severely limited by its lack of support for interactive elements.
Sampling technology became widespread as the 1990s got underway. Sampling created two distinct directions for composers to follow. On one hand, it gave composers the ability to create richly layered digital recordings of acoustic and electronic instruments. On the other, it promoted a kind of creative thievery, where composers “borrowed” the licks and grooves of previously recorded artists to create new compositions.
Sampling is most prominent in hip-hop and in the work of disc jockeys who “spin” dance mixes, but its use is widespread in the recording and music industry. Sampling was directly responsible for the repopularizing of “retro” grooves.
Several formats that support sampling technology are available on the PC. The most common of these is the wave (.wav) format. This format makes possible the high-bandwidth digital rendering of sound. A weakness of the wave file format has been the lack of a standard for the economical delivery of musical performances.
Combining MIDI, support for hardware acceleration and software synthesis, and an integrated delivery system for custom samples, DirectMusic provides an economical means of delivering professional quality musical performances. At the same time, DirectMusic’s variable and interactive architecture opens up uncharted territories in the creative landscape.
The DirectMusic API provides dramatic improvement to MIDI support on the Windows platform. Difficult timing issues have been resolved and features such as buffered, time-stamped events and a global time reference make for rock-solid performance.
One of the most powerful aspects of DirectMusic is its full implementation of the industry-ratified downloadable sounds (DLS) specification. The DLS format provides the consistent playback necessary to deliver a quality musical presentation. In the past, it was impossible to get consistent playback with MIDI and performance quality varied depending on the sound card or playback device. Using DLS collections, you can specify the exact timbres desired. Wave files can be imported into a collection and manipulated in the same way that MIDI controllers manipulate any synthesized sound source. Because DirectMusic features the compression of wave files within DLS collections, their use is more viable than ever.
The DLS standard also makes it easy for DirectMusic to take advantage of hardware acceleration through compatible sound cards. If the hardware is not present, then software synthesis guarantees consistent sound production with a minimum of cost to the CPU.
The multilevel DirectMusic architecture provides for user-defined variable performances. At the Pattern level, you can define up to 32 variations for each instrument. You can program the DirectMusic performance engine to randomly select variations to perform. This means that creating variations for only two instruments will result in 1,024 possible juxtapositions. Chordal harmonies can also be grouped and varied through the use of ChordMaps, which feature variable chord definitions, associated underlying scales, and variable chord paths.
Interactivity is at the heart of DirectMusic’s musical architecture. By abstracting musical approaches into separately defined variables, DirectMusic can alter notes, chords, scales, textures, tempos, and patch changes. Sections of music can be instantly started and stopped, or smoothly traveled through the use of transitions. By using DirectMusic Motifs, you can also interactively layer melodic and harmonic patterns and sound effects on top of existing music.
DirectMusic’s powerful tools make it easy for musicians and composers to utilize new techniques to extend and even to redefine the very nature of musical performance. Let’s look at a few possibilities:
DirectMusic provides powerful tools to create the most compelling musical experiences. But no matter how sophisticated the hardware or software, it takes talented musicians, producers and collaborative artists to realize their possibilities. Looking at the dawn of the new millennium, it is our collective responsibility to push audio technology to the limit, to envision fresh possibilities, and to use them to create a new paradigm for music and sound.
Let’s get busy.