Chanel Summers, Audio Technical Evangelist
Microsoft Corporation
December 1999
Summary: This article outlines the most common misconceptions about DirectMusic and presents the facts about its capabilities and design goals. (8 printed pages)
Microsoft® DirectMusic® technology represents an exciting new leap forward for music composition and performance on personal computers. It opens up fresh opportunities for composers and sound designers by providing them tools to create interactive, non-repetitious scores that sound great both in the studio and on the average personal computer. But as with most new technologies, the DirectMusic API requires a small investment of time and thought in order to take advantage of everything it has to offer. The objective of this article is to outline the most common misconceptions about DirectMusic and present the facts about its capabilities and design goals.
Some composers, on hearing about DirectMusic for the first time, have immediately thought: algorithmic music. If it’s interactive, the argument goes, it’s obviously generating music on the fly based on some simple parameters. Composers would be out of work, the doomsayers predicted, replaced by a few lines of code. In reality, once composers begin creating DirectMusic scores, they quickly realize that nothing could be farther from the truth.
DirectMusic empowers composers, providing a compelling set of tools that let them provide information in their score about how the score should evolve based on user actions.
In point of fact, DirectMusic is not about algorithmic composition. It doesn’t create music; it lets a composer control the way music is performed at run time by indicating which patterns and variations are appropriate to a particular mood or event. Composers are even more important to the process, because now they not only create the base score elements, they also provide information in the score that describes how those elements are modified based on game events.
In addition, DirectMusic provides an enormous degree of musical control. DirectMusic can enable composers and developers to alter notes, chords, scales, textures, tempos, volume, or the intensity of the music at any time; create musical transitions; and synchronize musical themes harmonically and rhythmically with the underlying score. Indeed, composers can even incorporate traditional linear music into a DirectMusic score if they choose to do so. In short, both the composer and programmer have tremendous control over how the music is actually performed regardless of what a user does, when they do it, or in what order it’s done.
As PC sound cards have evolved and improved, it has become more and more difficult for composers to create scores that sound consistent on their users’ systems. Because the method of synthesis used by older sound cards sounds far less convincing than that of the latest cards, many users and developers have associated low-quality reproduction with PC music in general. When those users or developers learn that DirectMusic uses MIDI as its underlying trigger mechanism, they assume that DirectMusic will sound like a standard MIDI file played through their sound card’s MIDI synthesis chip.
DirectMusic uses a new MIDI standard called Downloadable Sounds (DLS) to create custom sounds based on actual recordings of instruments. DLS can be used to deliver very high-quality instruments and sound effects with a soundtrack. Because DLS collections are based on wave files, which can contain a single note, a musical phrase, a sound effect, dialogue, or anything else, DLS assures composers that they will get precise timbral definition in their scores. What they create in their studios is what users will hear in their products.
In addition, DirectMusic supports two methods of synthesis: hardware synthesis, in which the sound card uses MIDI events to create the audio heard through a PC’s speakers, and software synthesis, in which the CPU itself creates the audio waveform. The DirectMusic software synthesizer acts like a sampler inside the user’s PC. Thanks to the software synthesizer, most users will hear scores that are created entirely inside the CPU itself and played back much like a recording of a performance. However, in those cases where a sound card provides capabilities beyond the software synthesizer, DirectMusic can use the sound card instead. That way, DirectMusic provides the best of both worlds: the excellent fidelity of the wave format and the compactness, flexibility, and interactivity of MIDI.
While developers can easily imagine interactive, non-repetitious sound tracks in certain types of games, it’s sometimes difficult to picture how DirectMusic might be applied to others.
Certainly, immersive, story-oriented games benefit greatly from interactive soundtracks. But a simple review of the lists of titles using DirectMusic reveals that the technology is being used now in nearly every genre:
Any game (and other types of applications, for that matter) that uses music can benefit from non-repetitious or interactive soundtracks. Users crave music that adapts to their actions, and continues to stay fresh and new.
Furthermore, many developers and sound designers are using the interactive, non-repetitious capabilities of DirectMusic for non-repetitious sound effects. Using “musical” techniques, they are beginning to trigger, layer, and vary their sound effects. Realizing that repetitive sound effects will destroy the reality a game is trying to create, sound designers have begun to deliver their sound effects in DLS collections and use patterns and variations to achieve easy control over the way those effects are selected, combined, and performed.
Similarly, because certain game genres lend themselves readily to specific musical styles, some observers have mistakenly concluded that DirectMusic is only appropriate for those types of music. They assume that MIDI or DLS limits composers to Techno music or rock music.
There is no inherent limitation in DirectMusic that prevents composers from delivering any kind of music that can be recorded and played back. In fact, the evidence shows that DirectMusic is being used now with nearly every style of music:
However, we have found that the quality of the music performed by DirectMusic is directly related to the quality of the DLS collection provided with the score. Just as better musicians can produce better sound quality, a DLS collection that provides a wide range of expression will create the foundation for a superior performance. In addition, the amount of work a composer puts into orchestration will be reflected in a richer, more lavish score.
Because DirectMusic introduces new concepts and terms, it’s possible to conclude that it requires a completely new approach to composition. Some composers might incorrectly assume they won’t be able to adapt.
DirectMusic was created by composers for composers and attempts to mirror, as closely as possible, the process that most composers go through to create a piece of music. DirectMusic is based on the notion that a score comprises basic thematic elements that can be used in combination and varied based on the context. These elements give the composer incredible flexibility when it comes to constructing the actual score at run time.
Because DirectMusic is based on concepts that should be familiar to most composers, most composers find the process simple to learn and quick to master. In addition, because DirectMusic Producer (the authoring tool for DirectMusic) is designed to augment a composer’s existing sequencer programs, importing standard MIDI files as Patterns or Segments is simple and straightforward.
Based on experience, Microsoft estimates show that most composers, regardless of musical background and skill set, are able to become productive in DirectMusic Producer with just a few hours of exploration, and proficient with as little as 40 hours’ work.
It takes far longer or costs much more to create a DirectMusic score than to create a traditional linear score.
Actually, most composers find that rather than spending more time and money on creating a score, they invest the same amount of time and resources but get significantly more music because of the ability of DirectMusic to play variations and change properties of the performance. The DirectMusic variations and other related techniques deliver composers far more music and sound effects in return on their investment.
Of course, composers can just use DirectMusic to play a standard MIDI track with DLS instruments and take advantage of the high-quality and consistent playback capabilities of DirectMusic. But most composers will quickly find that creating an interactive, variable soundtrack takes just a little longer than creating a linear soundtrack. And over time, as experienced composers become more productive with DirectMusic, they will start composing for interactivity from the beginning.
On the other hand, of course, it’s easy to get caught up in all the possibilities.
DirectMusic locks developers into a creativity-limiting API and forces Microsoft-defined file formats on them.
The DirectMusic architecture was engineered from the start to be dynamic and extensible, allowing for customization. DirectMusic was never intended to be the end-all, be-all interactive music solution, but rather a platform that can evolve and be built upon. All file formats are published and all interfaces are based on COM so as to be easily extended and enhanced. Even DirectMusic Producer can be extended with new objects.
Many developers and composers are already extending DirectMusic by creating their own track types, software synthesizers, music generation and processing objects, message types, custom property sets, and custom DLS chunks. For example, while the DirectMusic interactive music engine is not based on algorithmic composition, composers or developers can create their own generative music technology and integrate it into their DirectMusic-based applications.
DirectMusic also introduces the concept of “Tools,” small code routines that developers can write into their applications to process DirectMusic objects and events in real time, opening the door to boundless music manipulation possibilities.
Because it’s so new, DirectMusic won’t be a factor for at least another year.
DirectMusic has been in developers’ hands since February 1998, when it was introduced at Microsoft’s Meltdown conference. Dozens of high-profile titles are in development and began shipping in early 1999. Any developers or composers looking to enhance their products’ soundtracks should be evaluating the benefits of DirectMusic now. Experience has shown that the earlier a development team begins planning to incorporate DirectMusic, the more polished and effective the end results are.
DirectMusic is being rapidly embraced by the games development community, and interest in the technology is high among Web site designers. It is possible to mistakenly assume that DirectMusic is only suitable to one of those types of applications.
The fact is that DirectMusic is being used in many different ways, and will be seen in many types of applications as more and more developers and composers learn about it and begin exploring its benefits. As the technology evolves, it will continue to accommodate the needs of any application that can profit from a music soundtrack. It will be used in games, it will be used in other types of applications, it will be used in Web pages, it will be used to teach music, it will be used to create linear music. The possibilities are endless. The applications for DirectMusic are only limited by the creativity of the development team.
DirectMusic chews up CPU cycles and RAM.
It’s natural to assume that technology like DirectMusic taxes a system’s processor and requires large amounts of memory. However, DirectMusic has been designed to be very frugal with system resources.
Because it’s based on MIDI, the DirectMusic composition engine uses very little CPU and RAM.
In addition, the DirectMusic software synthesizer consumes just 0.12 percent of the CPU per voice on a system with a 266-megahertz (MHz) Pentium II. As a result, a full 16-voice soundtrack can be rendered with less than a 2 percent impact to the processor. If you choose to use our integrated software reverb, you will add a bit less than 2 percent to that figure, but you can still create very rich scores with little CPU utilization. Similarly, the software synthesizer’s latency is only approximately 100 milliseconds and has been optimized for a P166 with 32 megabytes (MB) of RAM. However, there is almost no latency or CPU load when using hardware acceleration.
DirectMusic is also very prudent about RAM usage. The software synthesizer renders into a 44-kilobyte (KB) buffer by default and loads and unloads DLS instrument regions as required. This means that only the regions of those instruments that you are actually using at any one time, and nothing more, will be resident in memory. Developers can also have complete control over when those instruments are loaded, if they choose. Significantly, because most recent sound cards can take wavetable data out of system RAM, it’s no longer necessary to fit instruments into a sound card’s on-board RAM—even if using a hardware accelerator. Finally, we allow developers to use any ACM-compatible codec to compress waves in a DLS collection, a feature that has obvious benefits for Internet developers.
To get the most out of DirectMusic, users will need new sound cards with special capabilities.
DirectMusic will work on any sound card, thanks to its software synthesizer. New sound cards will accelerate DirectMusic, providing slightly better audio quality and support for the next generation of DLS: DLS Level 2. DirectMusic accelerators will deliver lower latency than a software synthesizer, but the difference between software and hardware synthesis should be small enough that composers can author a piece of music confident in the knowledge that their scores will sound consistent from system to system.
Music isn’t important enough to warrant the extra work.
Most developers and composers recognize the crucial role music plays in their customers’ emotional lives, and the way it can alter their perceptions. Music can provide information or affect users in subtle ways that dramatically enhance their experience. In fact, such is the power of audio that studies have shown how poor audio quality can actually make users believe that an application’s graphics are of lower relative quality.
An investment in better music and interactivity will lead to increased customer satisfaction, direct competitive advantage, and increased sales.
As more and more applications deliver higher-quality sound reproduction, thanks to DirectMusic, more developers will recognize the value of music and the impact it can have on the perception of their applications.
It isn’t a lot of extra work to do what we’re proposing, and even if you don’t take music seriously, your competitors will!
DirectMusic makes the Microsoft Windows® operating system a great platform for MIDI playback. If nothing else, composers and developers should use DirectMusic to play MIDI files with better sequencer timing. However, by adding DLS instruments to a MIDI score, DirectMusic provides the ideal solution for music delivery in applications: high quality, flexible performance, and small file sizes. The DirectMusic software synthesizer and support for DLS ensure consistent playback across different audio hardware configurations, and DirectMusic allows composers to move beyond the limits of linear music to deliver interactive, non-repetitious musical scores that become an integral part of the application and an essential element of the user’s experience.
--------------------------------------------
The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of publication. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the date of publication.
This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS DOCUMENT.