About the Multimedia Streaming Architecture

This article describes the architecture of multimedia streaming and how software developers typically use streams in their tools and applications. It also covers the advantages of streaming as a base for data transfer and how to address multimedia programming issues, such as data transfer, performance optimization, and time stamping in streaming applications. Programmers who want to use multimedia streaming should be familiar with COM programming concepts.

This article contains the following sections.

Advantages of Multimedia Streaming

When developers use multimedia streaming in their applications, it greatly reduces the amount of format-specific programming needed. Typically, an application that must obtain media data from a file or hardware source must know everything about the data format and the hardware device. The application must handle the connection, transfer of data, any necessary data conversion, and the actual data rendering or file storage. Because each format and device is slightly different, this process is often complex and cumbersome. Multimedia streaming, on the other hand, automatically negotiates the transfer and conversion of data from the source to the application. The streaming interfaces provide a uniform and predictable method of data access and control, which makes it easy for an application to play back the data, regardless of its original source or format.

The following steps show how to implement streaming, from hardware device to rendered playback.

  1. A source of video data, such as Microsoft® DirectShow®, exposes the streaming interfaces.
  2. The application developer uses the multimedia streaming interfaces to handle data format conversion.
  3. The application developer uses the Microsoft DirectDraw® interfaces to render the resulting data.

The specification for multimedia streams comprises several interfaces; each interface includes methods that control a certain aspect of the streaming process or handle a certain type of data. See List of Multimedia Streaming Interfaces for additional information.

Object Hierarchy

The following diagram shows the basic object hierarchy used in multimedia streaming.

Object hierarchy

There are three basic object types defined in the multimedia streaming architecture:

  1. A multimedia stream, which supports the IMultiMediaStream interface.
  2. Media streams support the IMediaStream interface and are data specific. Every multimedia stream contains one or more of these media streams.
  3. Stream samples support the IStreamSample interface and are created by a media stream. These objects represent a basic unit of work for the stream.

Creating Multimedia Stream Objects and Stream Samples

Objects that support the IMultiMediaStream interface are the basic containers for multimedia data streams. The IMultiMediaStream interface includes methods that enumerate the object's data streams; these streams are typically video and audio data, but can include data of any format, such as closed-captioning, plain text, or SMPTE timecode. The IMultiMediaStream interface is a generic container, however; developers can create other versions of the interface that support specific data formats. Objects that implement the IAMMultiMediaStream interface, for example, can enumerate and control streams of any DirectShow data format. Because individual data streams are format specific, they support at least two different interfaces: one generic and one data-specific. Every stream supports the IMediaStream interface, which provides methods to retrieve its format and a pointer to the stream itself. The IDirectDrawMediaStream interface, on the other hand, has methods that deal specifically with rendering video data. Any interface derived from IMultiMediaStream also supports the creation of stream samples, the basic units of streaming data.

A multimedia sample is a reference to an object containing the media data. For a video image, this is a DirectDraw surface. The sample's exact content varies, depending on the type of media (sound, text, and so on). Because a sample is only a reference to the data object, any number of stream samples can refer to the same object. The IStreamSample interface provides methods that get and set a sample's characteristics, such as its start and stop time, status, and stream association. The IStreamSample::Update method refreshes the sample's data in the case of readable streams. For writable streams, it will write the sample's data to the stream. Typically, you use the Update method in a loop that renders, transfers, or stores streaming data. See Use Multimedia Streaming in DirectShow Applications for a practical example of this method in source code.

Using Multimedia Streams in Applications

The multimedia streaming interfaces greatly simplify the process of manipulating multimedia data by removing the dependency on specific characteristics of the hardware or software source and providing support for all Microsoft DirectX® media formats. Streams abstract the data to a very high level; applications can even move data from one stream to another without knowing anything about the data's format.

Perform the following steps to create a multimedia stream.

  1. Create the multimedia stream. The method of creation and initialization of the stream is architecture specific. DirectShow supports the IAMMultiMediaStream interface, which is used to initialize the stream. Other in-process server implementations of IMultiMediaStream will be created and initialized using different mechanisms.
  2. After the multimedia stream object is initialized, the application will use QueryInterface to retrieve the IMultiMediaStream interface for the object. Use this interface to determine the stream's properties and enumerate the streams themselves. You can retrieve a specific stream by calling the IMultiMediaStream::GetMediaStream method with a specific purpose ID. MSPID_PrimaryVideo and MSPID_PrimaryAudio, which represent the primary video and audio streams, are the most commonly used purpose IDs.
  3. Call IUnknown::QueryInterface for an interface specific to the stream's media type. If you want to render a video stream, for example, retrieve its IDirectDrawMediaStream interface. Media-specific interfaces define additional methods necessary for taking full advantage of a format's capabilities.
  4. Create one or more samples from the stream data. Every media stream supports the IMediaStream::CreateSharedSample method for sample creation. The resulting sample supports the IStreamSample interface, which provides control over the sample and its characteristics. Typically, the media stream supports a format-specific method of sample creation that is more powerful than the aforementioned IStreamSample methods. IDirectDrawMediaStream, for example, can create samples attached to a desired DirectDraw surface and clipping rectangle. In some situations, however, you must handle data without knowing about its data format, such as when constructing a cutlist. If you want to stream data independent of its format, use the IMediaStream::CreateSharedSample method to create the data samples.
  5. After creating all desired stream samples, start the stream by calling the IMultiMediaStream::SetState method and pass in the STREAMSTATE_RUN flag as its parameter.
  6. Call IStreamSample::Update to update the stream sample. When the IStreamSample::Update method exits, you can access the sample's data. The code sample in Use Multimedia Streaming in DirectShow Applications uses the IStreamSample::Update method in a loop to render a stream of video data. If you want a trigger a specific event or function call when the update returns, pass the appropriate pointers to the IStreamSample::Update method.

For more information on the multimedia streaming interfaces, see Multimedia Streaming.

Sharing Data Between Streams

Processing multimedia data typically requires a great deal of system resources; therefore, you should avoid copying data whenever possible. The streaming architecture supports shared stream samples, a mechanism that moves data from one stream to another without copying it. This buffer enables the efficient transportation of data between two streams even if the destination stream doesn't specifically support the underlying data format.

For example, assume that you have a multimedia stream with three data streams: video and audio, and URL data time-stamped to match the video content. You want to write an application that adds a copyright notice on every video frame and writes the data to another stream for storage, but your application doesn't understand any data formats except the video stream. For the video stream, you create a sample attached to the desired DirectDraw surface. You can then create an output stream by calling either the IDirectDrawMediaStream::CreateSample method with that pointer to the same surface, or IMediaStream::CreateSharedSample. In both cases, the input and output streams share the DirectDraw surface. Because you understand the video format, you can access this surface as needed.

To retrieve the other source stream pointers (audio and URL), enumerate the source container stream and grab pointers to the nonvideo streams. Each of these source streams has an associated output stream in the output stream container. Retrieve these output pointers by calling the IMultiMediaStream::GetMediaStream method on the output container with each of the source stream pointers. The following steps describe this process.

  1. Call IMultiMediaStream::EnumMediaStreams to retrieve a pointer to a source stream. Make sure that it's not the video stream, because your application already understands its format.
  2. Call IMultiMediaStream::GetMediaStream on the output container stream with the pointer from step 1. This returns a pointer to the desired output stream.
  3. Call AllocateSample on the source stream.
  4. Call CreateSharedSample on the output stream.
  5. Call Update on the source stream to read the data.
  6. Call Update on the output stream to write the data.

Repeat these steps for each stream whose format you don't support. When both samples finish updating, the output stream has all data from the source stream and you are done.

Top of Page Top of Page
© 2000 Microsoft and/or its suppliers. All rights reserved. Terms of Use.