DirectShow Animated Header -- Alternative Video Renderers DirectShow Animated Header -- Alternative Video Renderers* Microsoft DirectShow SDK 
*Index  *Topic Contents

*Previous Topic: Full-Screen Video Renderer
*Next Topic: Source and Target Rectangles in Video Renderers

Alternative Video Renderers


This article describes some of the more complicated implementation requirements of a renderer; these apply to most renderers, although some aspects are video-specific (such as EC_REPAINT and other notifications). In particular, it discusses how to handle various notifications, state changes, and format changes. It also provides a summary of the notifications that a renderer is responsible for sending to the filter graph manager.

Contents of this article:

Writing an Alternative Renderer

Microsoft® DirectShow™ provides a window-based video renderer; it also provides a full-screen renderer in the run-time installation. You can use the C++ classes in the DirectShow SDK to write alternative video renderers. For alternative renderers to interact correctly with DirectShow-based applications, the renderers must adhere to the guidelines outlined in this article. You can use the CBaseRenderer and CBaseVideoRenderer classes to help follow these guidelines when implementing an alternative video render. Consult the SampVid sample in the DirectShow SDK for an example of an alternative video renderer that uses these classes. Because of the ongoing development of DirectShow, review your implementation periodically to ensure that the renderers are compatible with the most recent version of DirectShow.

This article discusses many notifications that a renderer is responsible for handling. A brief review of DirectShow notifications might help to set the stage. There are essentially three kinds of notifications that occur in DirectShow:

This article discusses the responsibility of the renderer filter in handling stream notifications it receives and in sending appropriate filter graph manager notifications.

Handling End-of-stream and Flushing Notifications

An end-of-stream notification begins at an upstream filter (such as the source filter) when that filter detects that it can send no more data. It is passed through every filter in the graph and eventually ends at the renderer, which is responsible for subsequently sending an EC_COMPLETE notification to the filter graph manager. Renderers have special responsibilities when it comes to handling these notifications.

A renderer receives an end-of-stream notification when its input pin's IPin::EndOfStream method is called by the upstream filter. A renderer should note this notification and continue to render any data it has already received. Once all remaining data has been received, the renderer should send an EC_COMPLETE notification to the filter graph manager. The EC_COMPLETE notification should be sent only once by a renderer each time it reaches the end of a stream. Furthermore, EC_COMPLETE notifications must never be sent except when the filter graph is running. Therefore, if the filter graph is paused when a source filter sends an end-of-stream notification, then EC_COMPLETE should not be sent until the filter graph is finally run.

Any calls to the IMemInputPin::Receive or IMemInputPin::ReceiveMultiple methods after an end-of-stream notification is signaled should be rejected. E_UNEXPECTED is the most appropriate error message to return in this case.

When a filter graph is stopped, any cached end-of-stream notification should be cleared and not resent when next started. This is because the filter graph manager always pauses all filters just before running them so that proper flushing occurs. So, for example, if the filter graph is paused and an end-of-stream notification is received, and then the filter graph is stopped, the renderer should not send an EC_COMPLETE notification when it is subsequently run. If no seeks have occurred, the source filter will automatically send another end-of-stream notification during the pause state that precedes a run state. If, on the other hand, a seek has occurred while the filter graph is stopped, then the source filter might have data to send, so it won't send an end-of-stream notification.

Video renderers often depend on end-of-stream notifications for more than the sending of EC_COMPLETE notifications. For example, if a stream has finished playing (that is, an end-of-stream notification is sent) and another window is dragged over a video renderer window, a number of WM_PAINT window messages will be generated. The typical practice for running video renderers is to refrain from repainting the current frame upon receipt of WM_PAINT messages (based on the assumption that another frame to be drawn will be received). However, when the end-of-stream notification has been sent, the renderer is in a waiting state; it is still running but is aware that it will not receive any additional data. Under these circumstances, the renderer customarily draws the playback area black.

Handling flushing is an additional complication for renderers. Flushing is carried out through a pair of IPin methods called BeginFlush and EndFlush. Flushing is essentially an additional state that the renderer must handle. It is illegal for a source filter to call BeginFlush without calling EndFlush, so hopefully the state is short and discrete; however, the renderer must correctly handle data or notifications it receives during the flush transition.

Any data received after calling BeginFlush should be rejected immediately by returning E_UNEXPECTED. Furthermore, any cached end-of-stream notification should also be cleared when a renderer is flushed. A renderer will typically be flushed in response to a seek. The flush ensures that old data is cleared from the filter graph before fresh samples are sent. (Typically, the playing of two sections of a stream, one after another, is best handled through deferred commands rather than waiting for one section to finish and then issuing a seek command.)

Handling State Changes and Pause Completion

A renderer filter behaves the same as any other filter in the filter graph when its state is changed, with the following exception. After being paused, the renderer will have some data queued, ready to be rendered when subsequently run. When the video renderer is stopped, it holds on to this queued data. This is an exception to the DirectShow rule that no resources should be held by filters while the filter graph is stopped.

The reason for this exception is that by holding resources, the renderer will always have an image with which to repaint the window if it receives a WM_PAINT message. It also has an image to satisfy methods, such as CBaseControlVideo::GetStaticImage, that request a copy of the current image. Another effect of holding resources is that holding on to the image stops the allocator from being decommitted, which in turn makes the next state change occur much faster because the image buffers are already allocated.

A video renderer should render and release samples only while running. While paused, the filter might render them (for example, when drawing a static poster image in a window), but should not release them. Audio renderers will do no rendering while paused (although they may perform other activities, such as preparing the wave device, for example). The time at which the samples should be rendered is obtained by combining the stream time in the sample with the reference time passed as a parameter to the IMediaControl::Run method. Renderers should reject samples with start times less than or equal to end times.

When an application pauses a filter graph, the filter graph does not return from its IMediaControl::Pause method until there is data queued at the renderers. In order to ensure this, when a renderer is paused, it should return S_FALSE if there is no data waiting to be rendered. If it has data queued, then it can return S_OK.

The filter graph manager checks all return values when pausing a filter graph, to ensure that the renderers have data queued. If one or more filters are not ready, then the filter graph manager polls the filters in the graph by calling GetState. The GetState method takes a time-out parameter. A filter (typically a renderer) that is still waiting for data to arrive before completing the state change returns VFW_S_STATE_INTERMEDIATE if the GetState method expires. Once data arrives at the renderer, GetState should be returned immediately with S_OK.

In both the intermediate and completed state, the reported filter state will be State_Paused. Only the return value indicates whether the filter is really ready or not. If, while a renderer is waiting for data to arrive, its source filter sends an end-of-stream notification, then that should also complete the state change.

Once all filters actually have data waiting to be rendered, the filter graph will complete its pause state change.

Handling Termination

Video renderers must correctly handle termination events from the user. This implies correctly hiding the window and knowing what to do if a window is subsequently forced to be displayed. Also, video renderers must notify the filter graph manager when its window is destroyed (or more accurately, when the renderer is removed from the filter graph) to free resources.

If the user closes the video window (for instance by pressing ALT+F4), the convention is to hide the window immediately and send an EC_USERABORT notification to the filter graph manager. This notification is passed through to the application, which will stop the graph playing. After sending EC_USERABORT, a video renderer should reject any additional samples delivered to it.

The abort flag should be left on by the renderer until it is subsequently stopped, at which point it should be reset so that an application can override the user action and continue playing the graph if it desires. If ALT+F4 is pressed while the video is running, the window will be hidden and all further samples delivered will be rejected. If the window is subsequently shown (perhaps through IVideoWindow::put_Visible), then no EC_REPAINT notifications should be generated.

The video renderer should also send the EC_WINDOW_DESTROYED notification to the filter graph when the video renderer is terminating. In fact, it is best to handle this when the renderer's IBaseFilter::JoinFilterGraph method is called with a null parameter (indicating that the renderer is about to be removed from the filter graph), rather than waiting until the actual video window is destroyed. Sending this notification allows the plug-in distributor in the filter graph manager to pass on resources that depend on window focus to other filters (such as audio devices).

Handling Dynamic Format Changes

Video renderers in DirectShow accept only video formats that can be drawn efficiently. For example, the window-based run-time renderer will accept only the RGB format that matches the current display device mode (for example, RGB565 when the display is set to 65,536 colors). As a last resort, it also accepts 8-bit palettized formats, as most display cards can draw this format efficiently. When the renderer has Microsoft® DirectDraw® loaded, it might later ask the source filter to switch to something that can be written onto a DirectDraw surface and drawn directly through display hardware. In some cases, the renderer's upstream filter might try to change the video format while the video is playing. This often occurs when a video stream has a palette change. It is most often the video decompressor that initiates a dynamic format change.

An upstream filter attempting to change formats dynamically should always call the IPin::QueryAccept method on the renderer input pin (for filters based on CTransformFilter, this is implemented in CTransformFilter::CheckInputType). It is undefined as to which formats a renderer will allow an upstream filter to change dynamically. However, at a very minimum, it should allow the upstream filter to change palettes. When an upstream filter changes media types, it will attach the format to the first sample delivered in that new type. If the renderer holds many samples in a queue waiting to be rendered, it should delay changing the format until the sample with the type change is actually about to be rendered.

Whenever a format change is detected by the video renderer, it should send an EC_DISPLAY_CHANGED notification. Most video renderers pick a format during connection so that the format can be drawn efficiently through GDI. If the user changes the current display mode without restarting the computer, a renderer might find itself with a bad image format connection and should send this notification. The first parameter should be the pin that needs reconnecting. The filter graph manager will arrange for the filter graph to be stopped and the pin reconnected. During the subsequent reconnection, the renderer can accept a more appropriate format.

Whenever a video renderer detects a palette change in the stream it should send the EC_PALETTE_CHANGED notification to the filter graph manager. The DirectShow video renderers detect whether a palette has really changed in dynamic format or not. The video renderers do this not only to filter out the number of EC_PALETTE_CHANGED notifications sent but also to reduce the amount of palette creation, installation, and deletion required.

Finally, the video renderer might also detect that the size of the video has changed, in which case, it should send the EC_VIDEO_SIZE_CHANGED notification. An application might use this notification to negotiate space in a compound document. The actual video dimensions are available through the IBasicVideo control interface. The DirectShow renderers detect whether the video has actually changed size or not prior to sending these events.

Handling Persistent Properties

All properties set through the IBasicVideo and IVideoWindow interfaces are meant to be persistent across connections. Therefore, disconnecting and reconnecting a renderer should show no effects on the window size, position, or styles. However, if the video dimensions change between connections, the renderer should reset the source and destination rectangles to their defaults. The source and destination positions are set through the IBasicVideo interface.

Both IBasicVideo and IVideoWindow provide enough access to properties to allow an application to save and restore all the data in the interface in a persistent format. This will be useful to applications that must save the exact configuration and properties of filter graphs during an editing session and restore them later.

Handling EC_REPAINT Notifications

The EC_REPAINT notification is sent only when the renderer is either paused or stopped. This notification signals to the filter graph manager that the renderer needs data. If the filter graph is stopped when it receives one of these notifications, it will pause the filter graph, wait for all filters to receive data (by calling GetState), and then stop it again. When stopped, a video renderer should hold on to the image so that subsequent WM_PAINT messages can be handled.

Therefore, if a video renderer receives a WM_PAINT message when stopped or paused, and it has nothing with which to paint its window, then it should send EC_REPAINT to the filter graph manager. If an EC_REPAINT notification is received while paused, then the filter graph manager calls IMediaPosition::put_CurrentPosition with the current position (that is, seeks to the current position). This causes the source filters to flush the filter graph and causes new data to be sent through the filter graph.

A renderer must send only one of these notifications at a time. Therefore, once the renderer sends a notification, it should ensure no more are sent until some samples are delivered. The conventional way to do this is to have a flag to signify that a repaint can be sent, which is turned off after an EC_REPAINT notification is sent. This flag should be reset once data is delivered or when the input pin is flushed, but not if end-of-stream is signaled on the input pin.

If the renderer does not monitor its EC_REPAINT notifications, it will flood the filter graph manager with EC_REPAINT requests (which are relatively expensive to process). For example, if a renderer has no image to draw, and another window is dragged across the window of the renderer in a full-drag operation, the renderer receives multiple WM_PAINT messages. Only the first of these should generate an EC_REPAINT event notification from the renderer to the filter graph manager.

A renderer should send its input pin as the first parameter to the EC_REPAINT notification. By doing this, the attached output pin will be queried for IMediaEventSink, and if supported, the EC_REPAINT notification will be sent there first. This allows output pins to handle repaints before the filter graph must be touched. This will not be done if the filter graph is stopped, because no buffers would be available from the decommitted renderer allocator.

If the output pin cannot handle the request and the filter graph is running, then the EC_REPAINT notification is ignored. An output pin must return NOERROR (S_OK) from IMediaEventSink::Notify to signal that it processed the repaint request successfully. The output pin will be called on the filter graph manager worker thread, which avoids having the renderer call the output pin directly, and so sidesteps any deadlock issues. If the filter graph is stopped or paused and the output doesn't handle the request, then the default processing is done.

Handling Notifications in Full-Screen Mode

The IVideoWindow plug-in distributor (PID) in the filter graph manages full-screen playback. It will swap a video renderer out for a specialist full-screen renderer, stretch a window of a renderer to full screen, or have the renderer implement full-screen playback directly. To interact in full-screen protocols, a video renderer should send an EC_ACTIVATE notification whenever its window is either activated or deactivated. In other words, an EC_ACTIVATE notification should be sent for each WM_ACTIVATEAPP message a renderer receives.

When a renderer is being used in full-screen mode, these notifications manage the switching into and out of that full-screen mode. Window deactivation typically occurs when a user presses ALT+TAB to switch to another window, which the DirectShow full-screen renderer uses as a cue to return to typical rendering mode.

When the EC_ACTIVATE notification is sent to the filter graph manager upon switching out of full-screen mode, the filter graph manager sends an EC_FULLSCREEN_LOST notification to the controlling application. The application might use this notification to restore the state of a full-screen button, for example. The EC_ACTIVATE notifications are used internally by DirectShow to manage full-screen switching on cues from the video renderers.

Summary of Notifications

This section lists the filter graph notifications that a renderer can send.
Event notification Description
EC_ACTIVATE Sent by video renderers in full-screen rendering mode for each WM_ACTIVATEAPP message received.
EC_COMPLETE Sent by renderers after all data has been rendered.
EC_DISPLAY_CHANGED Sent by video renderers when a display format changes.
EC_PALETTE_CHANGED Sent whenever a video renderer detects a palette change in the stream.
EC_REPAINT Sent by stopped or paused video renderers when a WM_PAINT message is received and there is no data to display. This causes the filter graph manager to generate a frame to paint to the display.
EC_USERABORT Sent by video renderers to signal a closure that the user requested (for example, a user closing the video window).
EC_VIDEO_SIZE_CHANGED Sent by video renderers whenever a change in native video size is detected.
EC_WINDOW_DESTROYED Sent by video renderers when the filter is removed or destroyed so that resources that depend on window focus can be passed to other filters.

© 1998 Microsoft Corporation. All rights reserved. Terms of Use.

*Top of Page