Consider the following an implementation guideline for NT drivers that call KeInsertDeviceQueue or KeInsertByKeyDeviceQueue directly: The I/O Manager usually (except for FSDs) creates an associated device queue object when an NT driver calls IoCreateDevice. It also provides IoStartPacket and IoStartNextPacket, which drivers can call to have the I/O Manager insert IRPs into the associated device queue or call their StartIo routines with an IRP, as desribed in Section 7.2.
Consequently, very few NT driver designers find it necessary (or particularly useful) to set up their own device queue objects for IRPs. Likely candidates are drivers, such as the NT SCSI port driver, that must coordinate incoming IRPs from some number of closely coupled class drivers for heterogeneous devices that are serviced through a single controller or bus adapter.
In other words, the designer of an NT driver for a disk array controller is more likely to use a driver-created controller object than to set up supplemental device queue object(s), while the designer of an NT driver for an add-on bus adapter and of a set of class drivers is slightly more likely to use supplemental device queues.
The KeInsert..DeviceQueue routines return a Boolean value indicating whether the given IRP was inserted into the device queue. As mentioned in in Chapter 3, KeInsertDeviceQueue either inserts an entry at the tail of the device queue and sets the state of the device queue object to Busy, or it simply sets the state to Busy and returns FALSE to indicate that the IRP was not queued. A driver that sets up a supplemental device queue can impose a driver-determined order on the processing of IRPs by calling KeInsertByKeyDeviceQueue, which also returns a Boolean value indicating whether the given IRP was inserted into the device queue.
Consider the following an implementation guideline for NT drivers that call KeInsertDeviceQueue or KeInsertByKeyDeviceQueue directly: If the return value is FALSE, the caller must pass the IRP on for further processing by another driver routine immediately.
A call to KeInsertDeviceQueue or KeInsertByKeyDeviceQueue sets the state of the device queue object to Busy, even if the queue is currently empty and the input IRP is not inserted into the queue. A reciprocal call to KeRemove..DeviceQueue resets the state of an empty device queue object to Not-Busy.
Such a driver can remove an IRP from its supplemental device queue (or reset the device queue state to Not-Busy) by calling KeRemoveDeviceQueue. The driver also can remove a particular entry or determine whether it is currently queued by calling KeRemoveEntryDeviceQueue.
By calling IoStartPacket and IoStartNextPacket, an NT device driver’s Dispatch and DpcForIsr (or CustomDpc) routines synchronize calls to its StartIo routine using the device queue that the I/O Manager created when the driver created its own device object. For a port driver with a StartIo routine, IoStartPacket and IoStartNextPacket insert and remove IRPs in the device queue for the port driver’s shared device controller/adapter. If such a port driver also sets up supplemental device queues to hold requests coming in from closely coupled higher-level class drivers, it must “sort” incoming IRPs into its supplemental device queues, usually in its StartIo routine.
However, the closely coupled class drivers of such a port driver must identify their devices in the IRPs they send down so that the port driver can attempt to insert each IRP into the appropriate driver-managed device queue before attempting to program its shared controller/adapter to carry out the request. Thus, such a port driver can process incoming requests for all devices on a first-come, first-served basis until a call to KeInsertDeviceQueue puts an IRP into a particular class driver’s device queue.
However, such a port driver’s StartIo routine must determine which class driver sent any incoming request. When each class driver called IoGetDeviceObjectPointer to layer itself over the port driver, each class driver was given a unique file object pointer for its “open” of the port driver’s device object. Each class driver can set this file object pointer in the port driver’s I/O stack location to identify itself when it sets up IRPs for the port driver.
By using its own device queue for all IRPs to be processed through its StartIo routine, such an underlying port driver serializes operations through the shared device (or bus) controller/adapter to all attached devices. By sometimes holding IRPs for each supported device in a separate device queue, this port driver inhibits the processing of IRPs for an already busy device while increasing I/O throughput for every other device doing I/O through its shared hardware.
At the call to IoStartPacket from such a port driver’s Dispatch routine, the I/O Manager either calls that driver’s StartIo routine immediately or puts the IRP into the device queue associated with the device object for the port driver’s shared controller/adapter.
Such a port driver with supplemental device queues would have a StartIo routine with the following features:
{
PIO_STACK_LOCATION irpSp =
IoGetCurrentIrpStackLocation(Irp);
PDEVICE_EXTENSION deviceExtension =
DeviceObject->DeviceExtension;
PFILE_OBJECT whichClass;
PATTACHED_DEVICE_STATE attachedDevice;
: :
//
// Switch on MajorFunction code in IRP.
//
switch (irpSp->MajorFunction) {
case IRP_MJ_READ:
case IRP_MJ_WRITE:
//
// Get which device this request is for.
//
whichClass = irpSp->FileObject;
attachedDevice = GetAttachedDeviceState(
deviceExtension,
whichClass);
//
// Increment the following to get a unique value
// to give the error log routine, if necessary.
//
attachedDevice->SequenceNumber++;
: :
case IRP_MJ_DEVICE_CONTROL:
//
// Get which device this request is for.
//
whichClass = irpSp->FileObject;
attachedDevice = GetAttachedDeviceState(
deviceExtension,
whichClass);
: :
} // end switch
}
Note that this code fragment checks state information, based on the file object pointer that each class driver sets in the underlying port driver’s I/O stack location of the IRP, in order to determine the current state of the (attached) target device for any given request. The device object pointer input to its StartIo routine gives the port driver access to its own device object, representing the device controller/adapter. It does not give this port driver access to the class driver’s device object for the target device. Such a port driver must maintain its own state information about each of the heterogeneous devices that it services through the shared device controller/adapter.
NT driver writers who design class/port drivers with supplemental device queues in the port driver should keep in mind the following facts: By design, the I/O Manager does not provide a support routine for getting such a pointer. Moreover, the order in which NT drivers are loaded makes it impossible for lower drivers to get pointers for higher-level drivers’ device objects, which have not yet been created when any lower-level driver is initializing itself.
There is no way to synchronize access to a single device object (and its device extension) between two drivers in a multiprocessor-safe manner. Neither driver can make any assumptions about what I/O processing the other driver is currently doing.
Even for closely coupled class/port drivers, each class driver should use the pointer to the port driver’s device object(s) only to pass on IRPs with IoCallDriver. The underlying port driver must maintain its own state, probably in the port driver’s device extension, about requests that it processes for any closely coupled class driver(s)’ device(s).
Any NT port driver that queues IRPs in supplemental device queues for a closely coupled set of class drivers also must handle the following situation efficiently:
Consequently, such a port driver’s DpcForIsr must attempt to transfer an IRP from the driver’s internal device queue for a particular device into the device queue for the shared adapter/controller whenever the port driver completes an IRP, as follows:
Note that the preceding sequence implies that such a DpcForIsr routine also must call the internal GetAttachedDeviceState routine shown in the preceding StartIo code fragment. That is, the DpcForIsr must determine the device for which it is completing the current (input) IRP in order to manage internal queueing of IRPs efficiently.
If such a port driver attempted to wait until its shared controller/adapter was idle before dequeueing IRPs held in its supplemental device queues, the driver might starve a device for which there was heavy I/O demand while it promptly serviced every other device for which the current I/O demand was actually much lighter.