When handling receive data in the NIC driver, ensure that any memory receiving data from the netcard is not present in the cache before a read begins. If the memory is in the cache at the time of the read, the driver obtains the old cache data instead of the new data in physical memory. In addition, if the processor flushes the cache at the wrong time, it overwrites the new data with the old values.
Write your driver to flush cached memory with a call to NdisFlushBuffer. For receive data, set the direction parameter used in the call to FALSE, indicating a read operation. It is critical that you flush cached memory before the network interface card begins writing data into it. Be sure that the driver does not access memory in any way until the network interface card has completed its write. This ensures that memory is not in the cache, and forces subsequent reads to go to physical memory, which contains the current data.
Note For a buffer that does not begin and end on an address that is a multiple of the cache fill size, do not have your driver call NdisFlushBuffer with the write-to-device parameter set to FALSE.
Cache line tearing can occur during the flushing of data that covers one part of a cache block while another process in the operating system owns the rest of the block. Tearing is a problem with receive data in cached memory. It is illegal to make a DMA into memory that is on a cache block that the NIC driver does not completely own. If the driver makes such an access, the owner of the rest of the cache block can flush it while the DMA is in progress, overwriting the new data with stale data from the cache.
You can generally avoid cache line tearing, since the NIC driver allocates memory that the netcard uses to receive data. The driver also ensures that receive buffers do not cross cache lines. Memory the NDIS interface library wrapper allocates with the NDIS_MEMORY_CACHE_ALIGNED flag does not share cache blocks with any other memory. If your driver divides allocated memory, use NdisGetCacheFillSize to ensure that each piece does not share cache blocks. The driver can obtain the cache fill size using this function, and then examine virtual or physical addresses of the memory with the assumption that relevant low bits in memory are the same for virtual and physical addresses.
You must be careful that a netcard performs DMA as a slave device. For receive data, DMA occurs during MacTransferData processing. The NIC driver cannot control the alignment of the buffers that a transport driver passes in. If the NIC driver discovers that the beginning or end of a buffer is not on a cache line, it must DMA-transfer these pieces into a private buffer and then copy them. This is the only way to guarantee that the NIC driver reads data correctly.
When using noncached memory for receive data, make sure that the NIC driver calls the appropriate NdisReadRegisterXxx function for the data type (UCHAR, ULONG, USHORT). These functions ensure the data is in physical memory before they return. They also ensure sequential reads to memory in the order in which they occur and consecutive reads to the same memory, resulting in two distinct reads to memory. This is especially important when the driver accesses memory that corresponds to netcard registers.
Your driver should allocate any receive data that multiple transport drivers can read several times during indications as cached. In general, ownership of receive buffers is clear. The NIC driver can use cached buffers as long as it flushes them just before releasing them. This does not work if releasing the buffers involves writing to them, unless the ownership bits are not on the same cache block as the received data. A driver reads other receive data (such as receive descriptors) only once; so the NIC driver can allocate associated memory as noncached.
The NIC driver must flush receive data in multiples of the cache fill size. Some netcards receive consecutive packets into immediately adjacent memory. The end of one packet may occupy the same cache block as the beginning of the next. Therefore, the driver must allocate such memory as noncached because reading the first packet may cause stale cache data for the start of the second one.