PRB: Poor TCP/IP Performance When Doing Small Sends

4.00    | 3.10 3.50 3.51

The information in this article applies to:

• Microsoft Win32 Software Development Kit (SDK), versions 3.1, 3.5, 3.51, and 4.0

SYMPTOMS

When doing multiple sends of less than the Maximum Transmission Unit (MTU), you may see poor performance. On an Ethernet network, the MTU for TCP/IP is 1460 bytes.

CAUSE

When an application does two sends of less than a transport MTU, the second send is delayed until an ACK is received from the remote host. The delay occurs in case the application does another small send. TCP can then coalesce the two small sends into one larger packet. This concept of collecting small sends into larger packets is called Nagling.

RESOLUTION

There are a number of ways to avoid Nagling in an application. Here are two. The second is more complex but gives a better performance benefit:

• Set the TCP_NODELAY socket option for the socket. This tells TCP/IP to send always, regardless of packet size. This will result in sub-optimal use of the physical network, but it will avoid the delay of waiting for an ACK.
• Send larger blocks of data. The send() API call, when you include the overhead of the other network components involved, costs a couple of thousand instructions. One large send() call will be more efficient than two smaller send() calls, even if you need to do some buffer copies.

  Sending larger data blocks will also result in more efficient use of the physical network because packets will typically be larger and less numerous. This option is much better than the first (enabling TCP_NODELAY) and should be used if at all possible.

REFERENCES

More information about Nagling and the Nagle algorithm can be found in RFC 1122.