Bottlenecks Are Moving Targets

The other thing to keep in mind is that during processing, the bottleneck may shift around from one piece of equipment to another. Each second of operation might yield a different bottleneck if looked at in isolation. If you want to improve the situation as a whole, you'll need to look at the big picture. And even then the situation can be tricky. Let's take a look at a real example to illustrate these points.

The first thing we have to do is run the diskperf -y command to enable disk performance counters, as discussed at the end of Chapter 2. (You did read Chapter 2, didn't you?) After running diskperf, you'll have to reboot the computer being monitored before the disk performance counters can be activated.

Figure 3.1 Chart of processor and disk usage

In Figure 3.1 we have charted data from a Performance Monitor log file. The black line is the Logical Disk: % Disk Read Time, and the highlighted, white line is the Processor: % Processor Time. Activity is divided into two distinct phases. During the first phase, the disk is clearly the bottleneck, with the processor a somewhat distant second. During the second phase, the processor becomes the bottleneck, with the disk even less in use. The overall data is provided in Figure 3.2.

Figure 3.2 Report of processor and disk usage

This disk is utilized at 52.8% and the processor at 53.3%. Because the time window is 44 seconds, this means we used 23.23 seconds of disk time (this is demand[disk]) and 23.45 seconds of processor time (demand[processor]). In the formal terms of the last chapter, the maximum throughput at which the disk can accomplish this task is 1/23.23 * 3600 = 155.0 times per hour, and for the processor 153.5 times per hour. Technically, the processor is the bottleneck. In reality, both components are nearly equally to blame.

One way to think of this is to imagine how much faster this task would go if either component were infinitely fast. In this case, the activity would be accomplished in half the time if either component were blindingly fast. Is there a bottleneck? Yes, it is the processor. Could you improve performance by attacking either component? Yes. But you could only improve the first phase if the disk were improved, whereas in this case a faster processor would help the second phase a lot, and the first phase a little bit. Which brings us to Rule #3 of Bottleneck Detection.