Choosing Test Metrics

PT application testers could have detected and recorded a number of test metrics, such as — to judge stress on memory — page faults per second or bytes of memory available, or — to judge stress on the processor — hardware interrupts per second. Other metrics provide data more specifically oriented toward Web servers, such as the number of current connections, the maximum number of connections that the server has experienced during a test run, or the bytes sent and received by the server.

Note  Although the Windows NT Performance Monitor collected much data of these types (see Analyzing PerfMon Data), the testers studied it not to judge the functioning of the PT application itself but to learn more about the system environment in which the application ran during the test.

PT application testers compiled one metric for these tests: the transaction. They chose this metric for several reasons. First, a transaction corresponds to a typical user task, which includes keyboard and mouse actions and a certain amount of "sleep" time. For this reason, transactions can be perceived by users (see About Transactions) more easily than many other metrics. Most importantly, because transactions occur on multiple tiers, they represent aggregate system performance.

About Transactions

A transaction begins when a keystroke (typically pressing the ENTER key) or a mouse-click at the browser causes an action — at the browser itself, at IIS, or beyond at SQL Server — and something (often, data) is returned from that action. The transaction is considered complete when the action is completed, defined as the moment Microsoft Internet Explorer displays Done in the status bar.

Results of the tests on the PT application are published in transactions per second (TPS).