Optimizing ASP Performance in Site Server 3.0 Commerce Edition

August 1999

Microsoft Corporation

Introduction

The equation for electronic commerce is simple: poor performance can cost you money. If your commerce server is operating at or near capacity and if the user's experience is being compromised by long response times, then you are probably turning away potential shoppers and penalizing yourself with missed sales opportunities.

For Site Server 3.0 Commerce Edition (SSCE), there are a number of ways of addressing  performance problems. You can:

These are all perfectly legitimate approaches to improving SSCE performance. A number of documents have been written to address SSCE performance from all three perspectives. This paper focuses on the issue of writing better Active Server Page (ASP) code. Better code, in this context, means code that is performance-sensitive; or, in other words, code that is written to address performance issues that are specific to the SSCE service.

There is, of course, always a trade-off. How much development effort needs to be directed towards performance? How much improvement can one reasonably expect by using the coding techniques described in this document? These are the kinds of questions that are of interest to site planners, project managers, and system administrators, so it is important to view ASP code optimization not only as a good technical practice, but also as a good business practice. For this reason, this document is split into two main sections: one devoted to specific ASP code techniques, and the other devoted to the impact of optimized ASP code on SSCE performance and capacity.

For those interested in learning more about SSCE capacity planning, tuning, and ASP code optimization issues, there are two documents worth reading:

Site Server 3.0 Commerce Edition

Site Server 3.0 Commerce Edition (SSCE) is a comprehensive Internet commerce server, optimized for Microsoft® Windows NT® Server and Microsoft® Internet Information Server (IIS), version 4.0. It enables businesses to cost-effectively engage in transactions with customers and business partners online. Site Server 3.0 Commerce Edition is designed to enable businesses to author new commerce sites, add commerce capabilities to existing Web sites, and to extend Intranet sites to select business partners. By integrating with IIS and Windows NT® Server, Site Server 3.0 Commerce Edition provides a secure, reliable, and comprehensive Internet commerce platform with limited need for customization.

There are two resources provided with SSCE to facilitate the creation of a commerce site: the Site Builder Wizard and the SSCE sample sites. The Site Builder Wizard provides a simple, step-by-step interface for designing and creating an electronic commerce site. Based on options you choose, the Site Builder Wizard creates the Active Server Page (ASP) files needed to create a fully functional commerce site. You can then customize your site using the coding examples provided in the SSCE sample sites.

Two sample sites are contained in the default installation of SSCE:

Sample Site Description
Clocktower A fictitious retailer of specialty clocks, an example of a basic SSCE site.
Volcano Coffee A fictitious retailer of products for coffee lovers, an example of a site providing more complex elements, such as shopper registration, product promotions, and more sophisticated navigation options.

This document compares different implementations of the Volcano Coffee site and demonstrates how effective ASP coding practices can greatly improve SSCE server performance.

SSCE Code Optimization

An SSCE site is comprised of a series of transactions, or shopper operations. Every shopper who logs into the site performs a certain number of shopper operations before logging out of the store. Each operation is comprised of one or more ASPs. Therefore, ASP requests can be viewed as a reflection of shopper activity. As more shoppers log into the site, the rate of ASP requests increases.

Because ASP processing is CPU-intensive, the number of ASPs that can be processed at one time is limited by available CPU resources. As more shoppers log into an SSCE site, CPU utilization rises until available resources are effectively depleted, at which point the ASP processing rate stops increasing. In Figure 1, peak ASP throughput is reached at approximately 28.5 ASP requests per second.

Figure 1.  ASP Throughput

Another way to view SSCE performance is in terms of ASP latency. As more shoppers log into an SSCE site, latency (or response time) gradually increases. Shopper capacity is reached when ASP latency is greater than one second.

In Figure 2, latency remains fairly stable for the first 500 shoppers. Then ASP performance starts to degenerate, as shown by the increase in latency. With 800 shoppers connected, latency is well beyond the one-second threshold, indicating that capacity has been exceeded.

Figure 2.  Latency

If ASP latency can be improved, more ASP requests can be processed within a given period of time, and more shoppers can be supported on an SSCE site. The goal of ASP code optimization is to minimize ASP processing time, thus increasing ASP throughput, which ultimately translates into increased shopper capacity.

Optimizing Performance in Volcano Coffee

Following the release of Site Server 3.0 Commerce Edition, a new version of Volcano Coffee was developed to address ASP performance issues related to this sample site. This new version, called VC-Turbo, was released to the Web as the Volcano Coffee Sample Site and Performance Kit. (The Volcano Coffee Sample Site and Performance Kit can be found at http://www.microsoft.com/siteserver/commerce/DeployAdmin/VolcanoCoffee.htm.) VC-Turbo offered numerous approaches to ASP code optimization, which resulted in significant performance gains over its predecessor (referred to as VC30). (The performance improvements are discussed in detail in the Performance white paper included in the Volcano Coffee Sample Site and Performance Kit.) Those performance improvements can be summarized as follows:

Pipeline design Splitting the plan.pcf pipeline into two separate pipelines (basket.pcf and total.pcf) to eliminate unnecessary overhead resulting from processing stages not required for the specific context.
Site design Removing costly navigational buttons and making promotions (cross-sell and up-sell) less intrusive.
Database schema Eliminating use of the DBStorage object in favor of a more traditional database design in which database tables are directly manipulated and queried from the ASP page, thus reducing overhead caused by an unfortunate interaction between the DBStorage object and the Microsoft Transaction Server.
Database queries Avoiding use of “select *” statements, using multiple joins to reduce the number of query hits, and eliminating the use of a query map by pipeline components.
Object creation Reducing the need to create Command and recordset objects as well as using the <Object> tag to eliminate the need to make calls to Server.CreateObject.
URL generation Using the new pageURL function to optimize generation of URLs.

A second revision to the Volcano Coffee Sample Site, referred to as VC-Rocket, is the subject of this paper. Although the additional performance enhancements offered by VC-Rocket are not nearly as extensive in terms of detail and variety, the performance results are every bit as impressive.

The fundamental concept behind VC-Rocket is simple enough. Most of the operations performed in the Volcano Coffee store require access to an SQL database. Because SQL database calls greatly increase ASP latency, reducing the number of calls allows ASPs to be processed more quickly.

If the ASPs used for shopper operations are stored in cache, SQL calls can be avoided altogether. This is an effective practice as long as the SQL data remains relatively static. If the SQL data changes often, ASP caching is impractical. But for those ASPs that query data that changes very infrequently, caching can greatly enhance performance.

Overview of ASP Caching Methodology

VC-Rocket caches ASPs that perform database queries, whenever practical. For example, there are specific Volcano Coffee shopper operations that query data that can remain static for extended periods of time (days, weeks, or even months). VC-Rocket optimizes these operations by storing the results of the database queries in cache, along with the rendered ASP code.

The first time a page is requested, the cache is checked to see if the requested page exists. If the page is not found, the ASP code is rendered in static HTML and stored in cache. The cached page is then sent to the browser requesting the page. The next time the page is requested, the cache is searched for a copy of the page. If found, the page is retrieved from cache and sent to the browser. Figure 3 shows how this works.

Figure 3.  Requesting a Page to View in VC-Rocket

The ASP Caching Process

Caching an ASP is a two-step process:

  1. ASP code is rendered into one or more static HTML fragments.

  2. The rendered fragments, referred to as Views, are stored in cache.

The ASP rendering function is performed by an instance of the Commerce PageGen object. The PageGen object accepts as input a template based on the original ASP file. The resulting output, a static HTML fragment referred to as a View, is stored in an instance of the Commerce Dictionary object called ViewManager, which performs the function of caching the View. Figure 4 shows how ASP code rendering precedes storing the rendered View in the ViewManager cache.

Figure 4.  Caching an ASP Request

Note that a single ASP request may generate multiple HTML fragments. Each of the fragments is rendered and cached separately. The original ASP becomes a container for the processing of multiple HTML fragments. So, from a broader perspective, the ASP caching process can be visualized as shown in Figure 5, with an ASP “shell” processing multiple PageGen templates, each template being rendered as an HTML fragment stored in ViewManager as a distinct View.

Figure 5. Rendering and Caching HTML Fragments

Aging Cached Views

Cached Views are accessed within ViewManager using keys. The key is unique to the View being requested. If the key cannot be located in ViewManager, the requested View has not yet been stored in cache. If the View is found in cache, the age of the View is determined, as shown in Figure 6. Cached Views need to be refreshed after a period of time to reflect the anticipated rate of change of data in the Volcano Coffee database. If the age of the cached View exceeds some predetermined time limit, the View needs to be updated.

Figure 6. Searching ViewManager for a Recent Copy of a Requested View

ASP Design and Coding Considerations

To facilitate ASP caching in Volcano Coffee, the ASP is redesigned to incorporate the rendering and the caching functions. This requires a new design based on three file components (Figure 7):

The ASP “shell” passes the template files as arguments to the RenderView function, which, in turn, generates and caches the rendered HTML fragment.

Figure 7. The Three File Components of a Cacheable ASP

Implementing the Design

In the example shown in Figure 8, ASP caching uses the Listing.asp file, which is included with the Volcano Coffee sample site. Listing.asp queries the product table to create a listing of all the products in the Volcano Coffee store. Although the variety of products in the store changes over time, the frequency of change is relatively low, making this ASP page a good candidate for caching.

Using the design illustrated in Figure 7, Figure 8 shows Listing.asp split into a “shell” file (called Listing2.asp) and two template files (called Header2.asv and Listing2.asv). The Listing2.asp file is used to perform the RenderView function calls, passing the Header2.asv and Listing2.asv templates as arguments to the RenderView function located in ViewUtils.asp.

Figure 8. Design for a Listing.asp

Retrieving a View from Cache

When the Listing2.asp file makes a call to the RenderView function, it actually passes four arguments:

<% =RenderView(MSCSViewManager, "list", "listing2.asv", MSCSPage.URLShopperArgs()) %>

The four arguments in this example are identified as follows:

MSCSViewManager
The global caching object defined in global.asa
"list"
A key to uniquely identify the View in cache
"listing2.asv"
The URL of the PageGen template file
MSCSPage.URLShopperArgs()
The Shopper ID

In the following code section, RenderView receives the four arguments passed from Listing2.asp and checks ViewManager (using “list” as the specific key for this page) to see if the requested View is in cache. If the requested View cannot be found, the GenView routine is called and a new View is created:

Function RenderView(oViewMgr, sKey, sURL, sShopperArg)
    Dim oView

If Not isObject(oViewMgr.Value(sKey)) Then
    Set oView = Server.CreateObject("Commerce.Dictionary")
    On Error Resume Next
    Call GenView(oView, sURL)
    On Error Goto 0

Generating a View

The GenView routine is called by the RenderView function to generate a new View to store in cache. GenView accepts two arguments:

View Temporarily stores the newly rendered View
URL Pointer to the PageGen template file

GenView uses an instance of the Commerce PageGen object to render the View.

But first, a full server-side path/file name must be generated from the URL pointer. The contents of the file are then read into the Input variable and passed to the PageGen.GenpageText method, which returns the rendered View to the Output variable. Data in the Output variable is then stored in the View object and returned as an argument to the RenderView function:

Sub GenView(oView, sURL)
    Dim oPageGen, oCtx, sSrcFile, oAdminFiles, sInput, sOutput
    sSrcFile = Server.MapPath("/" & mscsPage.SiteRoot() & "/" & sURL)
    Set oAdminFiles = Server.CreateObject("Commerce.AdminFiles")
    sInput = oAdminFiles.ReadFromFile(sSrcFile)
    
    Set oPageGen = Server.CreateObject("Commerce.PageGen")

    . . .

    sOutput = oPageGen.GenPageText(sInput, False, "Ctx", oCtx, "VBScript")

    . . .

    oView.Data = sOutput
    oView.dateGenerated = Now
    oView.Size = Len(sOutput)

    . . .

End Sub

Using the PageGen Object

When using PageGen, there are two issues to consider with regards to processing of PageGen templates. First, PageGen cannot process ASP objects referenced by the template. The solution is to store data from the ASP objects in an intermediary Context object and pass the Context object to PageGen (Figure 9). This is illustrated in the following code example:

Sub GenView(oView, sURL)
    Dim oPageGen, oCtx, sSrcFile, oAdminFiles, sInput, sOutput

    . . .

Set oPageGen = Server.CreateObject("Commerce.PageGen")
    Set oCtx = Server.CreateObject("Commerce.Dictionary")
    Set oCtx.mscsPage = mscsPage
    Set oCtx.MSCSSite = MSCSSite
    Set oCtx.Request = Request
    Set oCtx.Response = Response
    Set oCtx.MSCSDataFunctions = MSCSDataFunctions
    oCtx.alt = alt
    oCtx.hd_image = hd_image

    . . .

    sOutput = oPageGen.GenPageText(sInput, False, "Ctx", oCtx, "VBScript")

Figure 9. Passing ASP Objects with the Template File to the PageGen.GenPageText Method

The Context objects are passed to PageGen to be used by the Listing2.asp template file as shown in the following example:

<%% 
REM - extract ctx objects:
Dim mscsPage, mscsSite, Request, Response, MSCSDataFunctions
set mscsPage = ctx.mscsPage
set mscsSite = ctx.mscsSite
set Request = ctx.Request
set Response = ctx.Response
set MSCSDataFunctions = ctx.MSCSDataFunctions

It is also important to recognize that the PageGen object cannot handle Include statements. In the original Listing.asp file, there are three Include statements that reference external files, as shown in Figure 10.

Figure 10. Listing.asp with Include Files

In order to work around this problem, any files included by the original ASP must either be incorporated directly into the template file or processed separately. In the solution chosen for this ASP, Shop.asp is incorporated directly into the Listing.asv template file, Header.asp is treated as a separate template, and Footer.asp is retained (unrendered and uncached) as an included file in Listing2.asp, as illustrated in Figure 11.

Figure 11. Listing2.asp with Include Files

This solution is illustrated in the following code, which is taken from the Listing2.asp file:

<% =RenderView(MSCSViewManager, "header", "include/header2.asv", MSCSPage.URLShopperArgs()) %>
<% =RenderView(MSCSViewManager, "list", "listing2.asv", MSCSPage.URLShopperArgs()) %>
<!--#INCLUDE FILE = "include/footer.asp" -->

Caching Views with ViewManager

Central to the design of VC-Rocket is the concept of the ViewManager, which is an instance of the Commerce Dictionary object. Once a PageGen template string has been rendered as static HTML, this View is stored in ViewManager. As described earlier, ViewManager stores a collection of Views, which are referenced by unique keys (Figure 12).

When Listing2.asp calls the RenderView function, RenderView searches ViewManager for the requested View, based on the specific key. In the following code example, the header View is retrieved using “header” as the key, and the listing View is retrieved using “list” as the key:

<% =RenderView(MSCSViewManager, "header", "include/header2.asv", MSCSPage.URLShopperArgs()) %>
<% =RenderView(MSCSViewManager, "list", "listing2.asv", MSCSPage.URLShopperArgs()) %>

Figure 12. Views Created for Listing2.asp and Stored in ViewManager

In addition to caching Views, ViewManager has certain properties and counters that are initialized in the global.asa file, as shown in the following code example:

Function InitViewManager
            ' Set parameters:
            '   timeRetain = Number seconds to retain the page before recaching
            '   nRequests  = Total number of pages requested
            '   nCached    = Number of requests satisfied from cached pages
            '   nUncached  = Number of requests satisfied from uncached pages
            '   nErrors    = Number of errors encountered
            '   nMaxPages  = Max number of pages to retain in the cache
            MSCSViewManager.timeRetain = 60*30
            MSCSViewManager.nRequests = 0
            MSCSViewManager.nCached = 0
            MSCSViewManager.nUncached = 0
            MSCSViewManager.nErrors = 0
            MSCSViewManager.nMaxPages = 5
            Set InitViewManager = MSCSViewManager
        End Function

The values for timeRetain and nMaxPages control how the cache is used. The timeRetain value is used by the RenderView function to control the refresh frequency for cached pages. The nMaxPages value can be used to limit the number of pages stored in cache. In the CleanViewManager routine found in ViewUtils.asp, Views are deleted from ViewManager until the total number of cached pages is less than the maximum allowed by the nMaxPages value. (Oldest Views are deleted first.)

The ViewManager counters initialized in global.asa can display ViewManager statistics. In the DebugViewManager routine (also found in ViewUtils.asp), page request counts (cached and uncached) and error counts can be displayed in conjunction with a list of Views currently found in ViewManager.

Summary

Caching ASP pages that perform database queries on relatively static data is an effective way to optimize ASP performance for a Site Server 3.0 Commerce Edition site. Caching ASP pages is a two-step process:

Each View stored in the cache is identified by a unique key, which can be used to retrieve the View with a subsequent ASP request. Cached Views need to be refreshed after a given period of time to ensure that cached data stays current with data stored in the database.

Rendering and caching is performed by a function called RenderView, which is found in ViewUtils.asp. The RenderView function accepts as data a PageGen template, which is a file containing most of the original ASP code. The original ASP file is actually split into two separate components: a “shell” file, which performs the RenderView function calls, and one or more PageGen template files. Each template is rendered separately, then cached as a separate View in ViewManager.

The actual rendering of ASP code into Views is performed by an instance of the Commerce PageGen object. When using PageGen, there are two issues to consider:

Volcano Coffee Performance and Capacity

This section compares performance and capacity measurements for the three versions of the Volcano Coffee sample site (VC30, VC-Turbo, and VC-Rocket). Performance is measured in terms of ASP throughput, or maximum ASPs processed per second. Capacity is measured in terms of shopper load, or maximum number of concurrent shoppers supported with a response time of less than one second.

VC-Rocket is the second performance enhancement to the Volcano Coffee sample site shipped with Site Server 3.0 Commerce Edition. Each version is an enhancement of the earlier site, but requires that the earlier version already be installed before it can be used.  These three versions can be compared as follows:

Version Features
VC-30 The original site, containing examples of shopper registration, price, cross-sell and up-sell promotions, search, buy-now, navigation buttons, and manager pages.
VC-Turbo Performance optimizations in site design, pipeline design, database schema, database queries, object creation, and URL generation.
VC-Rocket Performance optimizations resulting from pre-rendering listing.asp and product.asp pages.

For each Volcano Coffee site, we tested shopper operations using the InetMonitor test tool. (For a description of InetMonitor, please refer to the The InetMonitor 3.0 Performance Testing Tool in the Appendix.) We measured maximum ASP throughput for each shopper operation by running a sequence of tests using InetMonitor test scripts. Comparing ASP throughput measurements for each sample site is a good way to demonstrate how ASP code optimization directly impacts ASP performance.

However, since ASP performance is a pretty abstract concept, a better measure for SSCE might be shopper capacity. A methodology known as Transaction Cost Analysis (TCA) brings the cost/benefit ratio of optimizing ASP performance at an SSCE site into clear focus. Shopper capacity can be difficult to ascertain. For one thing, ASP performance does not directly correlate with shopper capacity. A 50-percent increase in ASP performance does not necessarily indicate that your SSCE site can support 50 percent more shoppers. For another thing, shopper capacity is dependent on shopper behavior, which can vary from site to site. Transaction Cost Analysis can accurately determine shopper capacity.

For a greater understanding of how to use Transaction Cost Analysis to evaluate performance and capacity for an SSCE site, please refer to Using Transaction Cost Analysis for Site Capacity Planning, available in the Site Server 3.0 Commerce Edition Resource Kit.

The Volcano Coffee Sample Site

You can evaluate the performance of Volcano Coffee in terms of the individual shopper operations that make up the site. These shopper operations and the ASP pages associated with each operation are shown in Table 1. Test scripts are created to simulate each operation, using the InetMonitor test tool.

Table 1: Shopper Operations in the Volcano Coffee Store

VC Shopper Operation Description ASP Pages Used
Add item Add item to shopping basket xt_orderform_additem.asp, basket.asp
Basket View contents of basket basket.asp
Check out Purchase items in basket checkout-ship.asp, xt_orderform_prepare.asp, checkout-pay.asp, xt_orderform_purchase.asp, confirmed.asp
Clear items Empty shopping basket xt_orderform_clearitems.asp, basket.asp
Default VC home page default.asp
Delete item Remove item from basket xt_orderform_edititem.asp, xt_orderform_delitem.asp, basket.asp
Listing Show all items in store (product listing) listing.asp
Lookup Authenticate returning shopper xt_shopper_lookup.asp, home.asp
New Register a new shopper xt_shopper_new.asp, home.asp
Product Browse product information product.asp
Search Search for products search.asp, srchresult.asp
Welcome new user or Welcome return user Confirm entry into store welcome_new.asp or welcome_lookup.asp

ASP Throughput Comparisons

ASP throughput is a measure of how many ASP requests can be processed per second. Using the ASP Requests/second counter in Performance Monitor (PerfMon), we measured optimum throughput for each VC shopper operation.

Note   Optimum performance is measured as the maximum ASP throughput with context switching not exceeding 15,000/sec. Beyond this threshold, previous testing has shown that CPU efficiency is significantly diminished. In Transaction Cost Analysis, optimum throughput is more useful a measure than maximum throughput.

Note that certain operations are actually shown in combinations, such as Additem+Checkout, and Additem+Clearitems. In order to test the Checkout and Clearitems operations, it is necessary to prepare the shopping basket by adding product items to the basket (Additem). This is not out of line with a real shopping experience in which, for example, a shopper must first add a product to the shopping basket before purchasing the product.

Figure 13 shows a significant performance improvement in the Listing operation (for VC-Rocket), which specifically takes advantage of the ASP caching technique described in this paper.

Figure 13: Comparing ASP Performance for VC30, VC-Turbo, and VC-Rocket

Transaction Cost

Transaction cost is used to calculate shopper capacity. It measures the number of CPU cycles required to process a single operation. Using the ASP Requests/second and %Total processor counters in PerfMon, you can calculate transaction cost for each VC shopper operation. (See the Transaction Cost Table for VC-Rocket in the Appendix for a description of the process for calculating transaction cost.) Note that there is a significant improvement in CPU resource usage for the Listing operation (shorter bars in Figure 14 indicate better performance). Chart measurements for the Add+Checkout operation have been reduced by a factor of ten (for all sites) to fit into scale.

Figure 14: CPU Cost by ASP for VC30, VC-Turbo, and VC-Rocket (lower is better)

Shopper Profile

Shopper behavior can vary dramatically from site to site, depending on any number of factors, including the type of product being sold, the number of products available in the store, and the quality of the shopping experience.

In conjunction with transaction cost, shopper behavior defines shopper capacity. Not only do costly shopper operations (such as Checkout) impact shopper capacity, but the frequency with which operations are performed also has a major impact. In these shopper profiles, Listing and Product are the most commonly used operations, which underlines the value in optimizing ASP code for these operations.

A commonly used and simplified approach to defining shopper behavior is to determine the browse-to-buy ratio, or purchasing percentage (for example, a browse-to-buy ratio of 50:1 means that 2.0 percent of all shoppers visiting this site actually make a purchase). Using this as a guideline, three different shopper profiles are suggested, based on different percentages of purchases. Table 2 illustrates the shopper profiles used in this report.

Table 2: Shopper Profiles Used in this Report

VC Shopper Profile

Shopper Operations

Low purchase = 1.0%

Transactions per 8-minute session

Mid purchase = 2.0%

Transactions per 12-minute session

High purchase = 5.0%

Transactions per 16-minute session

Additem 0.170 0.240 0.450
Add+Checkout 0.010 0.020 0.050
Add+Clearitems 0.020 0.040 0.100
Basket 0.400 0.750 1.000
Default 1.000 1.000 1.000
Listing 1.200 2.500 3.500
Lookup 0.750 0.750 0.750
New 0.250 0.250 0.250
Product 2.500 4.200 5.500
Search 0.700 1.250 2.400
Welcome 1.000 1.000 1.000
Total operations per session 8.00 12.00 16.00

Shopper Capacity

Figures 15, 16, and 17 show predicted shopper capacity for each sample site, using the three shopper profiles defined in Table 2. Each figure clearly illustrates the performance benefits from ASP caching. In each case, VC-Rocket provides nearly double the capacity of either the VC-Turbo or VC30 sample sites.

Figure 15: Calculated Shopper Capacity for VC30, VC-Turbo, and VC-Rocket (session = 8 minutes and purchase = 1.0%)

Figure 16: Calculated Shopper Capacity for VC30, VC-Turbo, and VC-Rocket (session = 12 minutes and purchase = 2.0%)

Figure 17: Calculated Shopper Capacity for VC30, VC-Turbo, and VC-Rocket (session = 16.0 minutes and purchase = 5.0%)

Appendix

System Configuration

All testing for VC30, VC-Turbo, and VC-Rocket was performed using the following system configuration.

SSCE server CPU: 4 x 400-MHz Pentium II (Xeon) w/512KB L2 cache
  Memory: 2 GB ECC buffered EDO RAM
Disk: 3 x 18.2 GB SCSI LVD (10,000 RPM)
Network: 100 MB switched Ethernet
Software: Windows NT® Server 4.0 with Service Pack 5, Internet Information Server (IIS) 4.0, Site Server 3.0 Commerce Edition (SSCE) with Service Pack 2, MDAC 2.1 Service Pack 1
SQL Server CPU: 2 x 400-MHz Pentium II w/256KB L2 cache
  Memory: 256MB ECC RAM
Disk: 1 x 8 GB SCSI LVD & 1 x 16 GB SCSI LVD (10,000 RPM)
Network: 100 MB switched Ethernet
Software: Windows NT® Server 4.0 with Service Pack 4 and Microsoft SQL Server 7.0

The InetMonitor 3.0 Performance Testing Tool

InetMonitor 3.0 is a performance-testing tool designed to simulate user load for a specific Internet service. InetMonitor can show how a particular system configuration will respond under different loads prior to implementing the system in a production environment. When you use InetMonitor in conjunction with the PerfMon performance monitor, you can monitor the transaction rate for the service to determine maximum throughput and maximum user capacity.

Note   PerfMon is part of the Windows NT® 4 operating system.

You can also monitor resource utilization to identify hardware bottlenecks. If, for example, you find that CPU utilization for a particular configuration is consistently operating above 90 percent on a single-processor Web server, using a faster processor on the Web server or adding a second Web server would be a reasonable approach to removing the CPU bottleneck.

InetMonitor uses a scripting language to simulate specific types of transactions. If you want to test Volcano Coffee, you can create scripts to simulate any logical combination of VC shopper requests performed by a Web browser. For example, you could create a script to register a new shopper in the store, view the store directory, browse products, add a product to the shopping basket, and check out of the store.

With InetMonitor, you can also control user load (how many shoppers are concurrently connected to your site). Thus, you could, for example, test a specific InetMonitor script for different numbers of concurrent shoppers, using PerfMon to evaluate the behavior of the service for changing load conditions.

For more information about InetMonitor, refer to the InetMonitor Operations Guide, Version 3, located in the Site Server 3.0 Commerce Edition Resource Kit.

Critical PerfMon Counters

All of the counters listed in this section can be found in the PerfMon performance monitor. You can use the ASP counters listed in this section to identify maximum throughput (requests/sec) and latency (request execution time + request wait time).

On dual-processor and quad-processor configurations, context switching can be problematic and should be monitored. Geometric increases in context switching are the rule, and this ultimately determines maximum ASP throughput for most shopper operations. More context switches require a greater percentage of processor resources.

ASP Object

Requests executing

Requests execution time

Requests wait time

Requests queued

Requests/sec

System Object

Context switches/sec

%Total Processor

Transaction Cost Table for VC-Rocket

Table 3 illustrates how transaction costs are calculated for each VC-Rocket operation. To build a transaction cost table, you must:

  1. Determine the number of ASPs used to perform each shopper operation.

  2. Measure maximum ASP throughput and %CPU for each operation.

  3. Calculate CPU cost per ASP and CPU cost per operation from measurements for maximum ASP throughput and %CPU, as follows:

Cost per ASP=CPU utilization (MHz) ÷ Maximum ASP Requests/sec

Cost per operation=Cost per ASP × total ASPs per operation

Transaction cost is measured in terms of Mcycles, which is equivalent to millions of CPU cycles. If, using the Additem operation as an example, %CPU is measured at 62.95 percent on a 4 x 400 MHz Pentium II computer, this equates to a CPU utilization of 1007 Mcycles/sec (1600MHz × 62.95%). Since maximum ASP throughput is 35.35 requests per second, CPU cost is calculated as follows:

Cost per ASP=1007 MHz ÷ 35.35 ASP requests/sec=28.49 Mcycles

Cost per operation=28.49 Mcycles × 2 ASPs per operation=56.98 Mcycles

Table 3: Transaction Costs for Each Shopper Operation in VC-Rocket

VC Shopper Operation gets posts re-directs total Optimum ASP Req/sec % CPU CPU cost per ASP CPU cost per operation
Additem 0 1 1 2 39.18 72.07% 29.43 58.86
Add+Checkout 1 3 3 7 14.54 55.30% 60.84 425.88
Add+Clearitems 0 2 2 4 36.17 71.36% 31.56 126.25
Basket 1 0 0 1 10.66 67.09% 100.73 100.73
Default 1 0 0 1 176.22 95.17% 8.64 8.64
Listing 1 0 0 1 82.45 66.87% 12.98 12.98
Lookup 0 1 1 2 142.99 98.20% 10.99 21.98
New 0 1 1 2 124.13 83.82% 10.80 21.61
Product 0 0 1 1 137.76 91.58% 10.64 10.64
Search 1 1 0 2 124.56 90.26% 11.59 23.19
Welcome 1 0 0 1 177.21 93.80% 8.47 8.47

Information in this document, including URL and other Internet web site references, is subject to change without notice.  The example companies, organizations, products, people and events depicted herein are fictitious.  No association with any real company, organization, product, person or event is intended or should be inferred.  Complying with all applicable copyright laws is the responsibility of the user.  Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation.

Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document.  Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.

© 1999-2000 Microsoft Corporation.  All rights reserved.

Microsoft, MS-DOS, MS, Windows, and Windows NT are either registered trademarks or trademarks of Microsoft Corporation in the U.S.A. and/or other countries/regions.

The names of actual companies and products mentioned herein may be the trademarks of their respective owners.

Additional Disclaimer

NOTICE REGARDING INFORMATION AND SERVICES AVAILABLE IN THIS DOCUMENT. In no event shall Microsoft, and/or its respective suppliers be liable for any special, indirect or consequential damages or any damages whatsoever resulting from loss of use, data, or profits, whether in action of contract, negligence, or other tortious action, arising out of or in connection with the use or performance of any guidelines, documents, provision of or failure to provide services, or information available from this document.