Using Transaction Cost Analysis for Site Capacity Planning

August 1999

Microsoft Corporation

Methodology

The purpose of capacity planning for Internet services is to support transaction throughput targets with acceptable response times, while minimizing the total dollar cost of ownership of the host platform. Conventional solutions often attempt to evaluate the cost of Internet services by extrapolating from generic benchmark measurements. However, a more effective methodology is based on Transaction Cost Analysis (TCA).

You can use TCA methodology to simulate client transactions on the host server by a load-generation tool that supports standard network protocols. By varying client load, transaction rate can be related to resource utilization. A usage profile designed to capture anticipated user behavior can then be defined. The usage profile determines the throughput target and other important transaction parameters from which resource utilization and capacity requirements can then be calculated.

TCA methodology can be used to measure the cost of individual shopper operations, such as browsing, adding an item to the shopping cart, checking out, searching, registering, and so on. The capacity of a site is determined by dividing the costs of shopper operations into the total CPU capacity available for the server.

Once you understand the cost and relative frequency of each shopper operation, you can view the performance numbers for a site’s pages to see where the bottlenecks are, and to see where you can optimize performance to improve the site’s capacity.

For example, TCA can help you determine which operations are the most costly, which pages are accessed the most frequently, and so forth. TCA can also help you do "what if" scenarios with the performance data to see how performance improvements (or degradation) affect your site. Once you know the cost of each operation, you can target those pages with high cost or high frequency of use for performance optimization. If you need additional capacity, identifying and lowering the cost of expensive pages will help. In addition, you’ll be able to clearly see the impact of adding features to a page and can plan to install additional hardware to service the load generated by additional shoppers or new code releases.

A Transaction Cost Analysis includes the following components:

Set Up a User Profile

The first step to running a Transaction Cost Analysis is to compile a user profile by analyzing customer Internet Information Server (IIS) logs. It’s best to use as long a period of time as possible (at least a week’s worth of IIS logs, if they are available), in order to obtain good averages. You can use the Site Server 3.0 Commerce Edition Usage Analysis tool (Report Writer Catalog) to gather the following types of numbers:

The Detail User Report and the Detail Visit Report provide this information. The number of visits to each page is used to profile typical shopper operations for the site. This information comprises the user profile for the analysis. The following example  shows a user profile for a sample site:

Shopper Operations Page Hits Ratio Of Hits
Default (Homepage) 58285 22.82%
Search (Good) 36641 14.35%
Search (Bad) 2599 1.02%
Add Item 4482 1.76%
Add Item + Delete 2726 1.07%
Add Item + Checkout 2800 1.10%
Browse 93507 36.61%
Login 4418 1.73%
Register 2700 1.06%
Zip Code 40771 15.96%
View Cart 6452 2.53%
Totals 255381 100.00%

Although the numbers in the example table account for only 10% of the pages (top page requests), they account for 90% of the traffic to this site. Other traffic numbers (including image and miscellaneous requests that generate a much lighter load) are not included in shopper operations in order to simplify the example.

Another important aspect of setting up a user profile is to determine the rate at which transactions are taking place. You use these numbers to create scripts for loading the sample site to verify the calculated capacity. For example, transaction rates at our sample site were:

Average session length 10 minutes
Checkout transaction rate/minute Checkout transaction rate/minute is equal to the number of transactions/number of visitors/time. At our sample site, there was 1 transaction/31 visitors/10 minutes, or 0.003 transactions/visitor/minute.

If you include non-cookie visitors, the transaction rate was approximately 1 transaction/40 visitors/10 minutes, or 0.0025 transactions/visitor/minute.

Browse-to-buy ratio Approximately 40:1 (for every 40 visitors browsing, 1 visitor performs a checkout/buy transaction). This coincides with a Forrester Report (The Look to Buy Imperative, 4/98).
Concurrent users 40

Measure the Cost of Each Operation

Once you have established a user profile, you can measure cost by creating a load-monitoring script to exercise each of the identified shopper operations. You then measure CPU utilization at that load level. InetMonitor, a load generation/simulation tool provided with the Microsoft BackOffice Resource Kit, is a good tool to use for this task. A set of sample scripts for measuring the cost of various shopper operations is provided in the section on Cost Measurement Scripts.

The objective of running a script exclusively for an individual operation is to load the IIS/ASP server with as many requests as possible, in order to achieve maximum ASP throughput per second. Maximum ASP throughput occurs when you measure a drop in ASP throughput with a higher shopper load or when you measure a sudden increase in operation latency or ASP requests queued. You can calculate operation latency using the following formula:

Average ASP latency (ASP execution time + ASP wait time) * number of ASP pages

Since it is difficult to determine averages before running the scripts, use an average ASP latency of less than 2 seconds, to account for additional download latency of modem users.

CPU utilization is recorded using the Microsoft® Windows NT® Performance Monitor when you reach maximum ASP requests/second (Active Server Pages Object—ASP requests/second, and the System Object—% Total Processor Time).

Increasing the number of users running the script increases the load. Typically, ASP requests/second and CPU utilization grow with the number of users. However, if adding more users results in lower ASP requests/second throughput, the previous step achieved the maximum throughput. If the number of ASP requests/second continues to grow until CPU utilization reaches 100%, the number of ASP requests/second at that point is then the maximum.

The result of TCA measurement is the CPU cost for each shopper operation, calculated on the basis of Megacycles (Mcycles) or 1Mhz. For example, a dual processor 400Mhz Xeon Pentium II has a total capacity of 800 Mcycles. Using the maximum number of ASP requests/second, you can calculate the cost per ASP request, as follows:

CPU utilization * number of CPUs * speed of CPU (in MHz) / ASP requests per second

For example, if you are using a server with two CPUs, and if the browse operation results in 11.50 ASP requests/second with CPU utilization of 84.10%, the cost per ASP page is then 84.10% * 2 * 400 / 11.50 = 58.50 Mcycles.

However, before you can calculate the cost of an operation, you also need to know the number of ASP pages involved in a shopper operation. For example, checkout operations typically involve several ASP pages (shopper information page, credit card page, shipping page, confirmation page, and so on). You calculate the cost of a shopper operation by multiplying the number of ASP pages involved by the cost per ASP page. Some ASP pages, such as action pages, are never seen, since they are usually posted to and redirected to a continuing page. You must also account for hidden ASP pages.

In the browse operation example, there is only one ASP page. Thus, the cost of the browse operation is 58.50 Mcycles. If there were two ASP pages, the cost would have been 58.50 * 2 = 117 Mcycles, and so on.

TCA measurement results in a set of CPU costs for each shopper operation. In the following table, the number of CPUs is two Xeon Pentium II processors. The clock speed of each CPU is 400MHz.

Shopper Operations CPU Utilization @ Maximum ASP Requests per Second Maximum ASP Requests per Second Cost (Mcycles) Cost per ASP page Number of ASP pages Cost per Operation
Default (Homepage) 57.80% 8.60 462.4 53.77 1.00 53.77
Search (Good) 63.80% 3.40 510.4 150.12 1.00 150.12
Search (Bad) 61.80% 23.50 494.4 21.04 2.00 42.08
Add Item 85.00% 5.20 680.0 130.77 2.00 261.54
Add Item + Delete 80.90% 7.20 647.2 89.89 4.00 359.56
Add Item + Checkout 30.40% 4.10 243.2 59.32 7.00 415.24
Browse 84.10% 11.50 672.8 58.50 1.00 58.50
Login 81.80% 7.80 654.4 83.90 3.00 251.69
Register 81.50% 20.90 652.0 31.20 2.00 62.39
Zip Code 82.80% 18.90 662.4 35.05 2.00 70.10
View Cart 67.60% 6.40 540.8 84.50 1.00 84.50

Estimate Capacity

Estimating capacity requires some assumptions about the user profile. Typical shopper operations are those of an average user. The behavior of actual shoppers can cause random peaks and valleys; however, over time these peaks and valleys statistically even out to average behavior. The user profile you create reflects average user behavior. In order to estimate capacity, you need to assume an average user.

In the following example, there are 11 shopper operations and transactions that occur 0.025 times every 10 minutes. To simplify the example, we created the user profile script to perform the 11 shopper operations in 11 minutes. This generated a ratio of operations for use in capacity calculations and for a verification script.

Although the sample site has a transaction ratio of 0.0025 checkouts/ user/ minute (1 checkout/40 users/10 minutes), we used 0.01 checkouts/ user/ minute (1 checkout/10 users/10 minutes = 0.01, or 0.1 checkouts per 10 minutes), to reflect a heavier load. This number is based on a 10:1 browse-to-buy ratio, instead of the actual 30:1 (or 40:1 when non-cookie shoppers are included) browse-to-buy ratio.

Although we used a heavier transaction rate (four times more checkouts), we kept the browse-to-buy ratio at 40:1 to anticipate future traffic at a higher volume (with the same page ratio mix) and to facilitate the simple conversion of capacity as number of concurrent users for the sample site. In this case, an estimated capacity of one user would translate to four users on the production/live sample site.

We then calculated the number of shopper operations during the course of 11 minutes, and, following that, the number of shopper operations per second. We need to know the number of shopper operations per second because the total shopper operation cost number is expressed in terms of clock speed per second (400MHz CPU speed is 400 Mcycles per second).

The cost table for the user profile is as follows:

Operations Ratio Of Hits User Profile Operations (over 11 minutes, 660 seconds) User Profile Operations per Second

(User Profile Operations / 660 seconds)

Cost per Operation (Mcycles) Cost per User Operation per Second

(Cost per Operation * User Profile Operations per Second)

Default (Homepage) 22.82% 2.5 0.003804 53.77 0.2045
Search (Good) 14.35% 1.6 0.002391 150.12 0.3590
Search (Bad) 1.02% 0.1 0.000170 42.08 0.0071
Add Item 1.76% 0.2 0.000293 261.54 0.0765
Add Item + Delete 1.07% 0.1 0.000178 359.56 0.0640
Add Item + Checkout 1.10% 0.1 0.000183 415.22 0.0759
Browse 36.61% 4.0 0.006102 58.50 0.3570
Login 1.73% 0.2 0.000288 251.69 0.0726
Register 1.06% 0.1 0.000176 62.39 0.0110
Zip Code 15.96% 1.8 0.002661 70.10 0.1865
View Cart 2.53% 0.3 0.000421 84.50 0.0356
Total   11.0     1.4496

The interesting number here is the total (1.4496), which indicates that the cost of the total user profile is 1.4496 Mcycles per user. This number reflects the cost of an average user executing shopper operations described by the user profile. This number is then used to estimate the capacity of the site, based on the assumed user profile.

For example, the cost of 100 concurrent users is 100 * 1.4496 = 145 Mcycles. These calculations are shown in the section on Calculate Capacity, and can also be plotted on a graph.

Another interesting number is the cost of a checkout transaction (Add Item + Checkout): it’s very high, at 415.22 Mcycles. This translates to a server capacity of only two checkout transactions per second, since our limit for this machine is 800 Mcycles. Although this seems like a very low capacity for this operation, the average frequency of this operation is also very low (0.000183 per second), so server capacity for concurrent shoppers is not overly affected. The cost per user operation per second shows a very low number at 0.0759 Mcycles (approximately 5% of the cost of the entire user profile).

On the other hand, the browse operation has very low cost, at 58.50 Mcycles. However the frequency of the browse operation is very high, at 0.006102, so the browse operation places quite a load on the site. The cost per user operation shows a high number of 0.3579 Mcycles (approximately 25% of the cost of the entire user profile).

The search operation shows a moderate cost, at 150.12 Mcycles. However, the frequency of the search operation is also high, at 0.002391, so it results in a high cost per user operation per second of 0.3590 Mcycles (approximately 25% of the cost of the entire user profile).

The numbers suggest that it would be best to start optimizing performance with the browse pages and the search pages to improve shopper capacity at the site.

Calculate Capacity

As you can see in the following table, we calculate the maximum capacity of the site to be 550 concurrent shoppers, with a cost of 797.5 Mcycles. This is the maximum limit, since the two CPUs provide the site with a budget of 800 Mcycles of capacity. A more conservative approach is to use 70% CPU utilization. This results in a CPU budget of 560 Mcycles, with a calculated maximum capacity between 350 and 400 concurrent shoppers.

Shoppers Cost (Mcycles)
0 0.0
50 72.5
100 145.0
150 217.5
200 290.0
250 362.5
300 435.0
350 507.5
400 580.0
450 652.5
500 725.0
550 797.5
600 870.0

Verify Capacity

You can verify site capacity by running a script that reflects the user profile with an increasing number of shoppers, then comparing the results against the table in the Calculate Capacity section. The verification script is simply a collection of all individual TCA measurement scripts aggregated and run as a single script.

The script records CPU utilization, expressed as the cost of the load, charts utilization, and compares it with TCA measurements for the individual operations. For the sample customer site (dual Pentium II Xeon 400 MHz CPUs), the table looks like the following:

Shoppers CPU Utilization Avg. Operation Latency (ms) ASP Req. per sec. Cost (Mcycles)
0 0.000% 0.000 0.000 0.000
100 14.706% 421.024 2.659 117.648
200 31.960% 439.151 5.391 255.680
300 57.939% 647.937 8.105 463.512
400 90.464% 15770.377 8.299 723.712
500 94.616% 25561.366 9.583 756.928

The table shows that the verified maximum capacity appears to be 300 concurrent shoppers. ASP request throughput is higher at a load level of 400 shoppers. However, at 400 shoppers, notice that operation latency (response time) jumps to 11.8 seconds, which is no longer an acceptable quality level. If latency is not a consideration, we see that shopper capacity is 500 shoppers, as ASP requests-per-second continue to climb at this load level, perhaps up to 550 shoppers.

Keep in mind that the verification script runs at four times the transaction and request rate of the live customer site. The maximum number of shoppers observed here, then, translates to approximately 4 x 300 = 1,200 shoppers on the customer’s live site.

See the section on Verification Scripts for a collection of sample scripts you can use for verifying capacity at your site.

Transaction Cost Analysis Scripts

This section contains sample TCA scripts for use in measuring cost (Cost Measurement Scripts) or verifying capacity (Verification Scripts). Note that these scripts are to be used with InetMonitor.

Cost Measurement Scripts

The following sample scripts exercise load for TCA cost measurements:

Default Home Page

The following sample script goes to the site home page:

REM *** Goto Home Page ***
GET url:/st/home.asp

Search (Good Results)

The following sample script measures the cost of search operations, for which the user obtains good results:

REM *** Goto Home Page ***
GET url:/st/home.asp
POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052

REM *** Search ***
LOOP 10
   GET url:/st/search/search-results.asp?RANDLIST(TCASearchGoodData.txt)
ENDLOOP

Search (No Results)

The following sample script measures the cost of search operations, for which the user obtains no results:

REM *** Goto Home Page ***
GET url:/st/home.asp
POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052

REM *** Search ***
LOOP 10
   GET url:/st/search/search-results.asp?RANDLIST(TCASearchNotFoundData.txt)
ENDLOOP

Add Item

The following sample script measures the cost of adding items to a shopping basket:

REM *** Goto Home Page ***
GET url:/st/home.asp
POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052

REM *** Add Item ***
LOOP 10
   GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
ENDLOOP

Add Item + Delete

The following sample script measures the cost of adding an item to a shopping basket, and then deleting it:

REM *** Goto Home Page ***
GET url:/st/home.asp
POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052

LOOP 10
   GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
   GET url:/st/xt_orderform_delitem.asp?index=0
ENDLOOP

Add Item + Checkout

The following sample script measures the cost of adding an item to a shopping basket, and then checking out (purchasing the item):

REM *** Goto Home page ***
GET url:/st/home.asp

REM *** Login as a Member ***
GET SEQULIST(TCAUsersLoginData.txt)

REM *** clear cart
POST url:/st/xt_orderform_clearitems.asp

LOOP 10
   REM *** Add Item
   GET url:/st/xt_orderform_additem.asp?sku=380157&qty=1
   REM *** Check out ***
   POST url:/st/member/orderform.asp?PROPERTIES:x=72&y=15
   POST url:/st/member/confirm_order.asp?PROPERTIES:cc_name=asdsad+asdsad&bill_to_name=asdsad+asdsad&bill_to_company=asdsad&bill_to_street=asdasd&bill_to_street2=asdas&bill_to_city=asdsad&bill_to_state=ma&bill_to_zip=01605&bill_to_phone=asdasd&cc_type=visa&cc_number=4111111111111111&cc_expmonth=5&cc_expyear=1999&po_number=&coupon_one=&coupon_two=&coupon_three=&x=74&y=18
   POST url:/st/member/xt_orderform_purchase.asp?PROPERTIES:_VERIFY_WITH=_total_total%3D8999&_VERIFY_WITH=bill_to_zip%3D01605&_VERIFY_WITH=_tax_total%3D0&bill_to_name=asdsad+asdsad&bill_to_company=asdsad&bill_to_street=asdasd&bill_to_street2=asdas&bill_to_city=asdsad&bill_to_state=ma&bill_to_zip=01605&bill_to_phone=asdasd&po_number=&coupon_two=&coupon_three=&cc_expyear=1999&coupon_one=&cc_name=asdsad+asdsad&cc_expmonth=5&cc_type=visa&cc_number=4111111111111111&IMAGE1.x=55&IMAGE1.y=12
ENDLOOP

Browse

The first few lines of script prior to the browse loop set up the user environment. In the following example, a zip code must be provided before the shopper can obtain pricing information. The InetMonitor loop command executes the browse requests. Looping and performing the browse request multiple times statistically reduces the noise resulting from the zip code setup:

REM *** Goto Home Page ***
GET url:/st/home.asp
POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052

LOOP 10
   REM *** Browse Departments ***
   %14 SKIP 1
   POST url:/st/department.asp?show=all&PROPERTIES:RANDLIST(TCACategoryIDData.txt)
   %15 SKIP 1
   POST url:/st/department.asp?show=one&PROPERTIES:RANDLIST(TCADepartmentIDData.txt)

   REM *** Browse SKUSet ***
   %23 SKIP 1
   GET url:/st/skuset.asp?RANDLIST(TCASKUSETData.txt)

   REM *** Browse Product ***
   %48 SKIP 1
   GET url:/st/product.asp?skusetid=RANDLIST(TCAProductBrowseData.txt)
ENDLOOP

Login

The following sample script measures the cost of logging into the site:

REM *** Goto Home page ***
GET url:/st/home.asp

REM *** Login as a Member ***
LOOP 10
   GET SEQULIST(TCAUSersLoginData.txt)
ENDLOOP

Register

The following script measures the cost of registering a new user:

REM +++ New Shopper +++
POST SEQULIST(TCARegisterUsersData.txt)

Zip Code

The following script measures the cost of obtaining a user’s zip code:

REM *** Goto Home Page ***
GET url:/st/home.asp
LOOP 10
   POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052
ENDLOOP

View Cart

The following script measures the cost of adding items, and then viewing the items in a shopping basket:

REM *** Goto Home Page ***
GET url:/st/home.asp
POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052

REM *** Add Item ***
GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
REM GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
REM GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
REM GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1

REM *** View Cart ***
LOOP 20
   GET url:/st/cart.asp
ENDLOOP

Verification Scripts

The scripts in this section are sample scripts that can be used to exercise load for TCA verification measurements:

Baseline Verification

The following sample script provides baseline verification for site operations:

REM *** VERIFICATION SCRIPT FOR TCA ***
REM *** 11 Operations in 11 minutes

REM *** Goto Home Page ***
GET url:/st/home.asp
POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052

REM *** Start Timer ***
STARTTIMER MAINTIMER

REM *** Goto Home page 2.5 operations ***
GET url:/st/home.asp
SLEEP RANDNUMBER(40000,80000)
GET url:/st/home.asp
SLEEP RANDNUMBER(40000,80000)
%50 SKIP 2
GET url:/st/home.asp
SLEEP RANDNUMBER(40000,80000)

REM *** Search Good Result 1.6 operations ***
GET url:/st/search/search-results.asp?RANDLIST(TCASearchGoodData.txt)
SLEEP RANDNUMBER(40000,80000)
%40 SKIP 2
GET url:/st/search/search-results.asp?RANDLIST(TCASearchGoodData.txt)
SLEEP RANDNUMBER(40000,80000)

REM *** Search Not Found 0.1 operations ***
%90 SKIP 2
GET url:/st/search/search-results.asp?RANDLIST(TCASearchNotFoundData.txt)
SLEEP RANDNUMBER(40000,80000)

REM *** Add Item 0.2 operations ***
%80 SKIP 2
GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
SLEEP RANDNUMBER(40000,80000)

REM *** Add Item and Delete 0.1 operations ***
%90 SKIP 3
GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
SLEEP RANDNUMBER(20000,40000)
GET url:/st/xt_orderform_delitem.asp?index=0
SLEEP RANDNUMBER(20000,40000)

REM *** Add Items and Checkout, Clear cart first 0.1 operations ***
%90 SKIP 12
GET SEQULIST(TCAUSersLoginData.txt)
SLEEP RANDNUMBER(8000,12000)
POST url:/st/xt_orderform_clearitems.asp
SLEEP RANDNUMBER(8000,12000)
GET url:/st/xt_orderform_additem.asp?sku=380157&qty=1
SLEEP RANDNUMBER(8000,12000)
POST url:/st/member/orderform.asp?PROPERTIES:x=72&y=15
SLEEP RANDNUMBER(8000,12000)
POST url:/st/member/confirm_order.asp?PROPERTIES:cc_name=asdsad+asdsad&bill_to_name=asdsad+asdsad&bill_to_company=asdsad&bill_to_street=asdasd&bill_to_street2=asdas&bill_to_city=asdsad&bill_to_state=ma&bill_to_zip=01605&bill_to_phone=asdasd&cc_type=visa&cc_number=4111111111111111&cc_expmonth=5&cc_expyear=1999&po_number=&coupon_one=&coupon_two=&coupon_three=&x=74&y=18
SLEEP RANDNUMBER(8000,12000)
POST url:/st/member/xt_orderform_purchase.asp?PROPERTIES:_VERIFY_WITH=_total_total%3D8999&_VERIFY_WITH=bill_to_zip%3D01605&_VERIFY_WITH=_tax_total%3D0&bill_to_name=asdsad+asdsad&bill_to_company=asdsad&bill_to_street=asdasd&bill_to_street2=asdas&bill_to_city=asdsad&bill_to_state=ma&bill_to_zip=01605&bill_to_phone=asdasd&po_number=&coupon_two=&coupon_three=&cc_expyear=1999&coupon_one=&cc_name=asdsad+asdsad&cc_expmonth=5&cc_type=visa&cc_number=4111111111111111&IMAGE1.x=55&IMAGE1.y=12
SLEEP RANDNUMBER(8000,12000)

REM *** Browse Products 4 operations ***
REM *** Browse Departments show all 14% ***
%86 SKIP 1
POST url:/st/department.asp?show=all&PROPERTIES:RANDLIST(TCACategoryIDData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Departments show one 15% ***
%85 SKIP 1
POST url:/st/department.asp?show=one&PROPERTIES:RANDLIST(TCADepartmentIDData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse SKUSet 23% ***
%77 SKIP 1
GET url:/st/skuset.asp?RANDLIST(TCASKUSETData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Product 48% ***
%52 SKIP 1
GET url:/st/product.asp?skusetid=RANDLIST(TCAProductBrowseData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Departments show all 14% ***
%86 SKIP 1
POST url:/st/department.asp?show=all&PROPERTIES:RANDLIST(TCACategoryIDData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Departments show one 15% ***
%85 SKIP 1
POST url:/st/department.asp?show=one&PROPERTIES:RANDLIST(TCADepartmentIDData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse SKUSet 23% ***
%77 SKIP 1
GET url:/st/skuset.asp?RANDLIST(TCASKUSETData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Product 48% ***
%52 SKIP 1
GET url:/st/product.asp?skusetid=RANDLIST(TCAProductBrowseData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Departments show all 14% ***
%86 SKIP 1
POST url:/st/department.asp?show=all&PROPERTIES:RANDLIST(TCACategoryIDData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Departments show one 15% ***
%85 SKIP 1
POST url:/st/department.asp?show=one&PROPERTIES:RANDLIST(TCADepartmentIDData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse SKUSet 23% ***
%77 SKIP 1
GET url:/st/skuset.asp?RANDLIST(TCASKUSETData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Product 48% ***
%52 SKIP 1
GET url:/st/product.asp?skusetid=RANDLIST(TCAProductBrowseData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Departments show all 14% ***
%86 SKIP 1
POST url:/st/department.asp?show=all&PROPERTIES:RANDLIST(TCACategoryIDData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Departments show one 15% ***
%85 SKIP 1
POST url:/st/department.asp?show=one&PROPERTIES:RANDLIST(TCADepartmentIDData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse SKUSet 23% ***
%77 SKIP 1
GET url:/st/skuset.asp?RANDLIST(TCASKUSETData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Product 48% ***
%52 SKIP 1
GET url:/st/product.asp?skusetid=RANDLIST(TCAProductBrowseData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** End Of Browse Products 4 operations ***

REM *** Login as a Member 0.2 operations ***
%80 SKIP 2
GET SEQULIST(TCAUSersLoginData.txt)
SLEEP RANDNUMBER(40000,80000)

REM *** Register New Shopper 0.1 operations ***
%90 SKIP 2
POST SEQULIST(TCARegisterUsersData.txt)
SLEEP RANDNUMBER(40000,80000)

REM *** Zip Code 1.8 operations ***
POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052
SLEEP RANDNUMBER(40000,80000)
%20 SKIP 2
POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052
SLEEP RANDNUMBER(40000,80000)

REM *** View Cart 0.3 operations ***
%70 SKIP 4
GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
SLEEP RANDNUMBER(20000,40000)
GET url:/st/cart.asp
SLEEP RANDNUMBER(20000,40000)

REM *** End Timer ***
ENDTIMER MAINTIMER

REM *** Kill ASP session ***
GET url:/st/quitsession.asp

Optimized Site Verification

The following sample script verifies site optimization:

REM *** VERIFICATION SCRIPT FOR TCA ***
REM *** 11 Operations in 11 minutes

REM *** Goto Home Page ***
GET url:/st/home.asp
POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052

REM *** Start Timer ***
STARTTIMER MAINTIMER

REM *** Goto Home page 2.5 operations ***
GET url:/st/home.asp
SLEEP RANDNUMBER(40000,80000)
GET url:/st/home.asp
SLEEP RANDNUMBER(40000,80000)
%50 SKIP 2
GET url:/st/home.asp
SLEEP RANDNUMBER(40000,80000)

REM *** Search Good Result 1.6 operations ***
GET url:/st/search/search-results.asp?RANDLIST(TCASearchGoodData.txt)
SLEEP RANDNUMBER(40000,80000)
%40 SKIP 2
GET url:/st/search/search-results.asp?RANDLIST(TCASearchGoodData.txt)
SLEEP RANDNUMBER(40000,80000)

REM *** Search Not Found 0.1 operations ***
%90 SKIP 2
GET url:/st/search/search-results.asp?RANDLIST(TCASearchNotFoundData.txt)
SLEEP RANDNUMBER(40000,80000)

REM *** Add Item 0.2 operations ***
%80 SKIP 2
GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
SLEEP RANDNUMBER(40000,80000)

REM *** Add Item and Delete 0.1 operations ***
%90 SKIP 3
GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
SLEEP RANDNUMBER(20000,40000)
GET url:/st/xt_orderform_delitem.asp?index=0
SLEEP RANDNUMBER(20000,40000)

REM *** Add Items and Checkout, Clear cart first 0.1 operations ***
%90 SKIP 12
GET SEQULIST(TCAUSersLoginData.txt)
SLEEP RANDNUMBER(8000,12000)
POST url:/st/xt_orderform_clearitems.asp
SLEEP RANDNUMBER(8000,12000)
GET url:/st/xt_orderform_additem.asp?sku=380157&qty=1
SLEEP RANDNUMBER(8000,12000)
POST url:/st/member/orderform.asp?PROPERTIES:x=72&y=15
SLEEP RANDNUMBER(8000,12000)
POST url:/st/member/confirm_order.asp?PROPERTIES:cc_name=asdsad+asdsad&bill_to_name=asdsad+asdsad&bill_to_company=asdsad&bill_to_street=asdasd&bill_to_street2=asdas&bill_to_city=asdsad&bill_to_state=ma&bill_to_zip=01605&bill_to_phone=asdasd&cc_type=visa&cc_number=4111111111111111&cc_expmonth=5&cc_expyear=1999&po_number=&coupon_one=&coupon_two=&coupon_three=&x=74&y=18
SLEEP RANDNUMBER(8000,12000)
POST url:/st/member/xt_orderform_purchase.asp?PROPERTIES:_VERIFY_WITH=_total_total%3D8999&_VERIFY_WITH=bill_to_zip%3D01605&_VERIFY_WITH=_tax_total%3D0&bill_to_name=asdsad+asdsad&bill_to_company=asdsad&bill_to_street=asdasd&bill_to_street2=asdas&bill_to_city=asdsad&bill_to_state=ma&bill_to_zip=01605&bill_to_phone=asdasd&po_number=&coupon_two=&coupon_three=&cc_expyear=1999&coupon_one=&cc_name=asdsad+asdsad&cc_expmonth=5&cc_type=visa&cc_number=4111111111111111&IMAGE1.x=55&IMAGE1.y=12
SLEEP RANDNUMBER(8000,12000)

REM *** Browse Products 4 operations ***
REM *** Browse Departments show all 14% ***
%86 SKIP 1
GET RANDLIST(TCAEarlyRenderCategoryIDData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Departments show one 15% ***
%85 SKIP 1
GET RANDLIST(TCAEarlyRenderDepartmentIDData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse SKUSet 23% ***
%77 SKIP 1
GET RANDLIST(TCAEarlyRenderSKUSETData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Product 48% ***
%52 SKIP 1
GET RANDLIST(TCAEarlyRenderProductBrowseData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Departments show all 14% ***
%86 SKIP 1
GET RANDLIST(TCAEarlyRenderCategoryIDData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Departments show one 15% ***
%85 SKIP 1
GET RANDLIST(TCAEarlyRenderDepartmentIDData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse SKUSet 23% ***
%77 SKIP 1
GET RANDLIST(TCAEarlyRenderSKUSETData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Product 48% ***
%52 SKIP 1
GET RANDLIST(TCAEarlyRenderProductBrowseData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Departments show all 14% ***
%86 SKIP 1
GET RANDLIST(TCAEarlyRenderCategoryIDData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Departments show one 15% ***
%85 SKIP 1
GET RANDLIST(TCAEarlyRenderDepartmentIDData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse SKUSet 23% ***
%77 SKIP 1
GET RANDLIST(TCAEarlyRenderSKUSETData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Product 48% ***
%52 SKIP 1
GET RANDLIST(TCAEarlyRenderProductBrowseData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Departments show all 14% ***
%86 SKIP 1
GET RANDLIST(TCAEarlyRenderCategoryIDData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Departments show one 15% ***
%85 SKIP 1
GET RANDLIST(TCAEarlyRenderDepartmentIDData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse SKUSet 23% ***
%77 SKIP 1
GET RANDLIST(TCAEarlyRenderSKUSETData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Product 48% ***
%52 SKIP 1
GET RANDLIST(TCAEarlyRenderProductBrowseData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** End Of Browse Products 4 operations ***

REM *** Login as a Member 0.2 operations ***
%80 SKIP 2
GET SEQULIST(TCAUSersLoginData.txt)
SLEEP RANDNUMBER(40000,80000)

REM *** Register New Shopper 0.1 operations ***
%90 SKIP 2
POST SEQULIST(TCARegisterUsersData.txt)
SLEEP RANDNUMBER(40000,80000)

REM *** Zip Code 1.8 operations ***
POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052
SLEEP RANDNUMBER(40000,80000)
%20 SKIP 2
POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052
SLEEP RANDNUMBER(40000,80000)

REM *** View Cart 0.3 operations ***
%70 SKIP 4
GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
SLEEP RANDNUMBER(20000,40000)
GET url:/st/cart.asp
SLEEP RANDNUMBER(20000,40000)

REM *** End Timer ***
ENDTIMER MAINTIMER

REM *** Kill ASP session ***
GET url:/st/quitsession.asp

HTML Verification

The following sample script verifies HTML code for the site:

REM *** VERIFICATION SCRIPT FOR TCA ***
REM *** 11 Operations in 11 minutes

REM *** Goto Home Page ***
REM GET url:/st/home.asp
REM POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052

REM *** Start Timer ***
STARTTIMER MAINTIMER

REM *** Goto Home page 2.5 operations ***
REM GET url:/st/home.asp
SLEEP RANDNUMBER(40000,80000)
REM GET url:/st/home.asp
SLEEP RANDNUMBER(40000,80000)
REM %50 SKIP 2
REM GET url:/st/home.asp
%50 SKIP 1
SLEEP RANDNUMBER(40000,80000)

REM *** Search Good Result 1.6 operations ***
REM GET url:/st/search/search-results.asp?RANDLIST(TCASearchGoodData.txt)
SLEEP RANDNUMBER(40000,80000)
REM %40 SKIP 2
REM GET url:/st/search/search-results.asp?RANDLIST(TCASearchGoodData.txt)
%40 SKIP 1
SLEEP RANDNUMBER(40000,80000)

REM *** Search Not Found 0.1 operations ***
REM %90 SKIP 2
REM GET url:/st/search/search-results.asp?RANDLIST(TCASearchNotFoundData.txt)
%90 SKIP 1
SLEEP RANDNUMBER(40000,80000)

REM *** Add Item 0.2 operations ***
REM %80 SKIP 2
REM GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
%80 SKIP 1
SLEEP RANDNUMBER(40000,80000)

REM *** Add Item and Delete 0.1 operations ***
REM %90 SKIP 3
REM GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
REM SLEEP RANDNUMBER(20000,40000)
REM GET url:/st/xt_orderform_delitem.asp?index=0
REM SLEEP RANDNUMBER(20000,40000)

%90 SKIP 2
SLEEP RANDNUMBER(20000,40000)
SLEEP RANDNUMBER(20000,40000)

REM *** Add Items and Checkout, Clear cart first 0.1 operations ***
REM %90 SKIP 12
REM GET SEQULIST(TCAUSersLoginData.txt)
REM SLEEP RANDNUMBER(8000,12000)
REM POST url:/st/xt_orderform_clearitems.asp
REM SLEEP RANDNUMBER(8000,12000)
REM GET url:/st/xt_orderform_additem.asp?sku=380157&qty=1
REM SLEEP RANDNUMBER(8000,12000)
REM POST url:/st/member/orderform.asp?PROPERTIES:x=72&y=15
REM SLEEP RANDNUMBER(8000,12000)
REM POST url:/st/member/confirm_order.asp?PROPERTIES:cc_name=asdsad+asdsad&bill_to_name=asdsad+asdsad&bill_to_company=asdsad&bill_to_street=asdasd&bill_to_street2=asdas&bill_to_city=asdsad&bill_to_state=ma&bill_to_zip=01605&bill_to_phone=asdasd&cc_type=visa&cc_number=4111111111111111&cc_expmonth=5&cc_expyear=1999&po_number=&coupon_one=&coupon_two=&coupon_three=&x=74&y=18
REM SLEEP RANDNUMBER(8000,12000)
REM POST url:/st/member/xt_orderform_purchase.asp?PROPERTIES:_VERIFY_WITH=_total_total%3D8999&_VERIFY_WITH=bill_to_zip%3D01605&_VERIFY_WITH=_tax_total%3D0&bill_to_name=asdsad+asdsad&bill_to_company=asdsad&bill_to_street=asdasd&bill_to_street2=asdas&bill_to_city=asdsad&bill_to_state=ma&bill_to_zip=01605&bill_to_phone=asdasd&po_number=&coupon_two=&coupon_three=&cc_expyear=1999&coupon_one=&cc_name=asdsad+asdsad&cc_expmonth=5&cc_type=visa&cc_number=4111111111111111&IMAGE1.x=55&IMAGE1.y=12
REM SLEEP RANDNUMBER(8000,12000)

%90 SKIP 1
SLEEP RANDNUMBER(40000,80000)

REM *** Browse Products 4 operations ***
REM *** Browse Departments show all 14% ***
%86 SKIP 1
GET RANDLIST(TCAEarlyRenderCategoryIDData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Departments show one 15% ***
%85 SKIP 1
GET RANDLIST(TCAEarlyRenderDepartmentIDData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse SKUSet 23% ***
%77 SKIP 1
GET RANDLIST(TCAEarlyRenderSKUSETData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Product 48% ***
%52 SKIP 1
GET RANDLIST(TCAEarlyRenderProductBrowseData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Departments show all 14% ***
%86 SKIP 1
GET RANDLIST(TCAEarlyRenderCategoryIDData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Departments show one 15% ***
%85 SKIP 1
GET RANDLIST(TCAEarlyRenderDepartmentIDData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse SKUSet 23% ***
%77 SKIP 1
GET RANDLIST(TCAEarlyRenderSKUSETData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Product 48% ***
%52 SKIP 1
GET RANDLIST(TCAEarlyRenderProductBrowseData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Departments show all 14% ***
%86 SKIP 1
GET RANDLIST(TCAEarlyRenderCategoryIDData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Departments show one 15% ***
%85 SKIP 1
GET RANDLIST(TCAEarlyRenderDepartmentIDData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse SKUSet 23% ***
%77 SKIP 1
GET RANDLIST(TCAEarlyRenderSKUSETData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Product 48% ***
%52 SKIP 1
GET RANDLIST(TCAEarlyRenderProductBrowseData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Departments show all 14% ***
%86 SKIP 1
GET RANDLIST(TCAEarlyRenderCategoryIDData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Departments show one 15% ***
%85 SKIP 1
GET RANDLIST(TCAEarlyRenderDepartmentIDData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse SKUSet 23% ***
%77 SKIP 1
GET RANDLIST(TCAEarlyRenderSKUSETData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** Browse Product 48% ***
%52 SKIP 1
GET RANDLIST(TCAEarlyRenderProductBrowseData.txt)
SLEEP RANDNUMBER(13000,17000)

REM *** End Of Browse Products 4 operations ***

REM *** Login as a Member 0.2 operations ***
REM %80 SKIP 2
REM GET SEQULIST(TCAUSersLoginData.txt)
%80 SKIP 1
SLEEP RANDNUMBER(40000,80000)

REM *** Register New Shopper 0.1 operations ***
REM %90 SKIP 2
REM POST SEQULIST(TCARegisterUsersData.txt)
%90 SKIP 1
SLEEP RANDNUMBER(40000,80000)

REM *** Zip Code 1.8 operations ***
REM POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052
SLEEP RANDNUMBER(40000,80000)
REM %20 SKIP 2
REM POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052
%20 SKIP 1
SLEEP RANDNUMBER(40000,80000)

REM *** View Cart 0.3 operations ***
REM %70 SKIP 4
REM GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
REM SLEEP RANDNUMBER(20000,40000)
REM GET url:/st/cart.asp
REM SLEEP RANDNUMBER(20000,40000)

%70 SKIP 4
SLEEP RANDNUMBER(40000,80000)

REM *** End Timer ***
ENDTIMER MAINTIMER

REM *** Kill ASP session ***
REM GET url:/st/quitsession.asp

Checkout Verification

The following sample script verifies the checkout procedure:

REM *** Goto Home page ***
GET url:/st/home.asp

REM *** Login as a Member ***
GET SEQULIST(TCAUsersLoginData.txt)

REM *** clear cart
POST url:/st/xt_orderform_clearitems.asp

LOOP 10
   REM *** Add Item
   GET url:/st/xt_orderform_additem.asp?sku=380157&qty=1
   SLEEP RANDNUMBER(40000,80000)
   REM *** Check out ***
   POST url:/st/member/orderform.asp?PROPERTIES:x=72&y=15
   SLEEP RANDNUMBER(40000,80000)
   POST url:/st/member/confirm_order.asp?PROPERTIES:cc_name=asdsad+asdsad&bill_to_name=asdsad+asdsad&bill_to_company=asdsad&bill_to_street=asdasd&bill_to_street2=asdas&bill_to_city=asdsad&bill_to_state=ma&bill_to_zip=01605&bill_to_phone=asdasd&cc_type=visa&cc_number=4111111111111111&cc_expmonth=5&cc_expyear=1999&po_number=&coupon_one=&coupon_two=&coupon_three=&x=74&y=18
   SLEEP RANDNUMBER(40000,80000)
   POST url:/st/member/xt_orderform_purchase.asp?PROPERTIES:_VERIFY_WITH=_total_total%3D8999&_VERIFY_WITH=bill_to_zip%3D01605&_VERIFY_WITH=_tax_total%3D0&bill_to_name=asdsad+asdsad&bill_to_company=asdsad&bill_to_street=asdasd&bill_to_street2=asdas&bill_to_city=asdsad&bill_to_state=ma&bill_to_zip=01605&bill_to_phone=asdasd&po_number=&coupon_two=&coupon_three=&cc_expyear=1999&coupon_one=&cc_name=asdsad+asdsad&cc_expmonth=5&cc_type=visa&cc_number=4111111111111111&IMAGE1.x=55&IMAGE1.y=12
   SLEEP RANDNUMBER(40000,80000)
ENDLOOP

Information in this document, including URL and other Internet web site references, is subject to change without notice.  The entire risk of the use or the results of the use of this resource kit remains with the user.  This resource kit is not supported and is provided as is without warranty of any kind, either express or implied.  The example companies, organizations, products, people and events depicted herein are fictitious.  No association with any real company, organization, product, person or event is intended or should be inferred.  Complying with all applicable copyright laws is the responsibility of the user.  Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation.

Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document.  Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.

© 1999-2000 Microsoft Corporation.  All rights reserved.

Microsoft and Windows NT are either registered trademarks or trademarks of Microsoft Corporation in the U.S.A. and/or other countries/regions.

The names of actual companies and products mentioned herein may be the trademarks of their respective owners.