Microsoft Site Server 3.0 Usage Analysis Capacity and Performance Analysis

August 1999

Microsoft Corporation

Definition of Terms

1. This performance metric does not include requests that are explicitly excluded (such as JPEGs and GIFs) when configuring a Site in Usage Import.

Overview

This document discusses the performance and scalability characteristics of the Microsoft® Site Server version 3.0 Analysis feature. Analysis consists of four main tools: Usage Import, Report Writer, Custom Import, and Content Analyzer. This capacity and performance analysis focuses on the Usage Import tool. This tool’s primary function is to import log files that contain usage data about your site into the Analysis database.

The information in this document was generated by a series of tests run on two servers. Issues that were targeted in these tests were performance bottlenecks and scalability issues when importing log files to the Analysis feature.

Note   Your performance numbers may vary from those provided here based on your hardware and software platform.

The potential performance bottlenecks associated with log file importing are enumerated here to assist you in capacity planning.

Processor and memory scalability are profiled in terms of import requests per second and bytes imported per second. This information should help you select the appropriate hardware configuration for your Analysis deployment.

The two servers used in these tests were configured as follows.

Usage Analysis Description

Analysis uses the log files generated by Web servers to analyze the usage and content of a site. The data in these log files falls into five categories:

• Hit. A line in a log file. Hits include requests for content, errors, blank lines, and Internet connection overhead. Hits are used to calculate summary hit counts and bandwidth data in reports.
  
  
• Request. Any hit that successfully retrieves content. Requests are related to content and user behavior. For example, if a user requests an HTML page that has three graphics files, the Web server might make a log entry for the three graphics files, the HTML page, and perhaps add some lines for overhead. However, only one request is counted by Analysis.
  
  
• Visit. A series of requests by a user. A visit begins in one of two ways: the user can be referred to the site through a link, or the user can type the site’s URL in his or her browser. A visit ends when a specified period of time (the default is 30 minutes) has passed without any additional requests to the site.
  
  
• User. Any entity (person or Internet gateway) associated with a host name (for example, example.microsoft.com) that accesses a site. If the user is an Internet gateway, the tally of users may be smaller than the actual number of people who visited the site. Usage Import’s preferred method of recognizing a user is a cookie, which is a small identifying chunk of data placed on the user’s hard drive. If no cookie exists, Usage Import attempts to use a registered user name. In the absence of a registered user name, it uses the host name.
  
  
• Organization. A group of related users that has registered one or more Internet domains with InterNIC, APNIC, or RIPE NCC. Typical organizations are businesses, government agencies, universities, and nonprofit groups.

Usage Import imports these log files into the Analysis database, where Report Writer can be used to analyze the log file content. For this capacity and performance analysis, a Microsoft® SQL Server™ database was used for the Analysis database.

Critical Program Execution Phases

First the log file is parsed. The data gets read into memory and put into a cache called the hits buffer. From the hits buffer, the inference engine takes the hits and matches the server, site, and user creating state data. In this phase, simple aggregations on the hit data are also performed. Once the user is identified, the inference engine will add requests to the user's visit. All the while the purging process periodically initiates and examines each user. The purge mechanism identifies users that have not performed any requests in more than 30 minutes (left the site), and flushes their data to the database. Each user contains the user information itself, plus their current visit and the requests that make up their visit.

Log File Profiles

All tests described in this document were run using pre-generated log files. Pre-generated log files are maximally dense; they do not contain entries that are filtered out of the input stream before import processing. Such entries that are filtered out before import processing include image file entries. A conservative estimate of the ratio of actual log file size to pre-generated log file size is 5 to 1. This ratio can be applied to important import performance metrics such as import requests per second and bytes imported per second. For example, if for a given hardware configuration and log file profile 300 requests per second is observed, then it is reasonable to assume that approximately 1500 requests per second would be observed with actual log files.

The Usage Import tests were run using a variety of pre-generated logs including those in the Table 1.

Table 1  Usage Import Test Suite

Summary of Scalability and Performance

Based on the data collected in this document, the following assertions can be made about scaling and performance for Analysis:

• Peak import rates in excess of 300 requests/second and 97 KB/second (5.7 MB/minute) were observed on one-processor configurations with pre-generated log files.  When conservatively adjusted for actual log files these performance metrics increase to 1500 requests/second and 485 KB/second (28MB/minute).
  
  
• Performance improvement of two-processor systems over one-processor systems is marginal as measured by requests/second or import bytes/second. When memory was not a bottleneck, these metrics only increased between six percent and 17 percent. Four-processor systems showed a decrease in performance relative to two-processor systems.
  
  
• Insufficient memory was a bottleneck with 64 MB of RAM when importing the 273 MB log file consisting of one million entries (with 1000 users, 100 visits per user, and ten requests per visit). Memory paging out increased to 135 pages out per second on a one-processor system. When memory was increased to 128 MB of RAM, pages out per second decreased to 5.8 and the performance metrics returned to approximately peak values.
  
  
• Memory does not appear to be a bottleneck as long as the total private bytes consumed by all Analysis executables leaves sufficient RAM space for execution of the Microsoft® Windows NT® operating system and other running services.
  
  
• For log files with more than one million entries (composed of 10,000 users with ten visits per user and ten requests per visit) substantial consumption of Analysis server processor resource and Microsoft® SQL Server™ disk resource may degrade performance.

Factors Affecting Usage Analysis Performance

The tests in this document used two classes of pre-generated log files: those with a fixed number of users and those with a variable number of users.

Log File with Fixed Number of Users

A 273 MB log file consisting of one million entries with 1000 users, 100 visits per user, and ten requests per visit was used in this section.

Memory

Charts 1, 2, and 3 show how total import time, import requests per second, and import bytes per second scaled in this test as a function of physical RAM. For systems with 64 MB of RAM, a substantial degradation in these performance metrics occurred with pages out per second increasing substantially to 135 pages out per second. The total number of private bytes consumed by all Analysis executables is approximately 44 MB, not leaving enough physical RAM space for Microsoft® Windows NT® and other required services.

CPU

Charts 1, 2, and 3 also show how total import time, import requests per second, and import bytes per second scaled as a function of the number of processors.

In systems configured with at least 128 MB of RAM, memory is not a bottleneck and the performance metrics change only marginally as the number of processors is varied. Two-processor systems improve import request rates and import byte rates between six percent and 17 percent over the rates of one-processor systems.

Chart 1  Import time scaling

Chart 2  Import requests per second scaling (actual log file)

Chart 3  (Actual log file) import bytes per second scaling

Log Files with Variable Number of Users

It is of interest to measure how the performance changes as the number of users, and the number of open visits, increases. Because the performance improvement of importing with two-processors is only marginal and because insufficient memory was shown to bottleneck performance substantially, a system configuration with one-processor and 512 MB of RAM was chosen to collect measurements.

Figure 1 shows that the import time is a linear function of the number of open visits.

Figure 1

Table 2 shows the import time as a function of the projected number of entries in an actual log file which consists of a variable number of users with ten visits per user and ten requests per visit.

Table 2

Performance Degradation Regimes

Figures 2 and 3 show that during the tests import requests per second and import bytes per second began to decrease as a function of the number of users once the number of users increased to about 10,000. The total private bytes consumed by all Analysis executables is shown in Figure 4 and did not exceed 294 MB for up to 100,000 users, therefore leaving sufficient RAM space for operation of Windows NT and other required services. This suggests that insufficient memory was not a bottleneck contributing to performance degradation in this case.  

Figure 2

Figure 3

Figure 4

Heavy Consumption of Analysis Server CPU and SQL Server Disk

Performance degradation may be explained by heavy consumption of Analysis server processor and SQL Server disk resources during the first 26 percent of total import time of the log file with 10,000 users. During the first six percent of the total import time, processor utilization is approximately 100 percent. This initial period of time extends beyond the time required to create the first 10,000 open visits. The next 20 percent of the total import time is spent handling substantial SQL Server disk activity. In particular, disk activity increases and averages approximately 48 percent overall with sustained periods of 100 percent disk utilization. After this first 26 percent of total import time is complete, all hardware resource consumption remains within reasonable bounds.

System Recommendations

The results of the tests discussed in this document suggest that for best performance, you follow these recommendations:

• Use a one-processor computer for the Analysis server.
  
  
• Monitor Analysis server memory consumption. The appropriate Analysis server RAM size will depend on the number of users recorded in the log file imported. The total number of private bytes consumed by all Analysis services is shown in Figure 4 (as a function of the number of users with 10 visits per user and 10 hits per visit). This Analysis RAM consumption should leave sufficient RAM space for operation of Windows NT and other required services.
  
  
• Monitor Microsoft® SQL Server™ disk activity for excessive utilization. The average SQL Server disk queue length should not exceed the number of physical disk drives.

Appendix A:  Scalability and Performance Data

Processor Usage

Analysis Server Processor Scaling

Memory Usage

Analysis Server Memory Scaling

Disk Usage

SQL Server Disk Scaling

Appendix B:  Critical Monitoring Counters

All counters noted can be found in the Microsoft ® Windows NT® Performance Monitor and can be used to capture potential performance bottlenecks due to excessive hardware resource utilization.

The counters in the Usage Analysis server memory, process, and system objects and the SQL Server physical disk object can be used to monitor capacity.

Memory

Available bytes

Committed bytes

Pages in per second

Pages out per second

Process

Private bytes UAS, Uimport

Working set UAS, Uimport

System

Percentage of total processor

Physical Disk

Percentage of disk time

Disk reads per second

Disk writes per second

Average disk queue length

Information in this document, including URL and other Internet web site references, is subject to change without notice.  The entire risk of the use or the results of the use of this resource kit remains with the user.  This resource kit is not supported and is provided as is without warranty of any kind, either express or implied.  The example companies, organizations, products, people and events depicted herein are fictitious.  No association with any real company, organization, product, person or event is intended or should be inferred.  Complying with all applicable copyright laws is the responsibility of the user.  Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation.

Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document.  Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.

©1998-2000 Microsoft Corporation.  All rights reserved.

Microsoft, Windows and Windows NT are either registered trademarks or trademarks of Microsoft Corporation in the U.S.A. and/or other countries/regions.

The names of actual companies and products mentioned herein may be the trademarks of their respective owners.