Microsoft Site Server 3.0 Direct Mail Capacity and Performance Analysis

April 1999

Microsoft Corporation

Definition of Terms

Chapter 1 Overview

This document evaluates the performance and scalability characteristics of Microsoft® Site Server version 3.0 Direct Mail. Also demonstrated are procedures for identifying these characteristics. This document can be used to assess the value of adding resources, and to identify which resources would satisfy greater capacity needs.

Analyzing Direct Mail

System Configuration

In the test scenarios, five computers were used in the following configurations:

Direct Mail Processor Scaling

Test Profiles

Direct Mail Description

Introduction to Direct Mailer

Direct Mail sends electronic mail to groups of users. In addition to sending bulk mail to existing customers, you can use Direct Mail to send bulk e-mail to Web site users. You can also personalize messages according to these factors:

• What parts of the site the user has visited (as recorded by usage logs).
  
  
• Information provided by the user, such as personal preferences, products already purchased, and so on.
  
  
• The number of times the user has browsed your site.

The mail process can be automated. For example, every Monday morning, Direct Mail can check which users have browsed the site for your new product in the past week. E-mail can then be sent to those users. Note that users who have never seen the site before will receive an introductory message; users who have been to the site more than once will receive more detailed information about the product. This task can be scheduled for low activity periods, such as before 4:00 A.M., so that your computers are free for your normal business activities.

Direct Mail utilizes several components:

• Direct Mailer. Sets up mailings.
  
  
• Microsoft Windows NT Scheduler Service. Determines when to send mailings.
  
  
• Site Server List Builder Service. Sends queries to Microsoft® Site Server version 3.0 Analysis for mailing lists.
  
  
• Site Server Analysis. Analyzes Web site usage, associating usage with user profiles imported from the Membership Directory.
  
  
• Message Builder Service. Composes messages and sends them to a Simple Mail Transfer Protocol (SMTP) server.

Interaction and process of Direct Mail

The following diagram shows the Direct Mail service broken down by the physical servers that house the interacting components.

The process of sending mail operates in three distinct phases:

• Distribution List Resolution. The Message Builder Service steps through the list and performs an LDAP search query to retrieve the e-mail name from the Membership Directory.
  
  
• Mail Address Sort. The Message Builder Service sorts the list to remove duplicates.
  
  
• Send and Receive:
  • Non-personalized static HTML. The HTML document is retrieved from the Web server and sent to the SMTP server for each e-mail recipient.
    
    
  • Non-personalized server side scripting (ASP). The ASP script is executed from the Web server, personalized for the first user in the distribution list, and then sent to the SMTP server for each e-mail recipient.
    
    
  • Personalized static HTML. For each recipient, the HTML document is retrieved from the Web server and then sent to the SMTP server.
    
    
  • Personalized server side scripting (ASP). The ASP script is executed and personalized for each e-mail recipient and then sent to the SMTP server.

Summary of Scalability and Performance

Based on the data collected in this document, the following assertions can be made about scaling and performance for Direct Mail:

• Direct Mail scales to two processors, because the distribution list expansion code has 2 threads. A four-processor system doesn't substantially increase throughput.
  
  
• Following the performance tuning procedures for the Membership Directory specifically for mail interaction will substantially reduce the time for the distribution list expansion. These procedures are described in the Site Server Service Pack 1 Readme.
  
  
• Personalized mail takes longer than non-personalized mail for static mailings by approximately 40 percent and for dynamic mailings by approximately 140 percent due to Web server interaction.
  
  
• Mail sent to a dynamic mailing list (dynamic mail) takes even more time and resources than static HTML-type mailings because the script must run on the web server to be processed before the mail can be sent. For example, in Non-personalized static versus dynamic mailing, the time to complete was between 9 and 22 percent faster.
  
  
• Non-personalized Static Mailing was the fastest to complete and it also had the highest send rate at 54 per second.

Chapter 2 Detail Discussion of Scalability and Performance

Time to Complete a Mailing

This chart shows that the time resource cost for adding personalization to a simple HTML is approximately 40 percent greater.

It also shows that a simple dynamic script can increase the time to complete a mailing by 20 percent over a static script. The script in this example is in Appendix B. It is an uncomplicated script. More complex scripts will increase maximum throughput and time to completion.

Time per mailing by mail type

Maximum transaction rates

These tables display the maximum mail send rate when it is in the send and receive mail phase. The rates show the maximum sustainable speed at different list sizes.

• List size does not affect the maximum rates.
  
  
• Non-personalized mail is faster since it does not have to go to the Web server.

Maximum send rate by mail type

Memory Utilization on the
Message Builder/List Builder Computer

This chart shows the maximum private bytes utilized by the Message Builder and List Builder Services while sending the different transactions. These memory consumption values are only for the mail services and do not include the memory used by Windows NT or any other process running on the server.

Memory utilization on Message Builder/List Builder by mail type

SQL Resource Utilization

The SQL server maintains the database for the Membership Directory. During the first phase of the mailing operation it handles the requests from the LDAP server for the user objects. This table details the CPU utilization for this system. The disk subsystem on the SQL server is also utilized. The specifications and recommendations for this specific subsystem are documented in a separate paper. The SQL server should be optimized as described in the Site Server 3.0 Membership Directory Configuration and Tuning document, which is also available from Microsoft.

The phase length in this table refers to the active period for the SQL server specific to the mailer operation. According to the description above this would be phase one.

However, if the personalized script makes use of the AUO object then another request could be made through the LDAP server to the SQL server.

LDAP Server Resource Utilization

The LDAP Service handles the requests for name resolution during the first phase of the mailing operation. The Message Builder Service computer opens a single connection to the LDAP server and makes a continuous stream of requests. The table below details the resource utilization for the LDAP server in this phase.

The phase length in this table refers to the active period for the LDAP server specific to the mailer operation. According to the description above this would be phase one.

However, if the personalized script makes use of the AUO object, then another request could be made through the LDAP server to the SQL server.

WWW Resource Utilization

In this table, the second phase of the mail operation is detailed. The primary resource utilization for this server is the CPU. It should be noted that the script that is run will greatly affect the overall outcome of these results. The script that was used in this test is contained in Appendix B.

SMTP Resource Utilization

The SMTP server was running an SMTP server emulator (the program is a tool called SuperSink.exe) that did not actually send the message. Refer to the SMTP scaling documents available from Microsoft that address specifications for this component. The transaction rate generated by the mailer directly corresponds to the amount of mail traffic the server should handle.

Appendix A:  Critical Monitoring Counters

All counters noted can be found in the Microsoft® Windows NT® Performance Monitor. These counters will be distributed among the machines in the Personalization and Membership (P&M) service group. The counters in the system and memory objects can be used to monitor capacity.

Physical Disk

Disk Writes/secDisk activity should not sustain maximum transaction rate

Disk Reads/sec

System Object

Context switches/secshould be less than 15,000

%Total Processor should be less than 80%

Processor Object

%Processor Utilization (average)should be less than 80% (for each processor)

Memory Object

Available Bytesshould be greater than 4 MB of RAM

Pages/secshould be less than 1 per second

Site Server Message Builder Service

Messages Sent/sec

SMTP Server

Messages Delivered/sec

Process Object

Private Bytesfor the inetinfo, TMMLBSvc, TMsgBld processes

Appendix B:  ASP Script for AUO

<%@LANGUAGE=VBScript%>
<%
   m_BindAsUserName = "cn=Administrator"
   m_BindAsPassword = "password"
%>
<!-- #INCLUDE FILE="dmheader.asp" -->

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
<html>

<head>
<meta name="GENERATOR" content="Microsoft FrontPage 3.0">
<!-- meta http-equiv="Content-Type" content="text/html"; charset="iso-8859-1" -->
<meta http-equiv="Content-Type" content="text/html">
<title>Direct Mailer Test - Simple AUO page</title>
</head>

<body>
<%
  response.write "Hello " & m_sFirstName & ", where do you want to go today?" & chr(13)
%>
</body>
</html>

HTML Page for Static Tests

<html>

<head>
<meta name="GENERATOR" content="Microsoft FrontPage 3.0">
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<title>Direct Mailer Test - Simple HTML Page</title>
</head>

<body>
Hello, World!
</body>
</html>

Information in this document, including URL and other Internet web site references, is subject to change without notice.  The entire risk of the use or the results of the use of this resource kit remains with the user.  This resource kit is not supported and is provided as is without warranty of any kind, either express or implied.  The example companies, organizations, products, people and events depicted herein are fictitious.  No association with any real company, organization, product, person or event is intended or should be inferred.  Complying with all applicable copyright laws is the responsibility of the user.  Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation.

Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document.  Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.

© 1999-2000 Microsoft Corporation.  All rights reserved.

Microsoft, Windows and Windows NT are either registered trademarks or trademarks of Microsoft Corporation in the U.S.A. and/or other countries/regions.

The names of actual companies and products mentioned herein may be the trademarks of their respective owners.