Microsoft Site Server 3.0 Membership Directory Capacity and Performance Analysis

April 1999

Microsoft Corporation

Definition of Terms

Membership-Specific Terms Meaning
User Individual user connected to a service.
Add Record Record used to add a user to the Membership Directory.
Concurrent User User active on the system; a subset of total users calculated based on user profile.
Attributes Specific information associated with a file or an object.

General Terms Meaning
Pentium Pro equivalent MHz (PPEM) A unit of measure for processor work. A computer with a 200 MHz Pentium Pro processor delivers 200 Pentium Pro equivalent MHz. A computer with two 200 MHz Pentium Pro processors delivers 400 Pentium Pro equivalent MHz

Chapter 1 Overview

This document evaluates the performance and scalability characteristics of the Microsoft® Site Server Membership Directory. The document also demonstrates procedures for identifying these characteristics. Using these procedures, you can determine how user-load impacts hardware resources and which resources are the most likely to be performance bottlenecks. You can then use this information to calculate maximum capacity for a particular hardware configuration, to assess the value of adding additional resources, and to determine which resources would satisfy greater capacity needs.

Analyzing the Membership Directory and LDAP Services

The Membership Directory is a central data repository that stores user records including personal profile information used in personalization and address book information. The persistent contents of the Membership Directory are stored in the Membership Directory database, typically using Microsoft® SQL Server. The Membership Directory contents are accessed using the Lightweight Directory Access Protocol (LDAP) Service, which provides platform-independent access to the Membership Directory. To obtain performance and scalability information, the Membership Directory database and the LDAP Service were analyzed in separate tests using various server configurations.

Projecting Total Resource Cost

A model is designed in which total resource cost is calculated for a variable number of concurrent users. From this model, the following equation is obtained:

C = n × K

C is the resource cost, n is the number of concurrent users, and K is the sum of the costs of each user operation.

Separate equations are created for each type of resource (CPU, memory, disk, and network). Thus, it is possible to intelligently predict the maximum number of users a particular hardware configuration can support, or conversely, the hardware resources required for a given number of users.

Verification Testing

For credibility, the accuracy of these calculations is confirmed by comparing the results of the calculations with a series of verification tests. A test script is created that simulates a typical user behavior defined in the User Profile. The predicted resource costs calculated from the model are charted against actual resource costs generated by running the verification script.

Scalability

Given a model for predicting resource requirements, it is important to test scalability. For example, does investing in a multiple processor system enhance the performance of the LDAP Service? How does partitioning the Membership Directory database across multiple servers affect overall performance, and LDAP Service performance in particular? How well does the LDAP Service scale to multiple servers?

System Configuration

In three test scenarios, two-to-four servers were used in the following configurations:

Note   The Microsoft Access database solution is not part of this document.

LDAP Server1 CPU 1, 2, and 4 X 200 MHz Pentium Pro
  Memory 512 MB of RAM
Disk 4 GB Raid 0
Network 100 BaseT (switched)
Software Microsoft® Windows NT® 4.0 operating system Option Pack, Microsoft ® Internet Information Server

1. One LDAP server was used to test LDAP processor scaling. Three LDAP servers were used to test scaling to multiple servers.

SQL Server2 CPU 4 X 200 MHz Pentium Pro
  Memory 512 MB of RAM
Disk System drive and SQL TempDB: 2 GB—Raid 0
  SQL data device drive: 6 X 9.1 GB—Raid 0
SQL transaction log drive: 9.1 GB—Raid 0
  Network 100 BaseT (switched)
Software SQL Server 6.5 SP33

2. To test Membership Directory database scalability, the database was partitioned across two Microsoft SQL Servers.

3. Microsoft SQL Server was configured with 75 percent available RAM, 300 user connections, no truncate on checkpoint, and 120 minute checkpoint interval.

Membership Directory Description

The Membership Directory is the central repository of user data, including personalization profiles and address book information. The Membership Directory stores both persistent data, which is retained indefinitely, and dynamic data, which is never written to disk, but is maintained in random access memory (RAM) while current. This report only details the static Membership Directory. The Dynamic Directory performance is discussed in the article Microsoft Site Server 3.0 Dynamic Directory Service Capacity and Performance Analysis.

The Membership Directory is accessed using LDAP version 3.0. LDAP is an Internet standard for accessing user information in directories, and allows standard, platform-independent access to the Membership Directory.

The Membership Directory data objects, their optional and required attributes, and the permitted parent-child relationships between objects are defined in the schema, which is stored in the Membership Directory and can be queried by applications. This arrangement allows applications to know the complete up-to-date structure of the Membership Directory at all times, and therefore allows you to make changes to meet your needs without interfering with the operation of system processes and tools.

Small Web sites can effectively deploy a Web server, a Membership Server containing an Active User Object (AUO), an LDAP Service, an Authentication Service, and a Membership Directory on a single computer. For increased scalability, larger sites can deploy the Membership Directory on a dedicated server. Additional performance advantages can be achieved by dedicating one or more servers to the LDAP Service. For environments with heavy transaction volumes, the Membership Directory can be allocated to different physical databases using Microsoft SQL Server.

Transactions

The following table highlights some standard Membership Directory transactions.

Tested Membership Directory Transactions

Transaction Description
Add Record Added a 20-attribute user to the directory.
Base Object Search Searched for a user by cn.
One Level Search Searched for a user using LDAP one level search.
Subtree Search Searched for two users by name.
Modify Add Attribute Added one attribute to an existing user.
Modify Replace Attribute Changed the value of one attribute in an existing user.
Connect and Bind Connected and bound to LDAP Service.

Large User Profile

The following profile captures typical Membership Directory transactions. The database contains 200,000 members in a single partitioned directory service.

Typical Membership Directory Transactions

Elements Value Description
Add Record 3% Add one user with 20 additional attributes to the Membership Directory
Modify Record 12% Adds and replaces records
Base Search 76% All attributes returned
One Level Search 9% All attributes returned
Operations Per Visit 10%  
User Life Cycle 20 Min.

A user with twenty additional attributes is a large user. In contrast, a user added by Auto-Cookie authentication only has three additional attributes.

Summary of Scalability and Performance

Based on the data collected in this document, the following assertions can be made about scaling and performance for the Membership Directory:

Note   As a general rule, groups larger than 750,000 elements should consider partitioning.

Scaling Projections

The LDAP portion of the directory service scales in more than one way. A single server scales to roughly 100 independent connection and anonymous binds per second, but the rate for search scales independently of this rate to rates discussed in the next chapter. The other way an LDAP server scales is in a server farm environment. Adding multiple LDAP servers scales linearly with the number of servers. Additionally, if the level of power is needed, the database back end can be partitioned across multiple SQL Server computers. Thus the LDAP Service can scale to meet almost every need.

The following section gives a quick view of the number of servers required to support five million to 50 million members. The following are the assumptions used:

Chapter 2 Detail Discussion of Scalability and Performance

Processor Usage

The PPEM costs of the transactions for the directory service are not constants. They vary with the rate of the transaction. In some cases, when the transaction rate is low, the costs tend to be higher. This table represents the least squares approximation of the PPEM versus Transaction Rate curves that were measured in the test environment. The relative accuracy of these measurements is confirmed in the following verification section.

The PPEM at a given rate, r on a given processor is: r * (C1 + C2r + C3r2).

Four-processor

Transaction Valid Range for r PPEM C1 PPEM C2 PPEM C3
  Base Object 1 - 180 2.6782 0.0048  
  Connect and Bind 1 - 100 1.7816 0.0569 -0.0001
  Modify Add4 1 - 110 1.4841 -0.0002  
  Modify Replace5 1 - 110 2.9435 -0.0158 0.0001
  One Level 1 - 180 2.0946 -0.0382 0.001
  Subtree Search 1 - 119 3.76    
  Add6 1 - 47 2

4. This maximum was disk-bound on the SQL Server.

5. This maximum was disk-bound on the SQL Server.

6. This maximum was disk-bound on the SQL Server.

7.The Adds measured had 20 additional attributes. Smaller Adds such as anonymous users were measured up to 11 per second.

Two-processor

Transaction Valid Range for r PPEM C1 PPEM C2 PPEM C3
Base Object 1 - 140 2.1223 -0.0087 0.0001
Connect and Bind 1 - 100 1.324 0.0248  
Modify Add8 1 - 106 1.7  
Modify Replace9 1 - 100 1.324 0.0248
One Level 1 - 140 1.3949 0.0082
Subtree Search 1 - 86 3.50  
Add10 1 - 4 2.00

8. This maximum was disk-bound on the SQL Server.

9. This maximum was disk-bound on the SQL Server.

10. This maximum was disk-bound on the SQL Server.

One-Processor

Transaction Valid Range for r PPEM C1 PPEM C2 PPEM C3
Base Object 1 - 100 1.9903 -0.0059  
Connect and Bind 1 - 100 1.74  
Modify Add 1 - 75 1.5
Modify Replace 1 - 80 1.9686 -0.0705 0.0026
One Level 1 - 100 2.2152 -0.0296 0.0008
Subtree Search 1 - 61 3.1    
Add11 1 - 4 5.7469 -2.187 0.4388

11. This maximum was disk-bound on the SQL Server.

Network Usage

The exact contents of the network packets for the LDAP server are specified in the LDAP RFC (RFC 1959). This table describes the typical bit usage of the transactions that were measured. These numbers should be in line with typical usage.

Transaction Size in Bits
Add Record 7,440
Base Object Search 8,480
One level Search 8,448
Subtree Search 17,064
Modify Add Attribute 960
Modify Replace Attribute 960
Connect and Bind 640

Disk Usage

There is no measurable local disk utilization on the LDAP server. The disk utilization occurs on the database server. The SQL Server database disk utilization is discussed in the next section.

SQL Server Usage

The SQL Server usage per transaction plays a large role in determining final system configuration. The table below outlines the CPU and disk usage for the transaction set.

SQL Server resource utilization per transaction type

Transaction PPEM Read data Write data Write log Read tempdb Write tempdb
Base 0.26 4 0 0 0 0
Base Cached 0.22 0 0 0 0 0
Connect and   Bind 0.23 0 0 0 0 0
Modify Add 2 6 2 2 0 0
Modify Replace 2 6 2 2 0 0
One Level Cached 1.2 4 1 0 2 1
One Level 1.4 4 1 0 2 1
Subtree Cached 7 0 0 0 2 2
Subtree 2 4 1 0 4 2
Add 15 40 20 5 0 0

The previous table gives the measured cost per transaction in our measured environment. The disk input/output (I/O) rate that is computed should be assessed, and the final number of spindles in the RAID devices should be computed based on the designated type of RAID setup.

Disk usage is sorted according to the device used to support the database: SQL Server can use a distinct device for storage, transaction log, and the TempDB. Storage contains the data and clustered indices. Transaction log holds the record of changes to the data. The TempDB is a temporary creation for computing the relational work necessary to compute a query.

It should be noted that the disk access numbers in the measured environment have a lower level of caching than might be expected from a different environment such as the environment presented in the authentication service.

Disk Space

The disk space calculations do not take into account the current user rate. The current user rate is the static space that all users in the system take up independent of their activity.

The following assumptions were made for this test:

This table enumerates the user profile for the data.

Attribute Information Distribution Size in Bytes Number of Elements
Average number of attribute values per user     20
String attributes 95% 64 19
String values >255 0.5% 2048 0.1
Integer attributes 2% n/a 0.4
Date and time attributes 2% n/a 0.4
Binary attributes 0.5% 2048 0.1
Percent of specific access control lists (ACLs)     0.5%

Resource Usage Calculations

Resource usage calculations are created to determine what it costs to support a given number of users. Once you obtain this information, you must determine the maximum number of users this configuration of resources can support.

To determine capacity for each resource, equations can be designed that measure resource cost against number of users.

Profile Calculations

Before looking at specific components of the CPU and disk and network calculations, the user profile will be examined. Using the number of concurrent users to determine maximum capacity is difficult if user behavior is not taken into consideration.

Taking the previous user profile, expand it to determine a frequency table.

Transaction Per Second
Add 0.00025
Modify Replace 0.001
Base Object Look-up 0.006333333
One Level Look-up 0.00075

Next, compute the number of transactions per second given a number of concurrent online users. This table does not take into account the length of a business day, or the percentage of potential users who would actively use the service. The table represents the number of concurrent users having the previous profile.

Concurrent Users Add Modify and Replace Base Object Search One Level Search
  3% 12% 76% 9%
500 0.13 0.50 3.17 0.38
1,000 0.25 1.00 6.33 0.75
1,500 0.38 1.50 9.50 1.13
2,000 0.50 2.00 12.67 1.50
2,500 0.63 2.50 15.83 1.88
5,000 1.25 5.00 31.67 3.75
7,500 1.88 7.50 47.50 5.63
10,000 2.50 10.00 63.33 7.50
20,000 5.00 20.00 126.67 15.00
25,000 6.25 25.00 158.33 18.75

Processor Calculations

The first step in determining the processor utilization on the LDAP server is to determine the transaction rate for each of the defined transactions. If this calculation is made with the user profile as previously described, the following table is generated.

Each processor type yields a different processor calculation for each transaction. The results can quickly be calculated in Microsoft® Excel to produce this table.

Four-Processor

Concurrent Users Add PPEM Modify and Replace PPEM Base Object PPEM One Level PPEM Sum of PPEM
500 0.25 1.47 8.53 0.78 11.03
1,000 0.50 2.93 17.15 1.55 22.13
1,500 0.75 4.38 25.88 2.31 33.32
2,000 1.00 5.82 34.69 3.06 44.58
2,500 1.25 7.26 43.61 3.80 55.92
5,000 2.50 14.34 89.62 7.37 113.83
7,500 3.75 21.23 138.04 10.75 173.78
10,000 5.00 27.96 188.87 13.98 235.81
20,000 10.00 53.35 416.25 26.20 505.80
25,000 12.50 65.28 544.38 32.44 654.59

Two-Processor

Concurrent Users Add PPEM Modify and Replace PPEM Base Object PPEM One Level PPEM Sum of PPEM
500 0.25 0.67 6.64 0.52 8.08
1,000 0.50 1.35 13.12 1.05 16.02
1,500 0.75 2.04 19.46 1.58 23.83
2,000 1.00 2.75 25.69 2.11 31.55
2,500 1.25 3.47 31.82 2.64 39.18
5,000 2.50 7.24 61.66 5.35 76.74
7,500 3.75 11.33 91.90 8.11 115.08
10,000 5.00 15.72 124.92 10.92 156.56
20,000 10.00 36.40 332.47 22.77 401.64

One-Processor

Concurrent Users Add PPEM Modify and Replace PPEM Base Object PPEM One Level PPEM Sum of PPEM
500 0.69 0.97 6.24 0.83 8.72
1,000 1.31 1.90 12.37 1.65 17.22
1,500 1.87 2.80 18.38 2.46 25.50
2,000 2.38 3.68 24.26 3.26 33.58
2,500 2.84 4.52 30.03 4.05 41.45
5,000 4.62 8.41 57.11 7.93 78.07
7,500 5.98 11.90 81.23 11.67 110.77
10,000 7.55 15.24 102.39 15.29 140.46

Note that the one- and two-processor formats do not allow for the higher user profiles.

Network Calculations

The table below contains the measured cost per transaction for the measured transactions in bits per second (bps). Plugging the user profile into the measure network bit traffic yields this table.

Concurrent Users Add bps Modify and Replace bps Base Object bps One Level bps Sum of bps
500 930 480 26,853 3,168 31,931
1,000 1,860 960 53,707 6,336 63,863
1,500 2,790 1,440 80,560 9,504 95,794
2,000 3,720 1,920 107,413 12,672 127,725
2,500 4,650 2,400 134,267 15,840 159,657
5,000 9,300 4,800 268,533 31,680 319,313
7,500 13,950 7,200 402,800 47,520 478,970
10,000 18,600 9,600 537,067 63,360 638,627
20,000 37,200 19,200 1,074,133 126,720 1,277,253
25,000 46,500 24,000 1,342,667 158,400 1,596,567

Memory Calculations

The following table represents the memory necessary to support the active number of connections. On the LDAP server all transactions have the same cost, and so the table can be simplified.

Each additional connection costs 5 KB.

Concurrent Users Transactions Per Second KB Memory Necessary
500 4.17 20.83
1,000 8.33 41.67
1,500 12.50 62.50
2,000 16.67 83.33
2,500 20.83 104.17
5,000 41.67 208.33
7,500 62.50 312.50
10,000 83.33 416.67
20,000 166.67 833.33
25,000 208.33 1,041.67

This table does not take into account the root memory overhead of running the LDAP server. The LDAP server caches the directory information tree (DIT) and other objects that will require additional RAM. Thus, results will vary based on the complexity of the directory structure.

SQL Server Calculations

This table represents the overall SQL Server utilization for processor and disk for the user profile sets of transactions. It is once again calculated with the user profile.

SQL Server Processor

Concurrent Users Add PPEM Modify and Replace PPEM Base Object PPEM One Level PPEM Sum of PPEM
500 1.88 1.00 0.82 0.53 4.22
1,000 3.75 2.00 1.65 1.05 8.45
1,500 5.63 3.00 2.47 1.58 12.67
2,000 7.50 4.00 3.29 2.10 16.89
2,500 9.38 5.00 4.12 2.63 21.12
5,000 18.75 10.00 8.23 5.25 42.23
7,500 28.13 15.00 12.35 7.88 63.35
10,000 37.50 20.00 16.47 10.50 84.47
20,000 75.00 40.00 32.93 21.00 168.93
25,000 93.75 50.00 41.17 26.25 211.17

Disk

This disk utilization is the calculated disk throughput at one arbitrary data point, 500 users.

Transaction Read Data Write data Write log Read tempdb Write tempdb Total Read Total Write
Add 2.50 2.50 0.63 - - 2.50 3.13
Modify and Replace 1.50 1.00 1.00 - - 1.50 2.00
Base Object 6.33 - - - - 6.33 -
One Level 0.53 0.75 - 0.38 0.38 0.38 1.13

Disk Space

Using the previous profile, data and indices of this size is generated.

Users in Directory MB of Device Space Necessary
10,000 108
50,000 540
100,000 1080
200,000 2,160
500,000 5,400
1,000,000 10,800

Site Configuration Example

This example illustrates how to determine an optimal configuration for a given number of users.

Sample Processor Calculations

The first step in determining the sample configuration is the user profile. The user profile is determined by modeling typical user behavior. The best possible way to do this calculation is to sample an existing similar configuration for exact transaction rates. However, if this configuration is not available, the analysis can be done to predict users’ behavior.

For this sample, the user profile that was described for typical per-user transaction rates is used. Then, the number of users that will be supported is determined.

For a medium Internet Service Provider (ISP) the calculation takes into account 50,000 current users, but the system to build out should be ready to grow to accommodate 200,000 users.

Assuming that, at most, two percent of users are online at any time, the average user-load is 4,000.

The calculations should be done at peak load time, which is roughly two times average for a maximum concurrent user load of 8,000 users.

The user load multiplied by the frequency generates this transaction rate table.

Concurrent Users Add Xact /sec Modify/Replace Xact /sec Base object Xact /sec One Level Xact /sec Total Xact /sec
8000 2.00 8.00 50.67 6.00 66.67

To carry out only Base Object search calculations, use the formula from the PPEM table for four processors:

PPEMBaseObject= 2.6782 + .0048r = 2.92

To achieve actual CPU usage, multiply PPEM by rate to yield 148.02.

This table expands all of the transaction across all of the processors.

Number of Processors Add PPEM Modify and Replace PPEM Base Object PPEM One Level PPEM PPEM Total % of CPU
4 4.00 22.59 148.02 11.41 186.01 23
2 4.00 12.18 98.20 8.66 123.05 31
1 6.26 12.57 85.70 12.40 116.92 58

The final column calculates the expected CPU rate for the server at this load by dividing the PPEM by the CPU capacity of the system.

Memory Calculation

66.67Peak transaction rate* 5 KB per connection = 333.3 KB

The memory calculations show that the server should have an additional 333.3 KB of memory for handling the peak LDAP traffic.

Network Calculation

2.0Add Peak Transaction Rate * 7440Add Network bit cost = 14,880 bps.

This calculation represents the network traffic that the LDAP server itself will generate in bps.

Add bps Modify and Replace bps Base Object bps One Level bps Total traffic bps
14,880 7,680 429,653 50,688 510,901

SQL back end the processor load

SQL PPEMAdd =15

15 * 2.0Add Peak Transaction Rate = 30 PPEM for Add.

Add PPEM Modify and Replace PPEM Base object PPEM One Level PPEM Total PPEM % of CPU
30.00 16.00 13.17 8.40 67.57 8

SQL Disk Utilization

Multiply the transaction rate by the disk cost per device to generate this table.

Transaction Add Modify and Replace Base Object One Level Total Disk Xact/sec
Read Data 40 24 101.33 8.4 173.73
Write Data 40 16 0.00 12 68.00
Read Log 0 0 0.00 6 6.00
Write Log 10 16 0.00 0 26.00
Read tempdb 0 0 0.00 0 0.00
Write tempdb 0 0 0.00 6 6.00
Total Read 40 24 101.33 14.4 179.73
Total Write 50 32 0 18 100.00

On the primary data device, the SQL Server system must support 173.73 read transactions per second and 68 write transactions per second.

Calculating the RAID array to support this rate occurs as follows:

RAID Level 0 =ceiling ((ReadMax/sec + WriteMax /sec)/60average maximum disk I/O speed) (Maximum disk I/O is the average rate for a 7200 rpm drive.)

RAID Level 1 = ceiling (ReadMax/sec + (2 * WriteMax /sec))/60average maximum disk I/O speed)

RAID Level 5 = ceiling ((ReadMax+ (4*WriteMax ))/60average maximum disk I/O speed)

ReadMax is the necessary maximum number of disk reads per second. WriteMax is the necessary maximum number of disk writes per second.

The denominator of 60 assumes a 7200 rpm, 4.3-gigabyte disk drive. Different controllers and faster disk drives will change performance. Refer to hardware benchmarks for more information.

RAID Level 0 for data disk = ceiling (250 + 100)/60 = ceiling(5.83) = 6

RAID Level 1 for data disk = ceiling (250 + 2 * 100)/60 = ceiling(7.5) = 8

RAID Level 5 for data disk = ceiling (250 + 4 *100)/60 = ceiling(10.8) = 11

Disk Space Calculation

The cost per user in this model is a constant, thus 200,000 users in the directory equals 2060 MB of disk space on the SQL Server device.

The transaction log should be large enough to handle all of the activity for the period between maintenance. The SQL Server administrative staff should determine this calculation.

Appendix A:  Testing Methodology

Calculating the Per Transaction Cost

To calculate the PPEM from observed data, the load was simulated on a per-transaction basis; in this case, Auto-Cookie cached with creation. An automated script was run that cycled through a changing user load. The values were captured in a perfmon.exe log file. The log file was then exported to a tab-delimited text file and imported into Microsoft Excel. Then, using a macro, the log was distributed and averaged.

Each row in the chart represents a several-minute run of the transaction load. Many values were captured, but these rates were the most important for calculating PPEM. To establish a valid range of samples, context switches were observed. The context switch rate must be below a certain threshold to be considered running optimally. However, the processor should be exercised across a full range.

The required calculation for this table is a multiplication of the total processor time by the clock capacity of the system. In this case the equation is 4 times 200. The product is then divided by the number of authentications per second. Next, the overall PPEM is calculated by taking a least squares approximation of the PPEM value at the transaction rate. This result generates the formulas presented in Chapter 1 for each transaction.

Bytes received /sec Bytes sent /sec Static search /sec % Total processor time Context switches /sec PPEM
1 1 1      
Site Server LDAP Service Site Server LDAP Service Site Server LDAP Service System System
233.90 2,817 2.89 1.11 199.24 1.54
488.31 5,860 6.01 1.86 233.76 1.24
982.53 11,760 12.06 4.08 297.95 1.35
1,968.59 23,569 24.18 11.08 409.24 1.83
3,981.96 47,604 48.88 23.21 827.17 1.90
7,896.08 94,424 96.95 49.72 2,219.16 2.05
10,637.17 127,130 130.55 77.46 7,598.68 2.37
11,590.19 138,494 142.33 95.98 15,053.74 2.70
11,500.97 137,432 141.23 97.54 15,525.24 2.76
11,430.89 136,767 140.39 97.56 16,700.77 2.78

The data points are plotted and then a polynomial fitting is applied using least squares to generate the line. The values of the lines is 1.37537 + .009077 * T.

Verification Sample:

The following chart represents measured CPU utilization versus actual CPU utilization for LDAP running on a two-processor computer. The calculated number represents the processor load calculated using the formulas with the rates taken from the real performance counters at each data point. The other line is the actual CPU load on the system.

Verifying Actual versus Calculated

The next two charts show the SQL Server verification. The same calculated transaction rates are pulled from the real performance counters and calculated using the PPEM rates in the SQL Server table.

The disk transaction rates for SQL Server are calculated using the previous table, with one important distinction. The measured disk load on a SQL Server system should be doubled to account for additional SQL Server overhead when running in a mixed environment.

The disk writes on the SQL Server are sporadic due to SQL Server's maintenance schedule and other hard-to-control activities. This chart represents a real scenario. The disk I/O lag that is evident in all numbers should be expected in every user’s final implementation.

Appendix B:  LDAP Service Processor Scaling

Every user that accesses the LDAP server increases the hardware utilization on that server. This chart shows the maximum rate of the transactions on a server with one, two and four Pentium Pro 200 MHz processors.

Maximum transaction rates per processor

Transaction 4 Processors 2 Processors 1 Processor
Base Object Search 180 140 100
One Level Search 180 140 100
Subtree Search 119 86 61
Modify Add 110 106 75
Modify Replace 110 100 80
Add 20 Attribute User 4 4 4
Connect and Bind 100 90 100

Adding More LDAP Servers

Another way to scale the LDAP Service is to add more computers running the LDAP Service independently. This table shows that adding more LDAP servers can increase the total number of transactions per second that a group of servers can handle. All three of these LDAP servers were making requests against one SQL Server. The costs were linear.

The table details reveals that the average CPU and transactions per second are equally balanced and that the cost per transaction is roughly the same as it is in a non-distributed environment.

Transaction costs using multiple LDAP servers

Scalability Transaction Tests Number of LDAP Servers Xact/sec %CPU PPEM SQL I/Os
Base object look-up in 200,000 member DS 1 156 46.5 2.38 158
  2 150, 143 41.7, 50.5 2.22, 2.83 300
3 150, 150, 183 53, 55, 60.5 2.84, 2.93, 2.64 487

Transaction Scaling Summary

This chart shows how the transaction rate levels off for the search type transactions. In contrast, however, the modify scripts will decay in performance as the SQL Server falls behind while writing the transactions to disk.

Partitioning the Membership Directory

Partitioning the Membership Directory allows the administrator to manage smaller databases within SQL Server. Partitioning allows for more reasonable manageability in both data and index size. Furthermore, by breaking up the data into separate databases, the locking inherent in SQL Server’s implementation of indices is avoided.

You can, thus, speed up the disk bound transactions that are typically the modify and the add by partitioning to multiple disk systems.

Partitioning the Membership Directory

Transaction LDAP Server SQL Server 1 SQL Server 2
  CPU Xact/sec PPEM Disk Read/sec Disk Write/sec Disk Read/sec Disk Write/sec
Base object search - cached 70.5 109.7 5.14 0 0 0 0
One-level search - cached 76.2 85.8 7.10 0 0 0 0
Subtree search - cached 84.5 61.15 11.06 122.45 129.43 0 .01
Base object search - random 88.3 110.8 6.38 120.7 .13 89.6 0
One-level search - random 77.9 104.2 5.98 114.5 55.1 86.0 42.9
Subtree search - random 70.9 63.5 8.92 230.6 101.6 160.5 85.1
Add new 20 attribute user 2.48 4.65 2.73 25.8 126.5 45.7 122.4
Modify replace 23.2 66.5 2.79 72.6 165.8 39.0 147.1
Modify add 21.0 63.16 2.66 41.6 142.2 24.1 124.3

Appendix C:  Critical Monitoring Counters

All counters noted can be found in Microsoft® Windows NT® Performance Monitor. These counters will be distributed among the computers in the Personalization and Membership (P&M) service group. The counters in the system and memory objects can be used to monitor capacity.

Site Server LDAP Service

Connection Current

Static Add per second

Static Modify per second

Static Search per second

Site Server Authentication Service

Auto-Cookie Authentication Successes per second

Clear Text Authentication successes per second

DPA successes per second

HTML Forms Authentication successes per second

Requests served from cache

SQL Server

SQL Server I/O transactions per second

Physical Disk

Disk Writes per secondDisk activity should not sustain maximum transaction rate

Disk Reads per second

ASP Object

Requests per second

Current requests

Web Object

NonAnonymous Users per sec

Get requests

System Object

Context switches per secondshould be less than 15,000.

%Total Processor should be less than 80 percent.

Processor Object

%Processor Utilization (average) should be less than 80 percent (for each processor).

Memory Object

Available Bytes should be greater than four megabytes.

Appendix D:  Membership Directory Transactions

Scripts

Base Object Script

CONNECT
LOOP RANDNUMBER(15,30)
 SEARCH BASE RETURN ALL dn=cn=tstRANDNUMBER(1,200000),ou=members,o=microsoft; filter=(objectclass=*);
 SLEEP RANDNUMBER(200,350)
ENDLOOP
QUIT

One Level Script

CONNECT
LOOP RANDNUMBER(15,30)
 SEARCH ONELEVEL RETURN ALL dn=ou=members,o=microsoft;filter=(cn=tstRANDNUMBER(1,200000));
 SLEEP RANDNUMBER(200,350)
ENDLOOP
QUIT

Add One User Script

CONNECT
BINDSIMPLE dn=cn=administrator,ou=members,o=microsoft;password=pw;
LOOP 10
 ADD dn=cn=RANDALPHA(20),ou=members,o=microsoft;objectClass=member;guid=RANDNUMERIC(32);telephoneNumber=RANDNUMERIC(10);street=RANDALPHA(60);postOfficeBox=RANDNUMERIC(6);postalCode=RANDNUMERIC(9);postalAddress=RANDNUMERIC(60);st=RANDALPHA(2);givenName=RANDALPHA(40);sn=RANDNUMERIC(9);c=RANDALPHA(2);language=RANDALPHA(5);userComment=RANDNUMERIC(20);homepage=RANDALPHA(8);description=RANDALPHA(8);userpassword=password;mail=RANDALPHA(20)@corp.com;mailerNoEmail=0;mailerNumNDR=0;mailerEmailInvalid=0;
 sleep 1000
ENDLOOP
QUIT

Connect and Bind Script

CONNECT
BINDSIMPLE dn=cn=administrator,ou=members,o=microsoft;password=pw;
SLEEP 200
QUIT

Subtree Search Script

CONNECT
 LOOP RANDNUMBER(15,30)
 SEARCH SUBTREE RETURN ALL dn=o=microsoft; filter=(|(cn=tstRANDNUMBER(1,200000))(cn=tstRANDNUMBER(1,200000)));
 SLEEP RANDNUMBER(200,350)
 ENDLOOP
QUIT

Modify Add Script

CONNECT
LOOP RANDNUMBER(15,30)
 MODIFY userComment=RANDALPHA(32): A;dn=cn=tstRANDNUMBER(1,200000), ou=members, o=microsoft;
 SLEEP RANDNUMBER(200,350)
ENDLOOP
QUIT

Modify Replace Script

CONNECT 
LOOP RANDNUMBER(15,30)
 MODIFY postalcode=RANDNUMERIC(9):R;dn=cn=tstRANDNUMBER(1,200000), ou=members, o=microsoft;
 SLEEP RANDNUMBER(200,350)
ENDLOOP
QUIT

Verify Test

CONNECT
 BINDSIMPLE ANONYMOUS
 LOOP 100
 %29 SKIP 2
 SEARCH BASE RETURN ALL dn=cn=tstRANDNUMBER(1,200000),ou=members,o=microsoft; filter=(objectclass=*);
 SLEEP RANDNUMBER(200,350)
 %92 SKIP 2
 SEARCH ONELEVEL RETURN ALL dn=ou=members,o=microsoft;filter=(cn=tstRANDNUMBER(1,200000));
 SLEEP RANDNUMBER(200,350)
%97 SKIP 2
 ADD dn=cn=RANDALPHA(20),ou=members,o=microsoft;objectClass=member;guid=RANDNUMERIC(32);telephoneNumber=RANDNUMERIC(10);street=RANDALPHA(60);postOfficeBox=RANDNUMERIC(6);postalCode=RANDNUMERIC(9);postalAddress=RANDNUMERIC(60);st=RANDALPHA(2);givenName=RANDALPHA(40);sn=RANDNUMERIC(9);c=RANDALPHA(2);language=RANDALPHA(5);userComment=RANDNUMERIC(20);homepage=RANDALPHA(8);description=RANDALPHA(8);userpassword=password;mail=RANDALPHA(20)@corp.com;mailerNoEmail=0;mailerNumNDR=0;mailerEmailInvalid=0;
 SLEEP RANDNUMBER(200,350)
%89 SKIP 2
 MODIFY postalcode=RANDNUMERIC(9):R;dn=cn=tstRANDNUMBER(1,200000), ou=members,o=microsoft;
 SLEEP RANDNUMBER(200,350)
ENDLOOP
QUIT

Appendix E:  MDAC 2.0 Capacity and Performance

Processor Usage

The PPEM costs of the transactions for the directory service are constant due to the scalability enhancements in the MDAC 2.0 code. This table represents the average of the PPEM values that were measured in the test environment.

The PPEM at a given rate, r on a given processor, is: r x C1.

Four-processor

Transaction Valid Range for r PPEM C1
Base Object 1 – 354 2.237
Connect & Bind 1 – 188 2.979
Modify Add12 1 – 110 1.882
Modify Replace13 1 - 110 1.937
One Level 1 - 280 1.644
Subtree Search 1 – 167 3.223
Add14 1 - 415 3.238

12. This maximum was disk-bound on the SQL Server.

13. This maximum was disk-bound on the SQL Server.

14. This maximum was disk-bound on the SQL Server.

15. The Adds measured had 20 additional attributes. Smaller Adds such as anonymous users were measured up to 11 per second.

Two-processor

Transaction Valid Range for r PPEM C1
Base Object 1 – 230 1.704
Connect & Bind 1 – 188 1.979
Modify Add16 1 - 106 1.697
Modify Replace17 1 - 100 1.579
One Level 1 – 268 1.448
Subtree Search 1 – 105 3.048
Add18 1 - 4 1.610

16. This maximum was disk-bound on the SQL Server.

17. This maximum was disk-bound on the SQL Server.

18. This maximum was disk-bound on the SQL Server.

One-Processor

Transaction Valid Range for r PPEM C1
Base Object 1 – 131 1.507
Connect & Bind 1 – 122 1.639
Modify Add 1 - 75 1.462
Modify Replace 1 - 80 1.297
One Level 1 - 100 1.278
Subtree Search 1 - 67 2.955
Add19 1 - 4 1.390

19. This maximum was disk-bound on the SQL Server.

MDAC 2.0 Enhanced Processor Calculations

The results can quickly be calculated in Microsoft® Excel to produce this table. This table and the accompanying chart represents the new scalability results.

Four-Processor

Concurrent Users Add PPEM Modify and Replace PPEM Base Object PPEM One Level PPEM Sum of PPEM
500 0.40 0.97 7.08 0.62 9.07
1,000 0.81 1.94 14.17 1.23 18.15
2,000 1.62 3.87 28.34 2.47 36.29
5,000 4.05 9.69 70.84 6.17 90.74
7,500 6.07 14.53 106.26 9.25 136.10
10,000 8.10 19.37 141.68 12.33 181.47
15,000 12.14 29.06 212.52 18.50 272.21
20,000 16.19 38.74 283.35 24.66 362.94
25,000 20.24 48.43 354.19 30.83 453.68
30,000 24.29 58.11 425.03 36.99 544.42
40,000 32.38 77.48 566.71 49.32 725.89

Two-Processor

Concurrent Users Add PPEM Modify and Replace PPEM Base Object PPEM One Level PPEM Sum of PPEM
500 0.20 0.79 5.40 0.54 6.93
1,000 0.40 1.58 10.79 1.09 13.86
2,000 0.81 3.16 21.58 2.17 27.72
5,000 2.01 7.90 53.96 5.43 69.30
7,500 3.02 11.84 80.94 8.15 103.95
10,000 4.03 15.79 107.92 10.86 138.60
15,000 6.04 23.69 161.88 16.29 207.89
20,000 8.05 31.58 215.84 21.72 277.19
25,000 10.06 39.48 269.80 27.15 346.49

One-Processor

Concurrent Users Add PPEM Modify and Replace PPEM Base Object PPEM One Level PPEM Sum of PPEM
500 0.17 0.65 4.77 0.48 6.07
1,000 0.35 1.30 9.54 0.96 12.15
2,000 0.70 2.59 19.09 1.92 24.29
5,000 1.74 6.49 47.72 4.79 60.74
7,500 2.61 9.73 71.58 7.19 91.11
10,000 3.48 12.97 95.44 9.59 121.47
15,000 5.21 19.46 143.17 14.38 182.21

Note that the one- and two-processor formats do not allow for the higher user profiles.

This chart is a graphical representation of the new data in the preceding tables.

Information in this document, including URL and other Internet web site references, is subject to change without notice.  The entire risk of the use or the results of the use of this resource kit remains with the user.  This resource kit is not supported and is provided as is without warranty of any kind, either express or implied.  The example companies, organizations, products, people and events depicted herein are fictitious.  No association with any real company, organization, product, person or event is intended or should be inferred.  Complying with all applicable copyright laws is the responsibility of the user.  Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation.

Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document.  Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.

© 1999-2000 Microsoft Corporation.  All rights reserved.

Microsoft, Windows and Windows NT are either registered trademarks or trademarks of Microsoft Corporation in the U.S.A. and/or other countries/regions.

The names of actual companies and products mentioned herein may be the trademarks of their respective owners.