n my February 1999 Advanced Basics column, I wrote about the concurrency model of Microsoft® Internet Information Server (IIS) and how to optimize performance and increase scalability. I discussed techniques for state management, but limited the discussion to Web sites that are based on a single computer running IIS. This month I'll extend my discussion of scalability by introducing the concept of load balancing. First, I'll discuss the motivation for load balancing and discuss a few options for adding extra Web servers to your site. Then I'll look at some important design issues to consider when you want to run IIS-based applications in a Web farm environment.       First, let's review a couple of key points about scalability and IIS. The IIS/ASP framework conducts a thread-pooling scheme and maintains a request queue to accommodate peak traffic times. After the default installation, IIS is configured to allocate 10 threads per processor to service incoming ASP requests. If you need to maintain state on a per-user basis across requests, you can accomplish this in an IIS application by using an ASP Session variable.       Visual Basic® objects are apartment threaded. If you create instances of Visual Basic objects from ASP scripts, a basic rule of thumb is to make sure you release these objects at the end of every request. If you assign a Visual Basic object to an ASP Session variable, you will pin each user to a specific thread. This practice significantly decreases the scalability of an IIS application and should be avoided.       While holding onto Visual Basic objects across requests is a no-no, it is far more acceptable to store non-object-based state information inside ASP Session variables. For instance, you can use ASP Session variables to stash variants, arrays, or the contents of a property bag when you're creating an application that needs to maintain state across requests. This approach allows you to build state-based applications such as those that use shopping carts.       The one problem with using ASP Session variables is that you must make the assumption that you're processing each incoming HTTP request for a particular user with the same IIS computer, an assumption you can't make when running a Web farm. In this month's column, I'll look at the issues that come into play when you can't make that assumption. Increasing Scalability through Load Balancing       A Web site is like an aspiring Hollywood actor. In the beginning of its career, before a Web site has been discovered by the masses, it lives in obscurity. Its biggest problem is the paranoia that it will live in obscurity for its entire existence. However, once a Web site becomes famous, it has a new set of challenges. The volume of incoming requests increases dramatically. Moreover, the fans of a famous Web site have lofty expectations for the site's performance, and are highly critical when these expectations are not met. Unfortunately, these new challenges are sometimes more than a Web site can handle. Some Web sites respond to fame by going up in smoke—and they never recover. Other sites are able to handle the pressures of fame more gracefully. They go on to become the places that thousands of users return to again and again.       On their path to fame, some Web sites experience a user base that grows from hundreds to thousands to hundreds of thousands. Other sites (often the children of already popular Web sites) are famous from the first day they are launched. To meet the demands and the expectations of their fans, these famous Web sites must scale accordingly.       A simple definition of scalability is the ability of a system to accommodate a growing number of users and to give each user satisfactory levels of responsiveness. To reach this goal, a Web site must be able to supply an adequate number of processing cycles per second to handle the volume of incoming requests. As you might expect, more users require more processing cycles. So the question becomes: where do all these processing cycles come from?       At first, you might be able to increase your site's scalability with a single-server solution. You can upgrade your Web server to a computer with a faster processor and/or multiple processors. The new computer will handle a larger user base than the previous one.       However, at some point a single-server solution simply doesn't cut it. You can only scale up so far. Moreover, the computers at the high end of that market are prohibitively expensive. Once you hit a certain threshold, cost-effective scalability requires the use of multiple processors spread across multiple servers. This is where load balancing comes into play. You need to distribute the workload of incoming HTTP requests across several computers. I'm going to look at several different approaches to solving this problem.       Imagine you're designing a Web-based application that makes heavy use of Microsoft Transaction Server (MTS) objects built with Visual Basic. These MTS objects contain your business logic and data access code. You're creating these objects with ASP scripts and releasing them at the end of each request. Where's the best place to balance your load with this type of design?       One possible place to load balance is the point at which your ASP scripts activate COM objects, as shown in Figure 1. This style of activation-time load balancing is currently planned as a feature of the infrastructure of Windows® 2000 and COM+.

Figure 1: Windows 2000 Load Balancing

If you can't wait for Windows 2000 to ship, you can produce similar results by creating a custom broker object on the IIS server. For instance, an ASP script could create and call a local broker object that supports a method like CreateUserManagerObject. The implementation of this method could call the Visual Basic 6.0 function CreateObject and pass the name of the MTS server on which to activate the UserManager object. Your algorithm could rotate activation requests across a set of servers. This approach makes it possible to distribute the load across a set of MTS-enabled computers.       It turns out that this style of load balancing works pretty well inside a LAN environment with remote COM clients, but isn't that good for a Web-based application. The problem with this approach is that all incoming HTTP requests and ASP processing is handled by a single computer. This IIS computer becomes both a performance bottleneck and a single point of failure for your site. For a Web-based application, it's better to address load balancing at the point at which the HTTP request arrives at the site. This means that you need a technique to distribute incoming HTTP requests across a set of IIS computers. A site that uses this approach is known as a Web farm. Web Farming       Web designers have devised quite a few techniques to distribute HTTP requests across a set of servers. One simple approach is to design a Web site with a dedicated routing server, as shown in Figure 2.

Figure 2: Using a Dedicated Routing Server

The routing server usually has a well-known Domain Name Service (DNS) name (such as MyServer.com) and a dedicated IP address. The other servers in the farm have their own dedicated IP addresses and can optionally have a DNS names as well. When a user's initial request reaches the routing server, the request is redirected to one of the other servers in the farm. You can redirect your users from the routing server using the Session_OnStart event in the global.asa file.       Say you want to redirect each new user to one of three different servers in a farm:

Sub Session_OnStart Const SERVER_COUNT = 3 Dim sServer, sURL Randomize Select Case (Fix(Rnd * SERVER_COUNT) + 1) Case 1 sServer = "MyServer1" Case 2 sServer = "MyServer2" Case 3 sServer = "MyServer3" End Select sURL = "http://" & sServer & "/MyApp/default.asp" Response.Redirect sUR End Sub

Once a user is redirected to a particular server, a session is created and the user sends all future requests to the same server. For this reason, you can think of this as a session-based load balancing technique. Note that if you use this technique, you must make sure that a user's initial request is for a page with an .asp extension. The Session_OnState event will not be fired when the request is for a file with an .htm or .html extension, since IIS doesn't expect them to contain ASP code.       The code shown previously uses a random algorithm for redirection, but you could design a more elaborate load balancing mechanism. For instance, each server in the farm could send performance data back to the routing server. If each server periodically transmits a count of active sessions to the router, the load balancing code could redirect each new user to the server with the fewest number of current users.       Round-robin DNS is another common session-based load balancing technique. With a round-robin DNS, each logical DNS name (such as MyServer.com) maps to several IP addresses. When a browser attempts to resolve the DNS name, the DNS server sends back one of the addresses from the list. The DNS server rotates the addresses in order to distribute a set of users across a set of servers. Once a browser resolves a DNS name into an IP address, it caches the IP address for the duration of the user's session. Round-robin DNS is slightly faster than the redirection technique shown previously, yet it produces the same results. Different users are given different IP addresses to balance the load.       If you are going to set up a Web farm with one of these session-based load balancing techniques, you should write your pages and code in terms of relative URLs. For instance, you should use URLs such as \MyOtherPage.asp instead of absolute URLs like http://myserver/MyOtherPage.asp, which contain a server's DNS name or IP address. This will ensure that each user continues to send requests to the same server once a session has been started.       While both of these forms of session-based load balancing are easy to set up, they have a few notable limitations. First, load balancing is only performed once for each client at the beginning of a session. Second, it's possible for the load balancing scheme to get a little skewed. For instance, all the users that have been sent to MyServer1 may go to lunch while all the users who have been sent to MyServer2 continue to send requests. In this case, one server could become overloaded while another server is sitting by idly.       A more significant problem with session-based load balancing is that it exposes the IP addresses of the servers in the farm to the client-side browser. What happens when a server crashes or is taken offline? Your balancing algorithm needs to account for this as soon as possible, but doing so can be problematic. If you're passing out bad IP addresses, your users will start to receive "server not available" errors. In a round-robin DNS system, it still can take as long as 48 hours to fix the problem once you've discovered that one of your servers has crashed. This is due to the fact that the changes to your IP address mappings need to be propagated to DNS servers throughout the Internet.       To make things worse, users often add pages to their Favorites list. A user might attempt to reach a page that they put in their Favorites list last week on a server that crashed this morning. If you're using the redirection technique, an attempt to locate a Favorites page can result in a "server not available" error.       The load balancing techniques I've shown so far can compromise the fault tolerance of your site. There are more sophisticated approaches to load balancing that can significantly improve your system's availability. The solution lies in exposing a single IP address to every user. Designing a Better Web Farm       As you have seen, exposing multiple IP addresses for a single Web site can compromise both availability and load distribution. It's better to expose a single IP address that maps to several physical servers. As it turns out, this is a very difficult problem to solve because it requires low-level networking code to reroute incoming IP packets across a set of servers. Most companies decide to buy a solution rather than roll their own.       Let's take a quick look at two available off-the-shelf solutions. LocalDirector is a hardware-based solution from Cisco Systems. The Windows Load Balancing Service (WLBS) is a software-based solution from Microsoft that's part of Windows NT Server 4.0 Enterprise Edition. Many other vendors offer similar products. The marketplace is quite competitive and you should do some research to determine who can offer you the best combination of price and performance.       LocalDirector is a proprietary piece of hardware with an embedded operating system that can load balance incoming HTTP requests, as shown in Figure 3. LocalDirector listens for incoming requests on a single virtual IP address and is able to redirect them across a set of IIS servers. Each physical server in the farm has its own unique IP address. However, unlike the load balancing techniques discussed earlier, the IP addresses of the physical servers are never seen by users. Load balancing is performed every time a request comes in across the virtual IP address. This is a style known as request-based load balancing.

Figure 3: LocalDirector Routing

The WLBS provides a software-based solution for request-based load balancing. Don't confuse the WLBS with either the COM+ load balancing service or the Microsoft Cluster Server (known in its beta incarnation by its codename, Wolfpack). The WLBS is based on Convoy Cluster software purchased by Microsoft from Valence Research Inc. It can be used by many types of applications that rely on IP traffic, but in this column I'll focus on using the WLBS to create a Web farm. You can download the WLBS installation files from the Microsoft Web site (http://www.microsoft.com/ntserver/ ntserverenterprise/exec/feature/wlbs/default.asp) and install it on any computer running Windows NT Server 4.0 with an Enterprise Edition license.

Figure 4: WLBS Routing

Unlike LocalDirector, the WLBS doesn't require a proprietary piece of hardware. Instead, the WLBS is installed as a Windows device driver on each machine in the farm, as shown in Figure 4. The WLBS can accommodate a Web farm of up to 32 servers. The WLBS has an advantage over LocalDrector in that there isn't one piece of hardware that represents a singe point of failure. If you're using LocalDirector you can buy two hardware units instead of one to improve fault tolerance, but that starts to get expensive. The argument in favor of a hardware-based solution is that it's a one-time cost. With a software-based solution, you usually have to pay additional licensing fees every time you add another server to your Web farm.       When you set up a Web farm using the WLBS, each server is in constant communication with all the others. They exchange performance statistics and divide the responsibilities of handling incoming requests. You should note that every incoming request is seen by every server in the farm, and the WLBS has a proprietary algorithm to determine which server will handle each request. A full discussion of how the WBLS distributes the load between the servers is beyond the scope of this column.       The low-level plumbing details of the WLBS and LocalDirector are very different, but from a high-level perspective they produce the same results. Each user makes a request using a virtual IP address, and the request is routed to one of many servers in a Web farm. Both the WLBS and LocalDirector provide effective request-based load balancing that has many advantages over session-based solutions. Request-based load balancing is more granular because the load balancing algorithm is run far more frequently. This results in a more even distribution of requests across servers.       Request-based load balancing also provides higher levels of fault tolerance. With either LocalDirector or the WLBS, an administrator can issue a command to take a server offline. This allows an administrator to perform maintenance or to upgrade each server in the farm without an interruption in service. Both LocalDirector and the WLBS can also detect when a server has crashed and avoid directing future requests to an unavailable IP address.       Now that I've examined the advantages of request-based load balancing, it's time to discuss one of the main disadvantages. Managing state in a Web farm with request-based load balancing becomes more complicated because you cannot assume that a user's requests will all be serviced by the same IIS computer. The main side effect of this requirement is that you should not attempt to maintain state using ASP Session or Application variables. Instead, you must design a more sophisticated approach to formalize the same concept of a user session. Maintaining State in a Web Farm       If you're designing a Web application where you don't need to maintain state across requests, you don't have as many concerns. This might be the case if your users are simply browsing for information. However, if you're creating a Web application that involves something like a shopping cart, you must carefully consider how you want to build up and store state for each user across requests.       Start the design phase by asking yourself a simple question: where are the places you can maintain user-specific state information in a Web application? There are three places where it might be appropriate to store this data. First, you can store state in the client tier, inside the HTML you send to the browser. Second, you can store state in the middle tier, inside IIS. Third, you can store state in a backend database such as SQL Server™.       When you're designing a Web application for a site based on a single IIS computer, storing state inside IIS using ASP Session variables is usually the easiest and most attractive solution. If you have a Web farm that uses session-based load balancing, you can also store your state inside IIS. However, when you have a Web farm that uses request-based load balancing, storing data inside IIS doesn't work.       When you can't store state information in the middle tier, you must resort to storing it on the client or in a DBMS. The main advantage of storing state on the client is that it's fast; you don't need to make time-consuming trips to the backend database. The main advantages of storing state data in the backend database is that it's durable and you can store as much state information as you need. Storing state on the client has size limitations and raises security issues. You might want to avoid storing potentially sensitive information on the user's computer. Moreover, the user might have browser security settings that prevent you from writing to their hard disk.       You should note that when you store state in a backend database, you'll probably still be required to maintain at least a minimal amount of state information on the client. For example, it's common to explicitly pass a customer ID or some other type of primary key between the client and IIS with every request. This will allow you to map a user to one or more records in your database. Sometimes it's possible to get away with using something that HTTP sends anyway, such as the client's IP address or user name. As you can see, it's possible to maintain a little client-side state, but still keep the majority of your data in a database like SQL Server where it's safe and secure.       During the design phase you should ask yourself a few questions when deciding whether to write your user-specific state out to a backend database. How much of a performance cost is there when you access the backend database? Does the backend database represent a significant performance bottleneck? If the answer to this question is yes, you might consider eliminating or reducing trips to the backend. You should conduct some benchmarking and compare the response times of requests that involve database access against the times of requests that do not.       While it's usually possible to reduce the number of trips to the database, it's often impossible to avoid them completely. Here are a few more questions you should consider during the design phase: how costly is it if user-specific state gets lost or corrupted? Do you want user-specific state information to be available when the user moves to a different machine? Is your user-specific state sensitive to the point where you need to hide it from your users? Do you need to store so much user-specific state that it's not practical to store it all on the client? Answering these questions will usually indicate how much you need to write to your database.       You can often find a suitable compromise. For example, you might access the database when the user makes their initial request to validate a user name and password and to retrieve a set of preconfigured user preferences. Then, as the user starts to add items to their shopping cart, you can track their actions by storing state on the client. Finally, when they purchase the selected items, your application can make a second trip to the database to record the transaction. In this scenario, the user's session might involve 20 to 30 requests, only two of which require a trip to the backend database. Passing State       Whether you're maintaining lots of user-specific state data on the client or just a primary key, you need to know how to pass data back and forth between the browser and IIS. Let's look at three common techniques to accomplish this.       The easiest way to store state on the client is with cookies. The ASP Response and Request objects make writing and retrieving a cookie value relatively simple. For instance, if you're programming in a WebClass, you can write a cookie into an HTTP response like this:

Response.Cookies("UserID") = "TedP" Response.Cookies("UserName") = "Ted Pattison" Response.Cookies("ColorPreference") = "Mauve"

These cookie values will continue to flow back and forth with each request/response pair.       Cookies are just as easy to read using the ASP Request object. The previous example creates cookie values that will only live for the duration of the browser's session. You can also persist your cookie values to a user's hard drive. If you do this, these cookie values will live across sessions of the browser. This means that your site can remember all sorts of information and maintain user-specific preferences across visits. Users seem to appreciate sites that do this. To persist a cookie value to your user's hard drive, simply add an expiration date as follows:

Response.Cookies("UserID") = "TedP" Response.Cookies("UserID").Expires = "September 1, 1999"

If you have a shopping cart application, you can build up a complex data structure across successive hits. Note that you can also write multiple values to a cookie if you want to store line items. The upward limit for what you can store in a cookie changes from browser to browser, but most browsers supports cookies up to 4,096 bytes.       Cookies work great until you encounter users who have disabled cookie passing in their browsers. If you have a requirement to accommodate users such as these, you will not be able to store data on the client that will live across sessions of the browser. You must also come up with another technique to pass state data back and forth. One technique you can use is to append named values to your URLs. For instance, instead of setting your HTML form's action to \MyPage.asp, you could change it to \MyPage.asp?UserID=TedP. The extra data is known as a query string.       Appending named values into a query string requires additional effort. You can dynamically embed these named values onto the tail of every URL in your pages. However, Visual Basic 6.0 and the WebClass framework can transparently perform this task for you with far less effort. Each WebClass has an URLData property. If you assign a value to this property and then use either the URLFor method or the WriteTemplate method, the WebClass framework will append a named value to the end of your URLs (such as \MyPage.asp?WCU=TedP). You can easily query the URL Data property while processing a request to determine whether it was assigned a value during an earlier request.       The use of query strings and the URLData property have two limitations. First, you are slightly more limited in size (2KB) than you are with cookies. Second, you must use the POST method in your forms. Your app will not work correctly if your HTML forms are using the GET method.       There is one last client-side state management technique that you should have up your sleeve. This technique involves the use of a hidden field in an HTML form. If a user will not accept cookies and you want to store more information than can fit in the URLData property, hidden fields may be the solution you're looking for. Note that this technique requires the use of an HTML form and a Submit button. Here's an example of what your form might look like:

<FORM ACTION="MyPage.asp" METHOD="Post"> <INPUT TYPE="Hidden" NAME="UserID" VALUE="TedP"> <INPUT TYPE="Submit" VALUE="Get Past Purchases"> </FORM>

A Few More Words on Optimization       Both the WLBS and LocalDirector allow you to turn off request-based load balancing and revert to session-based load balancing. The WLBS has an affinity setting, which lets you instruct the service to route users over the same physical server once the first request has gone through. Likewise, LocalDirector has a sticky command that provides the same effect. The good news is that these features allow you to use ASP Session variables to store state in the middle tier.       The bad news is that using one of these features compromises scalability. In other words, if you convert either LocalDirector or the WLBS to using session-based load balancing, then the benefits of request-based load balancing are lost. This slows the system down by requiring it to maintain a user-to-server mapping and perform a lookup upon each request. It also makes your site vulnerable to the previously discussed problems with fault tolerance and distribution skew. It's kind of like buying a Porsche and driving 30 miles per hour on the freeway.       These session-aware features have been added mainly to support applications that have dependencies on middle-tier state. Your Web application will scale much better if you don't introduce these dependencies into your design.       Let's say you've been very disciplined throughout the design phase and you've created a Web application that has no dependencies on ASP session management. Once you've gotten to this point, you can and should perform one more optimization. As it turns out, IIS conducts lots of session management work on your behalf unless you explicitly disable this feature. The problem with ASP session management is that it unnecessarily wastes precious resources such as processing cycles, memory, and network bandwidth.       What happens if you don't disable ASP session management? Any time a user requests an ASP page, the ASP framework checks to see if the user is associated with an active session. If it's the user's first request, the ASP framework carries out a bunch of tasks to start tracking the user's session. ASP generates a unique SessionID and starts passing it back and forth in a cookie. ASP also executes the Session_OnStart event if it exists, and creates any session-level objects that have been defined to run on the server with the <OBJECT> tag. If the ASP framework finds a valid SessionID in an incoming request and determines that the user is part of an active session, it maps the current request to any state that may have been stored to an ASP Session variable in earlier requests.       The ASP framework also performs extra work to serialize all requests for any one session. While this prevents concurrency problems with session state, it is also costly and restrictive in cases where it is not necessary. For example, if your application fills two frames inside the browser, one frame would have to be completely filled before the second page could be processed. If session management has been disabled, it's possible for IIS to process both pages at the same time on two different threads.       As you can see, the session management features of ASP are expensive. If you don't need them, you should definitely turn them off. You can do this by disabling the Enable Session State option of your ASP application from the IIS administration tools. You can also turn off session management by placing the following declaration at the top of all your ASP pages:

<%@ EnableSessionState=False %>

I hope this month's column makes you think long and hard about scalability in your application's initial design phase. There are many designers and developers who deeply regret the dependencies their applications have on ASP Session variables. They were not prepared for fame and, consequently, their applications will probably never make it in Tinseltown. Will they ever re-architect their approach to state management and rewrite their applications for a Web farm environment? Maybe, maybe not. However, those who can make this leap will find it painful at best. The Internet, like Hollywood, is continually littered by thousands who can't handle the pressure. Are you prepared for fame? After all, it could always happen to you.

From the June 1999 issue of Microsoft Internet Developer.