The proprietor of one of the most prominent ASP community sites explains the design trade-offs involved in a successful development portal.

Web sites that serve as Internet programming references usually have sophisticated technology for the sole purpose of discussing that very technology. My site, 15 Seconds, is a Web publication about programming Active Server Pages (ASP) and administrating Microsoft® Internet Information Server (IIS), part of Windows NT® Server. Naturally, the Web site is driven using IIS and SQL Server™, which is one of the reasons for its success. I used these two technologies to grow the site to over 5,000 pages of information. Let's take a look at how I did it.

A Little History
      15 Seconds started when I tried to reply to a question posted on a list server about isapi, the foundation for the ASP technology. My email answer was basically everything I knew at that time about ODBC technology. The answer was so long that the list server rejected the response, so I decided to post it on a Web site instead. The comments from people reading that article made me realize that there was a need for more information about the first version of IIS (which was still in beta at that time). Thus, 15 Seconds was born, transforming me from a programmer to a publisher, a role that I would not understand fully until a few years later.
      The name comes from the fact that 15 seconds is the average amount of time a user will wait for a Web page to return. In the beginning, all the articles were about performance, which was a big issue with isapi and the first versions of ASP.
      I was not really suited for the role of publisher. I received average grades in English through college, and when I started 15 Seconds I had no knowledge of style or many other basic publishing concepts. I did know something about database design and programming. I also knew that I didn't want to format all the articles that I wanted to publish on 15 Seconds. So I designed a database to hold the articles, decided on an article format, conceived a way to list the articles, and designed an isapi DLL to run my publishing system. The principle behind this system has remained the same since the beginning: separate the content itself from the format of the content. Though the format of the Web site has changed, and the technology has evolved (isapi to IDC to ASP), this concept has not. In this article I will discuss how I applied this concept to the development of 15 Seconds.

Limitations
      The size of a Web site is limited by its design. Designing your site from the beginning to handle future growth can save you a lot of time in the long run.
      At first, most site designers do not have the content to create a large Web site, so they start with a navigation design that will only support a small site. Most of the time they begin by programming their site by hand or using an editing tool such as Microsoft FrontPage®. As more articles are published or additional content is added, the Web designer needs to expand the navigation to make all the articles reachable, while providing the reader with a consistent experience. To get the most traffic from your articles, you need to cross-link them and provide alternative means to reach them. This means that for every article written, old articles need to be adjusted to link to the new article.
      At some point, publishing a new article requires more work formatting and linking to the article than it does writing the article. This is the opposite of the design's purpose, and only gets worse as the site grows. In my experience, one person working on a site 40 hours a week can't publish and maintain a site of over 200 pages using only static HTML. You can add extra people to the team, but this is not a solution­—it is a workaround. Site owners who can't overcome this problem either get sloppy with their design, start to toss old articles (otherwise known as archiving), or adopt a database design.

Database Publishing
      Database publishing is a development technique for Web sites in which the content is stored in a database and is inserted into HTML by the Web server. When a request is made to the Web server from the browser, the address usually refers to a template and a query string that tells the server how to populate the template. The Web server then loads the template, calls the database to get the content, fills the template, and sends the HTML back to the browser. The template is usually written in a language like Perl, CGI, ASP, or Cold Fusion. When written in one of these languages, the Web server executes this code, which calls the database and fills in the template.
      On the 15 Seconds Web site, all the templates are written in ASP 2.0 and call SQL Server 7.0. The templates can be classified in two ways: leaf templates or list templates. List pages are lists of links to leaf pages. Leaf pages display one row from a table and link to other leaf pages related to the row that you are reading. For example, a leaf page could display a question and an answer to a FAQ. A list page will list all the FAQs in a specific category.

Figure 1: A FAQ on the 15 Seconds Site

Templates eliminate inconsistencies in formatting the HTML content of individual pages. Since each row in the table is formatted with the same template, each page comes out identically. On 15 Seconds, I have one table called FAQ. Each row in the table is an individual question; columns represent the question, the answer, and the person who answered the question. Since I have one leaf template called question.asp that displays every FAQ in the table, I have 700 pages that appear identically (see Figure 1).       To add a frequently asked question to the site, all I have to do is add a row to the FAQ table. That page can then be generated by calling the question.asp template with the primary key of the row. The FAQ_Id column in the FAQ table is an auto-incrementing identity. Each time I add a row, a unique number is inserted by the database in the FAQ_Id column. This number is refer­red to in the query string when the ques­tion.­asp template is called (see Figure 2).

Figure 2: Adding a Question to the FAQ Table

When adding ad­ditional content to the 15 Seconds site, all I have to do is create additional tables that represent that content and create a template that formats the content. For example, I created a Book table (see Figure 3) and a book.­asp template for book reviews, and a KB table and a kb.asp template for Knowledge Base articles. Once I had a system for creating leaf pages, all I had to do was link up those pages so that the user could find them.

Figure 3: The Book Table

Broken Links       Database publishing eliminates a lot of broken links that can be created when you program your Web site with just static HTML. One of the ways to avoid broken links is to make the template create the list of links. When I first started 15 Seconds, I used a simple query to generate a list of all the FAQs from the FAQ table (see Figure 4). All I had to do was query the rows and create a link to the faq.asp leaf template using the primary key of the table as the reference for the FAQ to display. If I added a row to the FAQ table, not only would I get a new page from the faq.asp template, I would get another link in the link's page. If I removed a row, the link would be removed from the link's page.

Figure 4: A List of all FAQs

At first, I didn't have many FAQs so I was able to list them all in a single page. As the number of rows in the FAQ table grew, the links page grew as well. With a lot of links in the page, readers were having a hard time finding the answers to their questions. Now they're divided into categories, so the reader can pick the category he wants before being presented with the list of links. Organizing and Exploiting the Categories       To create the category-driven FAQ, I first created a Category table (see Figure 5). This way, if I wanted to create another category or a new technology came along, all I would have to do is add a row to the table.

Figure 5: The Category Table

I created a many-to-one relationship between the categories and the FAQ with a CategoryFAQ table that held the primary key for the category and the primary key for the FAQ. This way, I could put the same FAQ in many categories. The FAQ might be about ActiveX Data Objects (ADO), but it might also belong in the Just Beginning category.       I also divided the books, articles, Knowledge Base links, and other content into separate categories by creating more many-to-one relationships with tables like CategoryBook and CategoryIssue.       Once I'd made these relationships, adding content to the site was a two-part process. You can add the row to the appropriate table, such as the FAQ table, and then add a row to the many-to-one relationship table, like CategoryFAQ. After you have done the work of getting relationships in place, you can do some very powerful things with the database.       One way that you can exploit your new relationships is by cross-promotion of other content within a leaf page. For example, if the user is reading a FAQ about ADO, they might want to know about articles, books, and Knowledge Base documents regarding ADO. In your leaf page, you can easily determine which categories relate to the FAQ they are reading and display the other content that might interest them.       This technique creates additional layers of navigation, including one between leaf pages. If the user has drilled down to a particular topic, it is more likely that they will want to stay in that topic rather than go back up the tree and to another topic. So linking related list pages gives the user an easier time navigating the tree of information. One way to think about it is as branches of information.

Figure 6: The ADSI Focus Section

Another way to use the category relationships is to link to all leaves in a category branch. These are called focus sections on the 15 Seconds site, and they're on the top level (see Figure 6). They are lists of all articles, books, offsite links, and Knowledge Base links in a particular category. If a reader wants to see everything 15 Seconds has on Site Server, she can call the focus section page.       This technique provides for an additional method of navigation that follows human nature, not the database. In other words, readers can now navigate by category instead of by content type. Bumps in the Road       Database publishing stands up very well in theory; in practice, there are a few problems that need to be handled. Most of them stem from the fact that the tools to handle dynamic pages are not as good as those handling static pages. A dynamic page is one constructed by the server as the result of a request from the browser. In other words, the server has to do some processing beyond locating and returning a page, and the page might look different depending on the database, the user, or the time of day. A static page, by comparison, is read directly from the hard drive and doesn't change because of the user.       Some search engine crawlers do not index dynamic sites, and with a publishing site it is very important that you get your content in all the search engines such as Yahoo! and Alta Vista. Since 15 Seconds is rich in keywords and information, it would get much more traffic if indexed.       I wanted to use Index Server to allow readers to search the site—Index Server currently only works with static pages. If I stayed with dynamic pages I would have to use a more expensive search package that could crawl the dynamic pages.       Another problem with dynamic pages is that the URL that appears in the user's address box doesn't describe the page in an intelligent manner. With references to primary keys and a generic page, the dynamic page doesn't really describe what the user is viewing. With static pages, you can name the page to match the content.       I also needed to move the site from my office up to an ISP with a faster serv­er. This became a prob­lem since I had to transfer the database and the templates at the same time. With a slow connection my team was not able to suc­cess­­fully transfer the SQL Serv­er database all of the time, leaving the site in chaos.       Finally, the dynamic site was slow in comparison to the static pages because a large number of SQL queries created each page. This sluggishness reduced the number of pages visitors wanted to view. The slowness of the site was the result of creating a rich navigation environment between content and categories, using almost a dozen SQL queries per page. I also realized that although a dozen queries per page per user were being made, the page itself was not changing. In fact, some of the pages didn't change for months on end.       What I needed was a tool that would let me design pages dynamically and then create a static version of the Web site to avoid these potholes. So I created XBuilder. XBuilder       XBuilder is a Windows-based application that traverses a dynamic Web site and creates static pages from it. For the 15 Seconds site, all the content is staged on http://dynamic.15Seconds.com, and then the XBuilder app produces the files that are served from http://www.15Seconds.com.       This technique allows me to add content all week to the staging site and then build the live site once a week using XBuilder. I use the dynamic site much like a magazine publisher does preproduction. Most large Web sites need to have a staging server or a development server to test their enhancements and layouts. Once I have the files built, I send them to the live server at Internet.com in Connecticut. Then I can start working on next week's updates on the dynamic site.       Sending the files to Internet.com is a painless process—all I have to do is FTP them. This avoids the problems associated with transferring the database. If I had a bigger site, I could send the same files to two different Web servers, creating a cluster. Or I could send the same files to different Web servers all over the world, creating mirrored sites.       With the static pages on the live site, search crawlers are able to index the site. This alone boosted traffic right away. I can also use Index Server to traverse the static files and use the Index Server catalog to offer searching functionality.       The biggest gain from using XBuilder was performance. Static pages are sent to the user much faster. Instead of each request to a page calling 12 SQL queries, I only call those queries once a week when I build the site. This is the same philosophy behind compiling your source code or caching your script; if you know that you are going to use the page again, you should keep a copy ready. As a result, traffic increased by 20 percent. I believe users were willing to surf more since the pages were served quickly.       Another benefit that I hadn't counted on was increased reliability. Without dynamic pages I didn't need to worry about SQL Server crashing or memory leaks from components or the ASP code itself. Static files can thus help the server stay up without problems for months on end.       I now sell XBuilder to other companies that are interested in getting the same results. It can be found at http://www.xbuilder.net. Summary       I believe that 15 Seconds is a success partially be­cause of the technology that I used to implement it. The philosophy of database pub­lishing solves the problems associated with static HTML and separates the content from the format of the content. Publishing sites can leverage the database to produce a robust navigation experience for the user. Turning a dynamic site into a static site can reduce some of the problems associated with a database publishing philosophy.

http://msdn.microsoft.com/workshop/ design/layout/site021599.asp and http://msdn.microsoft.com/library/devprods/ vs6/vinterdev/vidref/viovrdesigningsites.htm

From the November 1999 issue of Microsoft Internet Developer.