This article may contain URLs that were valid when originally published, but now link to sites or pages that no longer exist. To maintain the flow of the article, we've left these URLs in the text, but disabled the links.

MIND


This article assumes you're familiar with Active Server Pages and VBScript or JScript
Download the code (4KB)

Satisfy Your Data Cravings with Server-side XML
J.P. Morgenthal
     
XML is a flexible, extensible data description language that also happens to fill in many of the gaps left by HTML. Microsoft-provided parsers and interpreters can help you get up and running with XML in advance of its incorporation into Microsoft Internet Explorer 5.0.
Extensible Markup Language (XML) is rapidly emerging as a powerful tool for data processing. While some try to pigeonhole this language as Web-specific, its utility is quickly being recognized in many other sectors of the computing industry. XML is currently recommended for use in systems management by the Desktop Management Task Force (DMTF), for messaging by companies such as DataChannel, Microsoft, and webMethods, and for persistent data storage by companies such as POET and Object Design. XML is a long-overdue solution to a major industry need.
      Although there are many uses for XML, I'll focus on how XML can be employed to manage data on the server. Microsoft offers a powerful environment for manipulating XML data on the client with Microsoft Internet Explorer 4.x, but since much of the Web world also uses Netscape Navigator or earlier versions of Internet Explorer, most of the processing still needs to take place on the server.
      XML is more than just a simple language—it's a meta language, a language that can be used to create other languages. In addition, XML is designed so that both machines and humans can read it easily. Here's a simple instance of XML code: 

 <DWELLING type="residential" floors="2" family="single" gas="true" bedrooms="3" baths="2.5"> 
 125 East Chestnut Street, Elk Grove, NY 11234
 </DWELLING>
Most people could quickly decipher this information as a real estate listing for a home in Elk Grove, NY. The vocabulary is the same one used by real estate offices and in classified ads. And a machine can process it just as easily as part of a query for single family homes in Elk Grove, NY. Compare the previous XML example with this comma-separated electronic representation: 

 125, East Chestnut Street, Elk Grove, 11234, residential, single, 1, 3, 2.5
While some might make the assumption that this is real estate data, it's certainly not a given. Moreover, there's no way a processing environment could utilize this data unless it was imported into another well-known format, such as a database or spreadsheet.
      This difference between XML and traditional text-based data is leading to a very important change in the industry: the ability to provide enhanced data reuse and organization. In the beginning (well, in the days of COBOL) computing was data-centric. Tasks were written to run against a common data set. This eventually failed because there was little to no process reuse. If the data set changed, the program had to change to accommodate it.
      More recently, the computer industry has become process-centric, using objects to encapsulate the task with a hidden subset of data. Objects help the programmer develop and reuse logic faster and more efficiently, but when exposed to the general populace can lead to disaster. That is, when objects are actually used as the interim data representation in memory, it can be very difficult to troubleshoot problems and to reuse the data unless first created in a persistent form.
      With a metalanguage like XML, you can have the best of both worlds. You can be data-centric because the data is stored in a highly reusable and machine-readable format. But you can also use objects to access the XML instance in a model-oriented fashion. With this combination, if the XML instance changes, the object model does not necessarily need to change unless words are removed from the vocabulary.

Composition of a Vocabulary
      Now that I've explained the virtues of XML, let's look at the first step toward employing XML: developing a vocabulary. Vocabularies are made up of two key components, elements and attributes. While XML has constructs for many other types of information, such as comments, processing instructions, and entities, only elements and attributes are needed to build a vocabulary.
      When diving into XML for the first time, you may come up against a construct called a document type definition (DTD). This is a description of the vocabulary that you are creating, defined in such a way that an XML parser can determine if a particular XML instance conforms to your vocabulary. An XML instance that conforms is called valid, while one that does not is called well-formed. A well-formed document is not invalid. Indeed, it is just as useful as the valid document, but it doesn't match the expected vocabulary.
      Elements in XML define the beginning and end of a data stream. They are defined with < and > brackets, just like in HTML. XML also supports an end tag, which is defined with a forward slash. A matched pair of tags might look like this: 


 <HOME> </HOME>
For each start tag, XML requires a matching end tag. Each XML instance must have one root element that anchors the entire instance and is the first element defined in it. This is a correct XML instance: 

 <HOME>
 
 This is my house.
 I live here.
 
 </HOME>
The following is incorrect, because it doesn't have a root element wrapping it: 

 <STEP_1> Setup the VCR</STEP_1>
 <STEP_2> Put the tape in </STEP_2>
The incorrect example could easily be corrected by placing tags around all the steps like this: 

 <SETUP_INSTRUCTIONS>
     <STEP_1> Setup the VCR</STEP_1>
     <STEP_2> Put the tape in </STEP_2>
 </SETUP_INSTRUCTIONS>
      Elements can also contain attributes. These are additional pieces of information associated only with the beginning tag and are in the format key="value". For example: 

 <MAIL from="jp@ncfocus.com" to="fred@ogre.com"> Hello Fred </MAIL>
In this article, I'm only going to cover the elements and attributes components.

Microsoft XML Parsers
      There are several XML parsers freely available for download over the Internet, including the XML Java parser from Microsoft. If you'd like a better understanding of all the intricacies of parsing an XML instance, the Microsoft parser comes with source code. This is really helpful to users looking to build high-performance parsers for use on the server.
      The Microsoft Java parser is a validating parser, which means that it will ensure that the XML instance follows a defined DTD. In addition, Microsoft also provides the source code to its Data Source Object (DSO), an OLESimpleDB provider, which allows data binding inside Internet Explorer. This capability was originally included to support the Channel Definition Format (CDF), an XML-based vocabulary for handling subscriptions to Web sites. Microsoft also includes source code for the Microsoft XML object model, which is exposed from Internet Explorer through VBScript and JScript.
      Internet Explorer 4.0 also includes a nonvalidating C++ XML parser. This parser exports a COM interface for use by C++ and Visual Basic-based applications. However, this is a dated version of the parser that does not fully conform to the W3C recommendation for version 1.0.

XML Object Model
      The W3C is focused on delivering a common programmatic interface for manipulating XML instances called the Document Object Model (DOM). As the DOM develops, Microsoft will provide compliance with it. But in an effort to offer an easier development path, Microsoft has also provided an initial object model that can be used from scripting environments.
Figure 1: Parse Tree
Figure: Parse Tree
      The XML parser object, whose progid is MSXML, supports four key objects: Document, ElementCollection, Elements, and Error. The result of parsing an XML instance is the creation of a hierarchical tree (see Figure 1). The object model provides a programmatic way to navigate the tree and identify elements and attributes. This same object model can be used inside Active Server Pages (ASP) to manipulate an XML instance on the server.
      The Document object is the key object responsible for parsing XML instances and for acting as the container for the parsed tree. Figure 2 shows properties and methods supported by the Document object.
      To create a new instance of the Document object in ASP code using JScript, you'd call one line of code: 


 var document = new ActiveXObject("msxml");
The same is true when you want to create a new instance of the Document object using VBScript in ASP: 

 set document = Server.CreateObject("msxml")
      The parse tree contained within the Document object contains one root element, which has n children. The root element is an instance of the Element object. Key properties and methods for this object are listed in Figure 3, along with instructions for their use where necessary.
      The children property of an element represents a collection of the element's child elements. Remember that the parse tree is hierarchical; each element could contain a set of elements that are bound between its start and end tags. 

     <ITEM_ONE>
         <SUB_ITEM_1/>
         <SUB_ITEM_2/>
     </ITEM_ONE>
      ITEM_ONE has two children, SUB_ITEM_1 and SUB_ ITEM_2. Therefore, the ITEM_ONE element's children property would point to a collection of two elements.
      Element collections are indexed by one of three methods: numerically, by subject, or by subject/offset. They are represented in JScript like this: 

 var name = elem.children.item(0); // return the first child element
 var names = elem.children.item("Tom");// return a list of all elements named Tom
 var name = elem.children.item("Tom", 2); // return the third element named Tom
The VBScript method is identical except that var is replaced with set (and, of course, the line-ending semicolons get lost).
      Notice that the Collection object has an Item method. It also has a property length that identifies how many items are in the collection: var n = elem.children.length;

DSO and XSL Components
      The MSXML object that I'll use here is the nonvalidating C++ parser that ships with Internet Explorer 4.0. This parser can be used from within Internet Information Server (IIS), Personal Web Server (PWS), or Internet Explorer. Microsoft also supplies additional support components for manipulating XML instances. These are the DSO and the Extensible Style Language (XSL) components.
      DSO is a useful tool for performing data binding to the elements of an XML instance from within Internet Explorer. Data binding is made possible if OLESimpleDB providers that let you extract data based upon a consistent interface are available. OLESimpleDB providers can exist for any data source, including files, databases, and even Web pages. The DSO object provided for XML ships with the Java-based XML parser and is invoked as an applet, so it cannot be used directly from within the server environments.
      The XSL component uses a current-draft W3C specification of XSL to transform XML into HTML. An XSL Style Sheet defines a set of rules for transforming XML elements; for example, it might indicate that <CAR> should be replaced by <B><I>. The XSL component is an ActiveX control and can be used by IIS and PWS for uncomplex server-side transformations. (Complex transformations require tree traversals or calculations.)

Using XML for Data Persistence
      So how do you use XML on the server for persistent data, for processing form data from the client, for creating extensible server-side applications, and for producing usable results for any Web client? Databases are excellent repositories of large quantities of consistently formatted data that need fast retrieval. But sometimes data is not easily storable in a columnar mechanism. Real-world data is variable and contains important information throughout its content. Databases can also be overkill, limiting the performance of an application that needs to store structured data between client invocations. XML can provide much needed relief in these scenarios.
      XML-formatted data gives you excellent reusability. Sure, it would be very simple to open a file and write out a bunch of variables in no particular order, since the same programmer will most likely write the code to read that data back in. Solid code should not be treated as a write-once venture, but rather an asset that increases in value with reuse and maintainability. With XML and a programmatic interface such as the DOM or the Microsoft XML object model, you can create a reusable code base to handle data access and retrieval. To take advantage of this code reuse, all you need to do is produce an XML-compliant file.
      My next example uses a user identifier to manage state between client invocations. The vocabulary for the file is extremely simple. There is one element named USER that has two child elements: a user identifier (USERID) and the last page the user visited (PAGE). When the user logs into the system, the data file is parsed into memory. If the file does not exist, a new parse tree is created. When the user logs out of the system, the file is written to disk.
       Figure 4 shows a self-contained application that runs as an ASP script. It will generate the HTML necessary for thenext step. As you can see, the function writeXML manually writes out the elements. Unfortunately, the C++ parser that ships with Internet Explorer 4.0 has some problems building the parse tree manually, although it works fine as long as it is parsing an XML file. The addChild method on the Element object does not work properly when the parser has not been initiated with a URL. This may be due to the fact that the component was originally devised for the browser and not the server. Here's the file produced by running this application and typing "JPMorgenthal" in the input area of the logon form: 


 <?xml version="1.0"?><USER>
     <USERID>JPMorgenthal</USERID>
     <PAGE>PAGE 1</PAGE>
 </USER>
      You will have more success with server-side XML at this point if you use one of the available Java-based XML parsers. Another option is to download a beta version of Internet Explorer 5.0, which uses a W3C-compliant DOM interface instead of the Microsoft XML object model.
      This example is very similar to the MyInfo ASP component that's available from Microsoft. The MyInfo component allows users to keep personalized information loaded in the Web server. Unlike the file created above, which can store information about all visiting users as they move around the site, the MyInfo control only stores information in the MyInfo.xml file. The documentation states that this file can be found in the \Winnt\System32 directory under Windows NT and in the root directory under Windows 95, but I found my Windows 98 copy under \Windows\System\Inetsrv.
       Figure 5 shows the ASP code for MyInfo. Here's the contents of the produced file: 

 <XML><PersonalName>JP Morgenthal</><CompanyName>NC.Focus</>
     <CompanyStreet>11 Starfire Court</>
     <CompanyCity>Hewlett</>
     <CompanyState>NY</>
     <CompanyZIP>11557</>
     <CompanyPhone>(516) 792-0997</>
 </XML>
This file is not XML 1.0-compliant; the MyInfo control was developed before the XML specification was complete. The problem lies in the end tags, which need to be labeled properly with the tag name.

Managing Form Data with XML
      You can extend the concept of using XML for persistent data handling by managing Web forms with XML. Again, it is much more helpful if your forms will never change and you can develop a single database schema that will work for the lifetime of a form.
      Web forms reflect the desire to collect a variable set of information about a particular issue or thing. The information collected must then be placed into a medium that can be queried. For example, say you wanted to offer a download of a report or program, but in return wanted to collect some information about the person downloading. To do this you might set up a simple form that collects just the person's name and contact information. This works for a while, until you realize that you also need to capture the user's operating system. If you're using a database in this scenario, you'd need to modify it by adding a new field.
      Another scenario might be that you have two questionnaires on your Web site. While they both collect basic contact information, they also capture very different sets of information. A single user can fill out both forms. In this scenario, you must now decide if you're going to have duplicate information floating around your database, or get into handling multiple database tables.
      When you use XML as a persistent data format, there is no need to set in stone what information you'll want in advance. It is important to note that while I have looked at using XML from the perspective of flat files, XML offers more robust storage options. XML repositories are XML-based storage facilities that are implemented on top of database technologies—usually object-oriented databases. These repositories can handle multiple XML instances, and allow users to query their tags and text values.
      The trick to converting Web form data into XML instances is to ensure that non-form data can be filtered out very easily. Most Web servers offer some form of CGI or more advanced programming environment that lets you access the form data that's posted to the server. This string is usually in the format "field1=data1&field2=data2&field3=data3". Therefore, it is very easy to find all the keys and values for any given form. But some of the information you get, like the value of the submit button, is necessary only to handle the browser/server interaction. To strip these out, you can consistently preface a non-data field's name with a legal character like an underscore.
      Using a generic conversion routine for translating form data to XML for storage on the server has many benefits. For one, new forms can be posted on the Web server without having to first create a database for it. The data is highly reusable, which means that post-processing can sweep the server and push the data into a database if it is necessary for it to reside there. Indeed, this can be done effectively from within the PWS and IIS environments by combining the MSXML component with the database access components supplied by Microsoft.
      Note that I'm not promoting the use of XML over database management systems (DBMS). There will always be cases, like handling credit card transactions, where interaction with the database must be done interactively. But when your data is extremely volatile, XML can provide better performance than constantly issuing database calls. When the data is very dynamic and does not fit well into a standard schema, XML is also worth consideration.
      But the XML story doesn't stop at data persistence. It's also an interesting way to tackle the problems of messaging. Messaging is a broad category, so I'll start by defining the different categories of messaging: store and forward, publish and subscribe, and remote procedure calls.

Store and Forward
      Store and forward is the most common form of messaging. This category includes electronic mail and asynchronous message queues. Store and forward systems generally do not care about content, but they do have specific structures and protocols for communicating between stores, the message holding bins.
      XML can be integrated with store and forward messaging in two ways. First, XML can define the structure of the message that is used by a store and forward system, like so: 


 <MAIL>
     <FROM addr="jp@ncfocus.com">JP Morgenthal</FROM>
     <TO addr="info@ncfocus.com">Information Mailbox</TO>
     <SUBJECT>How do I use XML for Messaging?</SUBJECT>
     <MESSAGE>
         Line1
         Line2
         Line3
     </MESSAGE>
 </MAIL>
Here the store and forward system will actually use an XML vocabulary to define the message structure.
      Store and forward applications can also take advantage of XML by routing XML instances between applications. An example of this would be using SMTP to send an XML instance to another person. This method requires that the recipients be able to process the message themselves.

Publish and Subscribe
      Also known as distributed events services or push, publish and subscribe delivers a specific message to a group of users that have registered to receive it. In these systems, a consumer registers with a source for message delivery over a particular topic. The source then watches for information regarding that topic and passes it along to the interested parties when it arrives. These topics can be informational like a news update, or they can be system events like new files being added to the system.
      With publish and subscribe systems, the message usually has a proprietary data format. This makes it difficult to develop generic clients. For example, PointCast delivers its news updates in a special nonpublic format. Therefore, you need the PointCast newsreader to handle the receipt and display of updates. With the emergence of XML, it becomes relatively easy to deliver messages that use a publicly published vocabulary and allow recipients to decide how they want to use the data they receive.
      There was a tremendous buzz about push technology a year ago, but it seems to have fizzled badly. Much of the loss of interest is due to the requirement for new applications to be added to the desktop to handle proprietary message formats. It will be interesting to see if there is a push revival as more informational content providers adopt XML as a data format and publish their vocabularies.

Remote Procedure Calls
      A Remote Procedure Call (RPC) extends the concept of local function call programming over the network so that a totally different machine is actually performing the work and returning the results to the calling machine. If you write code in just about any language, you (hopefully) understand the concept of a function call. In a function call, the program jumps to a block of code that performs a certain routine. We can pass parameters to this function and obtain an output of its work. In some cases, the routine will actually change the parameters that were passed in as well as return a result.
      The Web server is an example of a very simple RPC. Each HTTP request is asking the Web server to execute a file retrieve function whose output is a stream of bytes. Generally, RPCs are much more complex than this, and are the heart of distributed computing. In an RPC, the local function call is now somewhere on another machine. This means that you must first package up the parameters into a stream of bytes to be shipped over the network. With local function calls, this is handled automatically by the memory and the processor. Unlike the Web server example, which just accepts a single string as input, you must now preserve type information, such as integer or string, and hardware-dependent byte ordering.
      Once this information is packaged, a connection to the remote machine needs to be established. This introduces a possible point of failure. When you execute functions locally, the code you call may be erroneous and fails; with RPCs the code can be perfect and yet still fail. Assuming the connection is made correctly, you need to communicate with a receiving process that will take your package, issue the function call on your behalf, and package and return the results. This process of packaging on both ends is called marshaling.
      Currently, there are many different RPC mechanisms available: Microsoft Distributed Component Object Model (DCOM), Distributed Computing Environment (DCE) RPC, Common Object Request Broker Architecture (CORBA), Internet Inter-ORB Protocol (IIOP), Java Remote Method Invocation (RMI), and a plethora of homegrown ones as well. These mechanisms all share the underlying principle of building the layer that marshals that code and executes the function.
      There has also been an emergence of XML-based RPC mechanisms. The key difference between the mechanisms listed previously and an XML RPC is the client-side requirements. With COM, CORBA, DCE, and Java, there must be a client that knows how to speak the same language. With XML, there only needs to be a client that knows how to make a connection and send data. The Web browser itself can be the issuer of an RPC. To clarify this picture further, I'll walk through two scenarios, one with CORBA and the other with XML RPC.
      If I want to develop a CORBA application that allows me to issue a function call on a remote CORBA object, I need to have a client that speaks IIOP and a piece of code that knows how to package up the parameters to the remote object properly. This piece of code is called a stub. The stub in turn calls the skeleton—another piece of code running on the remote machine that executes the call for me. If I want the Web browser to be the client for this application, I need IIOP built into the browser or I need to download it in the form of an application (Java or ActiveX).
      Things get easier if I develop the same application with XML RPC. First, my package will be defined as an XML message. Second, I only need a skeleton if my receiving application cannot handle parsing XML instances. Otherwise, I can send the XML instance from the Web browser directly into the receiving application. The return values will be formatted as an XML instance, allowing the Web browser to display the results on the screen.
      If you're using Internet Explorer 4.0 or later, the XML DSO component discussed earlier can be used to issue the RPC and display the results. The trick is to provide the XML instance as part of the URL that will be opened to receive the XML data stream: 


 <APPLET code="com.ms.xml.dso.XMLDSO.class" 
     MAYSCRIPT id=xmldso 
     WIDTH="100%" 
     HEIGHT="50">
 <PARAM NAME="URL"
     VALUE="http://gemhaddar/mind_1/rpc.asp?xml=<FUNCTION> 
     <NAME>reverse</NAME><PARAMETER>JP Morgenthal</PARAMETER></FUNCTION>">
 </APPLET>
       Figures 6 and 7 show the code to implement RPC functionality within your Web site. A single page provides a host of functions that the client can execute. In the example, I devised my own vocabulary for declaring a function call and identifying the parameters. A more robust solution would also identify the data types of the parameters as well as whether they were for input, output, or both.

Conclusion
      I have attempted to demonstrate the power XML holds for server-side and distributed computing. The tools used do not necessarily illustrate all that can be accomplished today. For example, using a Java parser on the server instead of the Internet Explorer 4.0 C++ parser will give you enhanced capabilities. Regardless, you should remember that XML is not the next version of HTML as some have heralded it, nor is it going to usurp all other data formats. XML is a flexible meta language that enables the development of new vocabularies, and a set of tools for processing these vocabularies that can be used by both the technical software developer and the business manager.

From the December 1998 issue of Microsoft Interactive Developer.