The A-Z of VBScript Data Dictionaries, MIND June 1998

This article may contain URLs that were valid when originally published, but now link to sites or pages that no longer exist. To maintain the flow of the article, we've left these URLs in the text, but disabled the links.

This article assumes you're familiar with VBScript and Visual Basic.
Download the code (134KB)

The A-Z of VBScript Data Dictionaries
Dave Cline

The scripting.dictionary object effectively patches the hole left in VBScript by the exclusion of collections.

Performance. For my Active Server Pages, I just want what everybody else wants: screaming-fast performance. Peregrine-falcon-dive-out-of-the-clouds-and-grab-you performance. Of the top three goals of app developers (performance, maintainability, and speedy development), performance remains the most elusive.
One way to get blazing performance in code developed with Visual Basic® is to use collections. Collections simplify many storage needs, allowing you to easily add items to a set and enumerate them. But collections don't exist in VBScript. Not to worry. The scripting.dictionary, a high-performance data storage object that Microsoft includes with Active Server Pages, effectively patches the hole left in VBScript by the exclusion of collections.
A note about ASPs. As with many of the objects designed for ASPs, the nature and functionality of the objects, methods, and properties are easy to understand and implement. But enterprise-level applications can't be seriously undertaken without complete comprehension on the coder's part of where and how these objects, methods, and properties are to be used. The scripting.dictionary stands as just such an object.

Server-side Memory Storage
      When striving for performance gains in Web applications, storing frequently accessed data in memory remains one of the more obvious enhancements. Do you want to read your data a hundred, a thousand, a million times an hour from your hard disk? Or read it once and just remember it for future use? The answer is a classic no-brainer. In general, if you need the same data more than one hundred times in one hour, or if every user gets sent the same set of data, it makes total sense to store it in memory.
      After making the commitment to implement server-side memory storage, your next steps are to Figure out what data to store, and then to store it. Candidates for application-level memory storage might be cross-reference tables created for proper normalization of your data. If you sell sporting goods, you might store hook sizes for the flies you tie, or equipment types your dealers stock. Other data sets could be common lookup tables like countries, states, ZIP codes, area codes, fund lists, or building supply items. Depending on your architecture, you might even store larger, more specific data such as customer records or employee information. Stretching the boundaries of normal data retrieval and storage concepts can often result in some great performance leaps.
      The genesis for this article came when I set out to explore and use just such server-side memory storage. What I found refined my ASP development style as well as enlightened my entire team's attitude toward performance. It was simple, really. In Application_OnStart, I'd read a set of arrays (yes, arrays) from a database to be used later to fill combo boxes and build a multitude of ASP pages (see Figure 1).
      I could put the code in Figure 1 in Global.asa or in an include file referenced in the first page of my Web app. This way I'd have my data loaded and ready for my pages. This seemed logical so I put about 30 of these data storage tables there. The entire memory footprint was less than 1Mb. Great, I thought. Half the arrays had fewer than a hundred records, but a few had close to a thousand. Two of them were wide, with over 70 columns, though most were narrow, eight or fewer columns. I thought I'd be clever. I thought I'd be quick. I ended up being neither.
      Why did I attempt this? Aside from my compelling desire for speed, I was using Microsoft® Access as a database. Microsoft Access is a great tool that delivers exceptional end-user functionality. But Microsoft Access was not designed for hard-hitting, multithreaded, Web-oriented data requests. Besides, my client refused to entertain the idea of hiring a database administrator to configure and maintain SQL Server™, so I had to try something.
      With my IIS environment set up and my data sets loaded into arrays, I was ready to test. I loaded my test ASP, which ran random gets against one of my arrays. I clicked submit. I waited. And waited. Then my ASP timed out. What was this? Had I broken something? No—other ASPs in other virtual directories were working fine. So I tried it again, and got the same results. As you can guess, my arrays were not performing up to snuff. Sure, they loaded like jackrabbits. But sequentially looping through an entire 1000 record array (more than once per page) looking for a specific data item to display obviously fails every performance guideline ever written.
      What I needed, besides a reread of Steve McConnell's Code Complete, was a scripting.dictionary. I didn't abandon arrays altogether. I use them for sequential loading of dropdown boxes or visually ordered lists. I kept those arrays that fit that description. (For more information, see the sidebar "Performance Metrics.") The others I converted to dictionaries.


 <OBJECT RUNAT=Server SCOPE=Application ID=d_Country 
	PROGID="Scripting.Dictionary"></OBJECT>
 <OBJECT RUNAT=Server SCOPE=Application ID=d_DeepWide 
	PROGID="Scripting.Dictionary"></OBJECT>
 <OBJECT RUNAT=Server SCOPE=Application ID=d_10List 
	PROGID="Scripting.Dictionary"></OBJECT>
 
 SUB Application_OnStart
 dim mConn
 set mConn = Server.CreateObject("ADODB.Connection")
 mConn.Open "DSN=Dictionary","",""
 
 BuildDictionary d_Country, "CountryCode", "SELECT * FROM Country", mConn
 BuildDictionary d_DeepWide, "ID1,ID2,ID3", "SELECT * FROM DeepWide", mConn
 BuildDictionary d_10List, "listID", "SELECT * FROM List", mConn
 
 set mConn = nothing
 END SUB

A quick note on disconnected ADODB recordsets. As of this writing, ADO 2.0 has yet to be made part of Internet Information Server (IIS) 3.0. (I'm staying with IIS 3.0 until a couple of good books describe in detail the hundred or more checkboxes buried within IIS 4.0, or until a client explicitly requests it.) Unfortunately, ADO 1.5 and below do not contain the JET/DAO equivalent commands FindFirst and FindNext. Until it does, there's no performance gain in having a recordset hang out in the ASP Application object for the storage of cross-referenced, quick-lookup data.

An Internal Look at scripting.dictionary
An ASP dictionary, or rather the scripting.dictionary object, derives its excellent performance from its internal structure. A hash table lives inside the dictionary. A hash table, as opposed to an array, is a fixed allocation of heap memory. Knowing the size up front allows you to divide or hash (chop into little pieces) this chunk of memory into a fixed set of pieces. Once you know how many pieces you have, you can build an algorithm (hashing function) which, when fed a key, returns an index to one of your memory locations, where you then insert your key:item pair (see Figure 2).

Figure 2: A Hash Table

Figure 2: A Hash Table

      Once inserted into the chosen memory location, retrieval or deletion can be elegantly performed by sending the hashing function the desired key. This function then hashes the key into an index that represents the exact location of your key:value pair. Your data, sir.
      A hash table's algorithm and collision handling sets the stage for good to great performance. Collision is what happens when two keys hash to the same memory slot. The scripting.dictionary's response to hashed key collision is called chaining. Chaining effectively allows multiple hashed keys to share the same hash table location by extending that location via a linked list (kind of a smart array) that stores the keys of previous and next records. You can scroll forward and back in the linked list to find the exact key being searched for. This type of collision handling exceeds the performance of even a balanced binary search tree.
      An array is nothing more than a dynamic chunk of memory given indexes and a storage area for storing pointers to the array's variant data. There is no linking between array items, as in a linked list; there are only sequential subscripts. Locating a certain index is a matter of starting at the beginning of the memory area and looking at each subscript until you find the one which matches the key.
      This knowledge is useful in understanding how the dictionary object works, but it's not vital when actually using it. What is important is knowing when to use it.
      Further discussion of the scripting.dictionary will involve specific code designed to aid your implementation of server side memory storage. For method/property syntax of the dictionary object, consult Microsoft's online scripting resources at http://www.microsoft.com/scripting/.

Application-level Storage
      There were a few things to sort out when I began my migration to dictionaries. One was where to put my data. Although I see no reason why a dictionary cannot be used at the session level, and in fact I've used it there successfully more than once, I concentrated my attention on application-level data. My other issues are more caveats than showstoppers. Be aware when storing dictionaries that, at the application level, they are required to conform to the apartment threading model. To use a dictionary at this level (or any other apartment-threaded object such as an ADO recordset), you must declare it using the object tags within Global.asa:

<OBJECT RUNAT=Server SCOPE=Application ID=d_Country PROGID="Scripting.Dictionary"> </OBJECT>

Declaring an object like this imparts a certain spin on the code you will use to manipulate the object. You won't dim or set a local variant to reference the dictionary. You'll just use the name straightaway:

vntLocalVar = d_Country.items("US")

Or even:

vntLocalVar = d_Country("US")

This last variation is possible because Items is the default method on the dictionary object. One minor point: prior to a reference within an ASP-instantiated COM object, an application-level dictionary must be set to a temporary variable. When this is done, your dictionary data will then be available through a reference to this temporary variable within your COM object. If you don't do this in your ASP, there will be no way to access your dictionary in your COM object.
      One last gotcha: since the dictionary object is apartment threaded, and you don't want simultaneous editing of your apartment threaded objects, you should use Application.Lock before and Unlock after you edit an application level dictionary. OK, now on to the screaming.

A Dictionary's Internal Storage
      So you've got your dictionaries decided upon and situated properly (thus far, in name only). Now it's time to load them with data. BuildDictionary is a function I designed for the loading of app-level dictionaries via a name, a SQL statement, a key, and a data connection (see Figure 3). The coding style of our application developer group encourages the modularization of code, so I created other functions as well as BuildDictionary. Please download them from this page and review them for applicability to your own projects. I housed them in one server-side include called Dictionary.inc, which I drop into the head section of my ASP file when I need to use a dictionary.
      Loading arrays into memory cannot be simpler:

ArrayVariable = Recordset.GetRows()

This amazingly powerful code will load a two-dimensional array of n columns by x rows. Dictionaries, on the other hand, must be loaded manually and with a bit of work. But once they're loaded, look out! They have essentially two columns, the key and the item, and can be x rows long. So how do you store a multifield table in a dictionary?
      The method I used is to convert each row of fields into a single-dimension array and then store the array as the item, yielding, essentially, a collection of arrays. But what do you use as the key? They have to be unique—duplicating dictionary keys will generate a runtime error. Take a look at the BuildDictionary routine in Figure 3. Note the parameters passed to this routine. Dict is the name declared in your Global.asa; strSQL is the Select for the recordset; mConn an ADODB.Connection, and strKeyField contains a comma-delimited string of field names that represent the index on the table to be loaded. Essentially, this means I used the table key stored as the dictionary key, guaranteeing uniqueness. I separated the fields with a bar after I ran into a set of keys which had values like "60, 6" and "6, 06". These, when concatenated, created duplicate dictionary keys, a definite no-no.
      The item as an array works out beautifully. Two VBScript methods, Join and Split, allow you to quickly manipulate each array. Additional development guidelines at my job require us to create constants for all data-set field positions and FetchDictionaryItem. Figure 4 demonstrates how these constants are put to good use.
      If you're balking at the complexity and seemingly heavy overhead of this memory storage practice, I refer you to the performance ASP I built expressly for testing this construction, and to the results of the performance tests in the sidebar.
      In coding more than two hundred of these types of collections, I've found I generally know a number of specifics (keys) required to find and show user data. For instance, if I have to run a stored procedure against a SQL database I typically will have to provide primary keys to return the data. The same is true for these server-side dictionaries. When I build the dictionaries, I know the primary keys, and when I feed routines such as FetchDictionaryItem like so:

vntLocalVar = FetchDictionaryItem(d_DeepWide, Session(ORG_ID) & BAR & Session(CLUB_ID) & BAR & Session(USER_ID), FIELD_POS_MEMBERSHIP_POLICY)

I get an almost instantaneous return of the membership policy for the key I fed: organization, club and user information. In the applications I've built this is handy and performance oriented.

More Developer Guidelines
      The most maintainable method for using dictionaries in your ASP code is to wrap the necessary services required in well-named routines and functions. For example:

PrintDictionary(Dict) FetchDictionaryItem (Dict, strKey, intItemPosition) FetchDictionaryArray(Dict) FetchDictionaryRecord(Dict, ByVal strKey) UpdateDictionaryItem(Dict, ByVal strKey, ByVal intItemPosition, ByVal strItemValue) UpdateDictionaryRecord(Dict, ByVal strKey, aFields) AppendDictionaryRecord(Dict, ByVal strKey, aFields) KillDictionaryRecord(Dict, ByVal strKey)

      These wrappers allow you to concentrate on the application, not on refiguring how dictionaries work. Attempting to build a complex (15 or more tables) ASP application without creating reusable modules and functions would be like shaking three 1000-piece puzzles together and then trying to assemble them blindfolded—you may eventually get them assembled, but the results won't be very pretty. The idea here is to encapsulate and reuse. The dictionary.inc file contains everything I need to deal with any applications set of dictionaries. I've encapsulated all my required dictionary services within their own error-trapped routines. I can now reuse this in any application I desire. Months from now when I have to go back into the apps that use this modular include, I won't have a bit of hesitation as to how to pull an extra piece of information from one of my storage dictionaries.

Conclusion
      The kicker here is performance. I want the pages I build to be an optimal combination of performance, maintainability, and quick development time. I find that scripting.dictionary takes me one step closer towards this nirvana.
      Few development platforms are simpler to work with than Active Server Pages. They're just that easy to write. Attack your next ASP project with the goal of increased performance and the simplification of the maintenance process. As I have shown, the scripting.dictionary will get you both of these.
      I'd like to thank Andrew Stipdonk of Edmonton, Canada for help in refining the dictionary include and a nameless engineer at Microsoft who aided me in my discovery of the internals of the scripting.dictionary.

From the June 1998 issue of Microsoft Interactive Developer.