The NT EventLog, Part 1: Meet the API

Richard Grimes

Windows NT provides a service called the EventLog that you can use to log administrative messages. It's vital to log exceptional conditions, but the EventLog API is arcane and has some pitfalls. This article explains the event logging API and how to use the SDK tools to create event message resource files.

THE NT EventLog is a service that starts whenever you boot NT. It reads and writes event messages to the event log files. The service uses Microsoft RPC, so you can log messages to—and read the event log from—remote machines.

Such a facility is vital for all but the most trivial applications. Your code is, of course, as bug-free as possible, and you take care to catch exceptions and check input data for invalid values; but you, as the developer, need a record of these events. Further, if your application depends on other applications, or vice versa, you want to be sure that these applications start up in the right order, and that each server is initialized and ready to accept requests before a client makes a request. The EventLog is a central facility for logging such events, and because logged events are marked with a source name and time, you can check the synchronization of applications.

The EventLog is based on files with the extension evt in the %systemroot%\system32\config directory. You shouldn't access these files directly, because other applications, including NT, might want to log or read events. Instead, you should access the EventLog through the Win32 EventLog API. This API is rather arcane and doesn't follow the programming practices used elsewhere in Win32. In fact, it's crying out to be replaced by a COM object someday.

There are two functions used to open the EventLog. OpenEventLog() gets you a handle to an EventLog to read events. RegisterEventSource() gets a handle through which you can write an event. The handle that these functions returns isn't an NT kernel object handle: Don't use CloseHandle() when you're finished with it. Instead, use CloseEventLog() or DeregisterEventSource(). In my experience, the handles from OpenEventLog() and RegisterEventSource() appear to be interchangeable.

Both functions take two parameters: the UNC name of the machine where you want to log the event, and the name of the source. The source is typically the name of your application or suite of programs, and it's used in the NT EventLog viewer to filter events, as you'll see in Figure 3 later in this article. It's also used to associate a message resource file with the messages from a particular source. I'll talk about resource files later.

You should register your source in the system registry. You don't use RegisterEventSource(), which opens the EventLog for writing—you should provide a registry script or call the registry API in your code instead. The relevant key is:

HKLM\SYSTEM\CurrentControlSet\Services\EventLog

It has at least three keys: Application, System, and Security. These are the EventLogs themselves, and they correspond to the files in the config directory. Under these keys will be a key for each registered source. For example, Figure 1 shows the registry entries for AutoChk (which is used when you type chkdsk at a command prompt).

Figure 1: The EventLog registry entries for AutoChk

This entry has two values that indicate the types of events that can be generated by this source and the path to the file that provides the format strings. There are other named values that can be used (see Table 3 later in this article). When you add a new entry into one of the EventLog keys, the service gets a notification, and it adds the name of the new source into a multi-string (REG_MULTI_SZ) value called Sources. Figure 2 shows this and other values pertinent to EventLog files.

Figure 2: The parameters for the Application EventLog

MaxSize is the maximum size that the EventLog will reach. When it's full, the EventLog service will determine what to do according to the value in Retention. If Retention is 0, new events overwrite the oldest events; if it is 0xffffffff, events aren't overwritten and you must manually clear the EventLog. Any other value is the number of seconds that the event remains in the EventLog before it can be overwritten. The EventLog service "wraps around" to the beginning of the underlying file. This further emphasizes that you should only access the EventLog through the API, which handles this wrap-around for you.

When you open an EventLog to write events, the service will go through all the keys in the EventLog key to see whether the source you've specified is in the Source's value. If it is, the handle returned will be to that EventLog, and when you report an event, it will be written to the file given in the File value. If the service can't find the source you specify, the service will write the event to the Application log. When you open an EventLog to read values, you need to specify one of the keys under the EventLog key.

Events are reported using the ReportEvent() function:

BOOL ReportEvent(HANDLE hEventLog, WORD wType, 
   WORD wCategory, DWORD dwEventID, PSID lpUserSid, 
   WORD wNumStrings, DWORD dwDataSize, 
   LPCTSTR *lpStrings, LPVOID lpRawData);

The first parameter is the handle of an open EventLog, the second is the severity of the event, and can be one of EVENTLOG_ERROR_TYPE, EVENTLOG_WARNING_TYPE, EVENTLOG_INFORMATION_TYPE, EVENTLOG_AUDIT_SUCCESS, or EVENTLOG_AUDIT_FAILURE. The first three are self-explanatory; the last two refer to successful and failed security events logged to the Security log.

The dwCategory is the category of the event. The value you use here is defined in the category file for the source. Typically, you'll define your own. These could be objects within your application (Customer, Product) or phases in the application's execution (StartUp, ProcessOrder, ShutDown). Categories are WORD values, but you can use a resource file to map these values to Strings. The NT Event Viewer will display these strings, or the number, if it can't find the resource file.

The dwEventID is an identifier for the event; if you're defining your own events, this is used to obtain a printf()-like format string from the message file for the source. This format string can have placeholders like printf(). Events reported with ReportEvent() only have string placeholders, but device drivers and services use a different API and therefore have a wider range of parameter types. These appear as a series of numbered parameters %n (where n is 1 to 99). The WORD dwNumStrings holds the number of strings that will be inserted where the placeholders appear. The strings themselves are in lpStrings, an array of string pointers.

You might decide that the event should report binary data—for example, a server communicating over sockets could receive a corrupted data packet, and it might be worthwhile to record this packet. This is the purpose of lpRawData, which has a size indicated by dwDataSize. Finally, you can optionally add a SID (security ID) which will indicate the user account that logged the event. Note that only the LocalSystem account can write to the Security log; however, Administrators and LocalSystem can write to the System log, and any account can write to the Application log.

Writing events to an EventLog is quite straightforward, but reading them isn't so simple. Win32 provides a function called ReadEventLog() to read from an open log. It takes seven parameters:

BOOL ReadEventLog(HANDLE hEventLog, 
   DWORD dwReadFlags, DWORD dwRecordOffset, 
   LPVOID lpBuffer, DWORD nNumberOfBytesToRead, 
   DWORD *pnBytesRead, 
   DWORD *pnMinNumberOfBytesNeeded); 

This function will read as many event records as it can fit into the buffer you pass in lpBuffer. If the buffer is too small to return any event records, ReadEventLog() will return false, and the pnMinNumberOfBytesNeeded will return the size of the buffer needed for the next event record.

The dwReadFlags determine how the data is read and in what direction. You can read in chronological order (EVENTLOG_FORWARDS_READ) or reverse chronological order (EVENTLOG_BACKWARDS_READ). Further, you can combine one of these flags with one of two others to read a particular event record (EVENTLOG_SEEK_READ), or the next record after the previous call to ReadEventLog() (EVENTLOG_SEQUENTIAL_READ). This last option implies that the EventLog service keeps a cursor for the read records based on the EventLog handle.

Remember, if the buffer is larger than a single record, the function will fill it with as many complete records as it can. If you want to read a specific record, you should specify its number in dwRecordOffset and call ReadEventLog() twice—first with nNumberOfBytesToRead with zero to get the size of the required buffer, and then a second time to get the actual record. To get the record number of the first (oldest) event record, you can call GetOldestEventLogRecord(), and to get the total number of records in an EventLog, you can call GetNumberOfEventLogRecords().

The data that's returned is an EVENTLOGRECORD. This is a variable-length structure because it contains strings rather than pointers to strings:

typedef struct _EVENTLOGRECORD { 
   DWORD Length; DWORD Reserved; DWORD RecordNumber;
   DWORD TimeGenerated; DWORD TimeWritten; 
   DWORD EventID; WORD EventType; WORD NumStrings; 
   WORD EventCategory; WORD ReservedFlags; 
   DWORD ClosingRecordNumber; DWORD StringOffset;
   DWORD UserSidLength; DWORD UserSidOffset; 
   DWORD DataLength; DWORD DataOffset; 
   // TCHAR SourceName, TCHAR Computername[]; 
   // SID UserSid; TCHAR Strings[]; BYTE Data[]; 
   // CHAR Pad[]; DWORD Length; } EVENTLOGRECORD; 

This structure is unusual because it has these comments that appear to define numbers after DataOffset. SourceName and ComputerName aren't pointers to an array of TCHARs, but the actual TCHAR arrays; Data is an array of BYTEs. Stranger still is the Strings member. This is zero or more insertion strings for the event, and corresponds to the lpStrings passed in ReportEvent(). These members appear as comments because they can't be defined as part of the structure, since their length is unknown.

When you get an EVENTLOGRECORD, you can access these members using the various Offset members, knowing that SourceName appears immediately after the last member of the EVENTLOGRECORD structure. For example:

PEVENTLOGRECORD pELR;
ReadEventLog(…, pELR, …);
LPTSTR SourceName = (LPTSTR) ((LPBYTE)pELR 
   + sizeof(EVENTLOGRECORD));
LPBYTE Data = (LPBYTE)((LPBYTE)pELR + pELR->DataOffset);

Ugh—this is the sort of code that should remain with C and shouldn't appear in C++ code. Still, you'll have to endure it until next month's article.

The EventLog doesn't store the entire formatted message—just the insert strings. This has two benefits: First, it cuts down the size of the log files, because most of the descriptive text can be stripped out and saved in a resource file. Second, the format strings can be made locale-dependent, which means that you can supply resource files for each locale, while the application can be locale-independent. The function that formats messages is, not surprisingly, FormatMessage():

DWORD FormatMessage(DWORD dwFlags, LPCVOID lpSource,
   DWORD dwMessageId, DWORD dwLanguageId, 
   LPTSTR lpBuffer, DWORD nSize, va_list *Arguments);

The first parameter specifies which way this function will work and is one or more of the following flags: FORMAT_MESSAGE_ALLOCATE_BUFFER, FORMAT_MESSAGE_IGNORE_INSERTS, FORMAT_MESSAGE_FROM_STRING, FORMAT_MESSAGE_FROM_HMODULE, FORMAT_MESSAGE_FROM_SYSTEM, or FORMAT_MESSAGE_ARGUMENT_ARRAY. If you've allocated the buffer, pass the size in nSize and a pointer to a TCHAR array (that is, LPTSTR) in lpBuffer. If you want the buffer allocated with LocalAlloc() by FormatMessage(), include FORMAT_MESSAGE_ALLOCATE_BUFFER in dwFlags: lpBuffer should be a pointer to a pointer of TCHAR (that is, LPTSTR*), and FormatMessage() will set your pointer. Don't forget to call LocalFree() on the returned value to free the memory.

The message uses a format string. This can come from one of three places, again indicated by dwFlags: FORMAT_MESSAGE_FROM_STRING means that lpSource is a format string, and FORMAT_MESSAGE_FROM_SYSTEM indicates that the system will provide a format string (used when you have an error code retrieved from GetLastError()). For EventLogs, you should use FORMAT_MESSAGE_FROM_HMODULE, and use lpSource to pass a HMODULE of a DLL or EXE loaded with LoadLibrary(). The function will search the module's resources for a message resource with the specified ID and use this for the format string. These resources will have an explicit locale, and if a module has more than one version of the format strings, you can specify which one to use with the dwLanguageId parameter.

The actual insert strings are passed into the function in the Arguments parameter. There are three values in dwFlags that affect how insert strings are treated. If you've specified FORMAT_MESSAGE_IGNORE_INSERTS, the function will just return the format string without merging the insert strings. This is useful if you want to get the format string from a DLL. If you use FORMAT_MESSAGE_ARGUMENT_ARRAY, the Arguments parameter is an array of 32-bit items that are either interpreted as pointers to strings or as 32-bit integers—depending on the placeholders in the format string. If you don't use this flag, the parameter will point to a va_list.

For the EventLog, you'll most likely use FORMAT_MESSAGE_ARGUMENT_ARRAY—but notice how the data is an array of LPTSTRs, as opposed to the data in EVENTLOGRECORD, which is an array of TCHARs, where each string is separated from the next with a NULL character. If you want to format a message obtained from ReadEventLog() with FormatMessage(), you need to extract the strings and put their pointers in an array.

When you read event records from an EventLog, you might find that for events that originated in device drivers, some insert strings appear to be placeholders (for example, %%2). Such placeholders refer to a message string resource in the ParameterMessageFile registered for the source. If you're reading events from the event log, you should check all of the insert strings for a "%%", and if you find one, extract the number, load the ParamMessageFile, and then call FormatMessage() with the dwMessageId as this number. You should then put the returned string in the original array of insert strings, and call FormatMessage() one more time.

So what are these message resource files? They're just code modules (EXE or DLL) that have a message string resource type RT_MESSAGETABLE (which is defined as 11). You can use your application itself as an EXE resource file. Since you use LoadLibrary() to load a resource file, there's no danger of the application being started again. You can't use Visual C++ to create these resources: Its ResourceView knows nothing about message resources. Instead, use the MC.EXE tool provided with Visual C++ (look in the bin directory).

This command line tool takes instructions from a text file that has two sections. The header section is for global values for the entire file, shown in Table 1.

Table 1. Items in the header section of an MC file.

The Message IDs you use in ReportEvent() are 32-bit values. MC generates these values from values you give in the MC file. To identify an event message in an MC file, you use a 16-bit value: the remaining 16 bits are calculated by MC from the severity and facility specified for that message. Don't confuse the severity of a message with the wType used in ReportEvent()—the EventLog doesn't use the severity and facility, it just treats them as part of the 32-bit Event Message ID. Thus, SeverityNames and FacilityNames are used to define new severity and facility types, but normally you should omit them so that default values will be used.

LanguageNames is useful. It allows you to define different format strings for different languages. Use this to list string identifiers of the languages that you'll use, the local ID of the language, and the compiled resource file that should be used. If you don't use this item, MC will mark the resource as international English (with an identifier of "English"). For example:

LanguageNames=(British=0x809:MSG00809)
LanguageNames=(French=0x40c:MSG0040c)

This example will cause MC to create three binary resources: one each for international English (by default in MSG00001.BIN), UK English (MSG00809.BIN), and French (MSG0040c.BIN).

In the message definition section, you give an entry for each message that you want to define. If you've specified more than one language, you must give each message a format string for each language and also for international English. So, for the previous example, for each message you must give a format string for "English," "British," and "French."

Each message definition can have the items in Table 2.

Table 2. Items in a message definition.

Each message must have the MessageId line and as many Language lines as the languages you're using. If you want to use the other items, define them once for each message. The MessageId line can take a value, but if it's empty, it will be incremented from the previous message definition (which assumes that there must have been a previous definition with a value for MessageId). After each language line, you enter the format string followed by a period on a new line. For example:

LanguageNames=(British=0x809:MSG00809)
LanguageNames=(French=0x40c:MSG0040c)
MessageId=0x1000
SymbolicName=MC_HELLO
Language=English
Hello! %1
.
Language=British
What ho! %1
.
Language=French
Salut! %1
.

This defines a single message with an ID of 0x1000 in each of three languages. When you compile this, you'll get—in addition to the bin files—a header and a resource script. The header will have the symbols for the messages—in this case, just the one:

#define MC_HELLO 0x00001000L

The resource script can be included in your resource file. For this message file, the resource script produced is:

LANGUAGE 0xc,0x1
1 11 MSG0040c.bin
LANGUAGE 0x9,0x1
1 11 MSG00001.bin
LANGUAGE 0x9,0x2
1 11 MSG00809.bin

I mentioned earlier that an event logged using ReportEvent() can have a category. You can use any integer you like, but if you want the NT EventLog Viewer to give a meaningful string for events of that category, you should create a resource file. These category strings are created just like the message format strings but should obviously be short, shouldn't have insert strings, and must start with a MessageId of 1. Thus, you could have a separate category file generated from:

MessageIdTypedef=WORD
MessageId=0x1
SymbolicName=CAT_GREETING
Language=English
Greeting
.
Language=British
Greeting
.
Language=French
Salutation
.

Notice here that I've specified MessageIdTypedef=WORD. This is because ReportEvent() uses WORD categories, and this ensures that the generated symbols are cast to WORD. The generated header file will contain:

#define CAT_GREETING ((WORD)0x00000001L)

If you balk at having a header creating a #define like this, you can always edit the header by hand. However, it does mean that you'll need to do this editing every time you add a new category. It's tricky to combine category and event ID definitions in the same file, because the event IDs should be DWORD and the MessageIdTypedef is global to all of the symbols specified with SymbolName. There's a way around it: Add the category symbol as a comment. Each line that starts with a semicolon is ignored by MC but is copied into the header (the source for MC in the Platform SDK erroneously remarks that a // is added to each comment). If your MC file looked like this:

;#define CAT_GREETING 1
MessageId=0x1
Language=English
Greeting
.
MessageId=0x1000
SymbolicName=MC_HELLO
Language=English
Hello! %1
.

You would get a header file with these two symbols:

#define CAT_GREETING 1
#define MC_HELLO     0x00001000L

As well, there will be many lines of MC-generated comments. However, the symbols for your categories and events will be in the same file and there won't be any horrible casts.

If you choose to provide a DLL as the message resource file, you should create a DLL with a stub entry point:

BOOL APIENTRY DllMain(HANDLE, DWORD, LPVOID)
{
    return TRUE;
}

The bin files generated by the message compiler should be bound as resources to the resource file. Whether you use an EXE or DLL as the message resource file, you need to register it. This is done by adding values in the registry entry for the source, as shown in Figure 1. The values you can add are shown in Table 3.

Table 3. Source registry entries.

EventMessageFile has the path to the resource file. As you can see from Figure 1, this can contain environment strings. TypesSupported is a combination of the message types that can be reported from this resource with ReportEvent(). Typically, you'll use a value of 7, which indicates that the source will generate EVENTLOG_ERROR_TYPE, EVENTLOG_WARNING_TYPE, and EVENTLOG_INFORMATION_TYPE events.

CategoryMessageFile is the path to the message resource file for the category descriptive strings. Because these are the same resource type as the event format strings, and the EventMessageFile and CategoryMessageFile can be the same file, you need to distinguish between the two types of resources. To do this, category IDs are incremented sequentially from 1, and to make sure that the NT Event Viewer doesn't use an event format string as a category string, you enter the number of categories in CategoryCount.

Finally, I mentioned that device drivers sometimes log events with insert strings that are placeholders. These placeholders are numbered identifiers to message string resources in the ParameterMessageFile. Typically, for user-mode applications, you won't log events using parameter insert strings. However, if you want to write an event log viewer, you'll need to take this into account.

Because you can't access the underlying EventLog files directly, Win32 provides functions to create backup copies that you can manipulate directly. BackupEventLog() takes an EventLog handle and the name of a file, and copies the event log to that file. You can open the file with CreateFile() or copy it to another location. You can use these backup files with ReadEventLog() if you get a handle by calling OpenBackupEventLog(). Another way to create a backup file is to call ClearEventLog(), which will also clear the underlying EventLog file.

The final API function you should be aware of is NotifyChangeEventLog(). You use this to get a notification when a new event is added to an event log. Use OpenEventLog() to open a log on the local machine, and pass this handle and a handle to an NT event kernel object to NotifyChangeEventLog(). Your application can now wait on this event object using WaitForSingleObject().

Now that you've seen the API functions, you'll want to know how to use them in practice. I've shown you how to log events this month, and next month, I'll show you how to read events. You can download a sample that includes a project called Greetings (available in the Subscriber Downloads at www.pinpub.com/vcd). This is the source for the event message file. I created this in Developer Studio as a Win32 Dynamic Link Library and created a C++ source file with a stub for DllMain().

The main point of the project is to compile the message file:

LanguageNames=(British=0x809:MSG00809)
LanguageNames=(French=0x40c:MSG0040c)

;#define CAT_GREETING 1
MessageId=0x1
Language=English
Greeting
.
Language=British
Greeting
.
Language=French
Salutation
.
MessageId=0x1000
SymbolicName=MC_HELLO
Language=English
Hello! %1
.
Language=British
What ho! %1
.
Language=French
Salut! %1

I added this as a source file to the project, and I added a custom build setting of mc $(InputName), giving the output files $(InputName).h, $(InputName).rc, and the three bin files. I compiled this once so that I could add the resource file to the project.

Next, I added a post build step to the project:

copy release\greetings.dll %SystemRoot%\System32
%SystemRoot%\regedit /s greetings.reg
echo Greetings.dll registered

The /s causes regedit to register the registry script without producing a confirmation dialog. This registration script is shown in Listing 1.

Listing 1. The registration script.

REGEDIT4

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventLog\Application\Greetings]
"EventMessageFile"="%SystemRoot%\\System32\\Greetings.dll"
"CategoryMessageFile"="%SystemRoot%\\System32\\Greetings.dll"
"TypesSupported"=dword:00000007
"CategoryCount"=dword:00000001

When you build this project, the message resource file will be copied to the system32 directory and the registry will be updated.

Now to use all of this. The following code is a command line project, called GreetingsTest, that takes one or two parameters. The first is the single insertion string for the message and the second is an account name. I use this in the code to get the SID of the account, and if no account is used, then "Guest" is used. Note that I've omitted the error checking that you'll find in the actual source code.

// GreetingsTest.cpp
#include <windows.h>
#include <stdio.h>
#include <tchar.h>
#include "..\Greetings\Greetings.h"

int _tmain(int argc, TCHAR** argv)
{
   if (argc == 2 || argc == 3)
   {
      LPTSTR strUser;
      if (argc ==2) strUser = _T("Guest");
      else          strUser = argv[2];
      HANDLE hEventLog;
      hEventLog = RegisterEventSource(NULL, 
         _T("Greetings"));
      PSID pSid = NULL;
      DWORD dwSidSize = 0;
      SID_NAME_USE eUse;
      LPTSTR DomainName = NULL;
      DWORD dwDomainName = 0;
      LookupAccountName(NULL,strUser, pSid, 
         &dwSidSize, DomainName, &dwDomainName, 
         &eUse);
      pSid = (PSID)new BYTE[dwSidSize];
      DomainName = new TCHAR[dwDomainName];
      LookupAccountName(NULL,strUser, pSid, 
         &dwSidSize, DomainName, &dwDomainName, 
         &eUse);
      ReportEvent(hEventLog, 
         EVENTLOG_INFORMATION_TYPE, CAT_GREETING, 
         MC_HELLO, pSid, 1, 0, (LPCTSTR*)&argv[1], 
         NULL);
      DeregisterEventSource(hEventLog); 
      delete [] pSid;
      delete [] DomainName;
   }
   return 0;
}

I ran this from the command line a few times, then checked EventViewer, which reported the Greeting events as seen in Figure 3.

Figure 3 is unavailable

The Category logged was 1. If you find that EventViewer shows (1) rather than "Greeting," it's because it can't find the category message file. There are two reasons for this: Either the path to Greetings.dll in the registry is incorrect, or EventViewer hasn't read the registry yet. To fix that, just restart EventViewer.

As you can see, the EventLog API is quite straightforward but rather tedious. Next month, I'll show you some classes to make writing events to, and specifically, reading event records from, the EventLog much simpler.

Download sample code here.

Richard Grimes is a freelance writer and consultant, the author of Professional DCOM Programming (WROX Press), and a co-author of Beginners ATL COM Programming (WROX Press). (For further details, see http://www.wrox.com.) He's previously worked as a trainer, as a developer for a UK-distributed objects software consultancy, and as a semiconductor researcher. dcom.dev@grimes.demon.co.uk.