Setting Up for Debugging

If you decide to use the kernel debugger to analyze the kernel STOP error, you need to set up the host and connect your host and target computers. To do this, you use either a null modem cable for a local debug session or a modem cable for a remote debug session. Before you can start debugging, you must complete several steps.

To prepare for debugging

1. Set up the modem connection.

2. Configure the target system for debugging.

3. Set up a symbol tree on the host system.

4. Set up the debugger on the host system.

5. Start the debugger on the host system.

Note

None of the procedures in this section are necessary if you use the Recovery dialog box to create a memory dump file. For information about that alternative, see "Creating a Memory Dump File," later in this chapter.

Setting Up a Remote Debugging Session on an Intel-Based Computer

If you enable the kernel debugger on your target computer, it sends debugging information to a host computer for a remote user to analyze. A support engineer often requests this to help analyze a fatal error in Windows NT that cannot be diagnosed from the Memory.dmp file or if a Memory.dmp file is not produced.

The process of remote debugging occurs when two computers are connected by means of modems over a phone line. The target and the host computer can thus communicate by using a special debugging API and protocol.

The following figure shows the connection between the host and the target computer for a remote debugging session.

Figure 39.1 Remote Debugging

To configure a system for remote debugging, you change the boot options to set Windows NT to load the kernel debugger. On an X86–based platform, you do this by editing the Boot.ini file. On a RISC-based system (DEC Alpha, MIPS and PowerPC processors), you change the boot options in the firmware menu. You must also connect an external modem to the appropriate COM port on the target computer and connect an inbound phone line to the modem.

Booting the Target Machine

If the target computer stops at a blue screen every time you boot it, or does not keep running long enough for you to edit the Boot.ini file to enable the debugger, you can try these options:

• If your boot partition is FAT, you can start MS-DOS from a boot floppy disk and use the MS-DOS-based editor to edit Boot.ini.

• If your boot partition is NTFS (or HPFS, if you are running Windows NT version 3.1 or 3.5), you can install Windows NT on a different partition and boot from that partition. (You must use this method because you cannot access files on an NTFS or HPFS partition from MS-DOS.)

• If you previously created a Windows NT boot recovery disk for the workstation that has the problem, you can use this disk on another machine to edit the Boot.ini file, and then boot the target machine.

Setting Up the Modem on the Target Machine

To set up a remote debugger session, you must connect an external modem to the target machine and reconfigure the modem parameters to meet the requirements of the kernel debugger. To configure the modem, you must be able to run Terminal.exe or some other communications program. If you are unable to run these programs on the target machine, connect the modem to a computer that is close to the target machine. Make sure you can move the modem back to the target machine without losing power to the modem. An internal modem does not work because rebooting the system resets the configuration changes you have made to the modem.

The modem must be connected to a spare COM port and must be configured as shown in the following table:

Consult your modem documentation for the correct string values to send to the modem during the configuration process. The following table gives an example of how to configure a USRobotics modem for a remote debugging session.

To configure the modem

1. Connect the modem to an unused COM port on the target machine or on another computer that is close enough to the target machine to connect by using a standard modem cable.

Note

If you connect the modem to a computer other than the target machine, make sure you can move the modem back to the target COM port without removing power from the modem.

2. Run Terminal.exe or some other communications program to configure the modem parameters.

3. Set the modem speed to 9600 bps. See your modem documentation to find out how to do this.

4. Turn off all hardware compression, flow control, and error detection.

How to do this varies widely from modem to modem. See your modem documentation for the correct strings to send to the modem.

5. Enable auto-answer by sending the string ATS0=1 to your modem. Consult your modem documentation to verify that this will work with your modem.

6. If the modem was configured on a machine other than the target computer, move it to the target computer without removing the power from the modem.

Editing the Boot.ini File on the Target Machine

To configure a target system for a remote or local remote debugging, you edit the boot options in the Boot.ini file to tell Windows NT to load the kernel debugger.

Debugger Options

The following table lists the boot options that can be used to configure the system for debugging. These options are the same on Intel X86 and RISC platforms, but the slash (/) is not required when used on a RISC platform.

When you use Debugport or Baudrate, you need not use Debug, as Windows NT assumes that the computer will load in Debug mode. You must use at least one of the options described in Table 39.1 to configure a computer for remote debugging. Otherwise, Windows NT does not load the debugger at all.

To set up the target computer on an Intel X86-based computer, edit the Boot.ini file by using a standard ASCII text editor and add the appropriate debugger options to the file. The Boot.ini file is located in the system root directory (usually the C drive) and has the Hidden, System, and Read-Only attributes set. These attributes must be changed.

To Change the Attributes of the Boot.ini File

1. Type the following at a command prompt:

attrib -s -h -r c:\boot.ini

2. To restore the Read-Only, Hidden, and System attributes when you finish debugging the system, type the following at a command prompt:

attrib +h +r +s c:\boot.ini \

To Configure the Boot Options in the Boot.ini File

To configure the target computer for remote or local debugging, add the /Debug and /Baudrate options to the Boot.ini file. If you cannot use the default COM port (COM 2) for debugging, use /Debugport=COMx where x is the COM port number. Use the MS-DOS-based Editor to edit the Boot.ini file.

1. At a command prompt, type:

edit boot.ini

The Boot.ini file appears in the MS-DOS Editor window. It looks similar to this:

[boot loader]

timeout=30

default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS

[operating systems]

multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows NT Version 4.0"

multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows NT Version 4.0"

[VGA mode] /BASEVIDEO

C:\="MS-DOS"

2. Select the startup option that you normally use and add the /Debug option at the end of the line.

3. To specify the communications port, add the option /Debugport=comx where x is the communications port that you want to use.

4. Add the option /Baudrate=9600.

This is the output if the Boot.ini file after it has been modified by steps 1-4:

[boot loader]

timeout=30

default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS

[operating systems]

multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows NT Version 4.0" /debug /debugport=com1 /baudrate=9600

multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows NT Version 4.0"

[VGA mode] /BASEVIDEO

C:\="MS-DOS"

5. Save the Boot.ini file and quit the text editor or the MS-DOS Editor.

6. Restart the computer to run under Windows NT.

Your technical support group can now call the modem to establish the remote debugging session.

Setting Up a Remote Debugging Session on a RISC-Based Computer

To prepare a RISC-based computer for a remote or local kernel debugging session, you edit one line in a startup file. But you access that file in a different way. The procedure for all Alpha systems is the same. The options you use to configure the PowerPC-based system are the same as the options you select to configure the MIPS-based system. However, the path to the firmware menus may vary for MIPS-based and PowerPC-based systems.

On RISC-based computers, the default COM port is always COM1, and the default speed is always 19200 bps.

Before you begin the procedure to configure the rarget machine, make sure you set it up properly for communication. If you cannot run Terminal.exe or any other communications programs on the target machine, connect the modem to a computer that is near the target machine. Make sure that you can move the modem back to the target machine without removing the power to the modem.

All modem parameters are configured for a RISC-based computer in the same way as they are for an X86-based system with the exception of the modem speed. The default speed is always 19.2 kbps for a RISC-based system. For more information, see "Setting up the Modem on the Target Machine," earlier in this chapter.

After you have set up your computer for communication, restart the computer. The ARC System screen appears, displaying the main menu, from which you can select an action. Now you are ready to configure.

To configure the target machine

1. On a MIPS RISC-based system, select Run Setup to display the Setup menu, then select Manage Startup. A menu of boot options appears.

On a Digital Alpha AXP RISC-system or a PowerPC RISC-based system, select the menu options listed in the following table to get to the Boot selections menu.

2. On the Boot Selections menu, select Change a Boot Selection. A list of the operating systems that are installed on this computer appears.

3. From the list of operating systems, select the Windows NT operating system. If you have more than one version of Windows NT installed, select the version that you want to debug.

A two-part screen appears with options for changing the current settings of the environment variables used to start the RISC-based computer. The environment variable that controls whether or not the RISC-based computer starts up in debug mode is the OSLOADOPTIONS variable.

4. Select the OSLOADOPTIONS variable from the list of environment variables.

You edit the value of the OSLOADOPTIONS variable to control whether the RISC-based computer starts up in debug mode.

After you select OSLOADOPTIONS, it appears in the Name box at the top of the screen.

5. Press enter to display the Value box.

6. Type the options that you want to add in the Value box separated by spaces. Press enter to save them and to turn on the debug mode.

You can also add a value that explicitly sets the communications port, as in the following example:

OSLOADOPTIONS debug debugport=com2

If you do not specify the debug port, the default debug port is set to COM1. Because RISC– based computers allow only a default modem speed of 19.2 Kbps, you do not need to specify the baud rate.

7. Press Esc to stop editing.

8. Return to the ARC System screen by using the method for your system:

If this is the first time that you have debugged a Digital Alpha AXP RISC–based system, follow these steps after connecting the local host computer to the target:

• Shut down both computers.

• Restart the host (debugger) computer.

• Run Alphakd.exe on the local host.

• Restart the target (Digital Alpha AXP RISC-based) computer while Alphakd.exe is running on the host computer to set up configuration information on the target computer, and prepare it for either local or remote debugging.

Note

After you complete steps 1-4, you can use either a local or a remote host to debug the target.

9. To run under Windows NT, restart the RISC-based computer.

You may now contact your technical support group or a trained technician and have them call the modem to establish a remote debugging session.

Setting Up a Local Debugging Session on a Host Computer

You need a local debug session for debugging in cases where a user-mode .dll or a device driver is causing server crashes. In such a case, you use a user-mode debugger (such as NTSD) and you build the server symbols on the host computer.

You can also use this setup if your Remote Access Service (RAS) account allows a Microsoft Support engineer to dial into your network and debug the computer. This debug option overcomes many modem-related issues.

You use a local debug setup in cases where:

• You debug a user-mode component in Windows NT by using NTSD or CDB.

• A live remote debug does not work because of modem connection issues.

• Customer has worked with a senior ESS debug engineer and the situation warrants a local debug session.

To debug a Windows NT–based target computer by using a local host system, you need to:

• Connect the host and the target computers by using a null-modem serial cable.

• Set up a symbol tree on the local host computer to match the version of Windows NT that resides on the target computer. If you are using NTSD or CDB, you will need to set up a symbol tree on the target computer, in the directory %SYSTEMROOT%\Symbols.

• Set up the debugging files on the host computer.

• Start the debugger on the host.

Figure 39.2 shows the connection between the host and the target computer for a local debugging session. It also shows how to use your RAS account to connect to the Microsoft Support Network for help in analyzing the debug information.

Figure 39.2 Local Debugging

Setting Up for Local Debugging

To set up for a local debugging session, you use a null-modem cable to connect the target and the host machines. For an x86-based system, the boot options in the Boot.ini file must be configured on the target machine to invoke the debugger and to set the data transfer rate between the target computer and the host computer. On a RISC-based system, the boot options are configured from a firmware menu.

For information on configuring the boot options for an x86-based system, see "Editing the Boot.ini File on a Target Machine," earlier in this chapter. For information on configuring a RISC-based system for a local debug session, see "Setting Up a Remote Debugging Session on a RISC-Based Computer," earlier in this chapter.

Be sure to start the host computer before restarting the target computer.

Setting Up a Null-Modem Connection

A modem is not used in a local debug session. Therefore, the procedure for setting up the null-modem cable is the same on both the host computer and target computer.

A standard, commercially available null-modem serial cable has this configuration:

• Transmit Data connected to Receive Data

• Receive Data connected to Transmit Data

• Ground connected to Ground

For 9-pin and 25-pin D-subminiature connectors (known as db9 and db25, respectively), the cable connects as follows:

• Pin 2 to pin 3

• Pin 3 to pin 2

• Pin 7 to pin 7

The debugger on the host does not depend on any control pins (such as Data Terminal Ready, Data Set Ready, Request To Send, or Clear To Send). However, you might need to put a jumper in the connectors on both ends of the cable from Data Terminal Ready to Data Set Ready and from Request To Send to Clear To Send, as follows:

Connect the null-modem cable to an unused serial port on both the host computer and the target computer.

Setting Up the Symbol Tree on the Host

You set up the symbol tree on the host machine to match the version of Windows NT that you are running on the target computer.

The Windows NT Server and Windows NT Workstation product CDs come with symbol trees already created. They are in the Symbols directories on the CD under Support\Debug\platform, where platform is I386, Alpha, MIPS, or PowerPC. The platform sprcification must match your target computer.

If you have not installed any service packs or hot fixes and do not have a multiprocessor system, you might need to specify only the path to the correct Symbols directory on the CD, or copy that directory to \Systemroot and use this as the symbol path.

If you have installed service packs or hot fixes to Windows NT, or if you are using any HAL (Hardware Abstraction Layer) other than the standard, single-processor HAL, you must construct a symbol tree.

To construct a symbol tree

1. Copy the correct tree from the Support directory on the CD to your hard drive.

2. Copy the symbols into this tree for the updates you have applied in the same order in which you applied the updates, so that the later versions overwrite the earlier versions.

3. If you are using kernel debuggers to debug a multiprocessor system, or a single-processor system that is using a special HAL, you must rename some of the symbol files. The rest of this section discusses what to rename and how to rename it.

The kernel debuggers always load the files named Ntoskrnl.dbg for kernel symbols and Hal.dbg for HAL symbols. Therefore, you need to determine which kernel and HAL you are using, and rename the associated files to these filenames.

If you have a multiprocessor computer, you only need to rename Ntkrnlmp.dbg to Ntoskrnl.dbg. These files are in the \Exe subdirectory of the symbol tree.

If your computer uses a special HAL, there are a number of possibilities. Tables 39.2-39.5 list the possible HAL files for each hardware platform. These tables list the actual name of the .dll file as it exists on the product CD and the uncompressed size of the file in bytes. Each .dll file has a corresponding .dbg file, which is in the \Dll subdirectory of the symbol tree. Determine which HAL you are using, and rename the associated .dbg file to Hal.dbg. If you are not sure which HAL you are using, compare the file size in the table with the Hal.dll file on the target system. The Hal.dll file can be found in \Systemroot\System32.

Table 39.2 HAL Files for I386 Systems

Table 39.3 HAL Files for DEC Alpha Systems

Table 39.4 HAL Files for MIPS Systems

Table 39.5 HAL files for PowerPC Systems

In some cases, a HAL file might have been supplied by your computer manufacturer. If so, you need to obtain symbols for the file from the manufacturer, rename that symbol file to Hal.dbg, and place it in the \Dll subdirectory of the symbol tree. For example, Compaq provides updated HAL files for their Proliant™ systems. This also applies if you have drivers from third-party sources. Obtain symbols from your third-party vendor and put them in the appropriate directory.

Setting Up the Debugger Files on the Host

To set up the debugger on the host, first ensure that you have the correct files available. Copy these files from the Support\Debug\platform directory to a debug directory on the hard drive, where platform matches the platform of the host computer.

Some files that you copy from the directory must match the platform of the target computer, as described in the following table. These files are necessary for kernel debugging.

For instance, if your host computer is a 486 computer and the target computer is a MIPS RISC-based system, you copy the following files from the \Support\Debug\I386 directory:

• Mipskd.exe

• Imagehlp.dll

• Kdextmip.dll

Once you have set up the symbol tree and copied the necessary files to it, use a batch file or command line to set the following environment variables on the host:

After these environment variables have been set, you can start the host debugger.

Note

Setting the _NT_LOG_FILE_OPEN variable does not always result in a log file being written. You can also create the log file from the debugger. The command format is:

.logopen pathname

You might also need to issue the !reload command to get this to work.

Starting the Debugger on the Host

You can start the host debugger from the command line or a batch file by using the name of the executable file as the command. Each debugger supports the following command-line options:

The most commonly used options are -v (verbose) and -m (for modem debugging).

Generally, the best way to start the debugger is to create a batch file with the necessary commands to set the environment variables, followed by the command to start the correct kernel debugger.

Using the Remote Utility to Start the Debugger

If the host computer is connected to a network, you can use the remote utility, included in the Windows NT Resource Kit, to start the debugger. Remote is a server/client utility that provides remote network access by means of named pipes to applications that use STDIN and STDOUT for input and output. Users at other computers on the network can then connect to your host debugger session and either view the debugging information or enter commands themselves. The syntax for starting the server (host) end of the remote session is as follows:

remote /s "command" Unique_Id [/f foreground_color|/b background_color]

For example:

REMOTE /S "i386kd -v" debug

You end the server session by entering the @K command.

To interact with this session from some other computer, use the remote /c command. The syntax of this command is as follows:

remote /c ServerName Unique_Id [/l lines_to_get|/f foreground_color|/b background_color]

To exit from the remote session on a client and leave the debugger running on the host computer, enter the @Q command.

For example, if a session with the ID debug was started on the host computer \\Server1 by using the remote /s command, you can connect to it with the command

REMOTE /C server1 debug

For more information on using the remote command, see the Rktools.hlp file on the Windows NT Resource Kit CD.

Examples

Assume the following:

• Debugging needs to take place over a null-modem serial cable on COM2.

• The symbols are on a CD on the E drive.

• A log file called Debug.log is to be created in C:\Temp.

Note

The log file holds a copy of everything you see on the debug screen during your debug session. All input from the person doing the debugging, and all output from the kernel debugger on the target system, is written to the log file.

A sample batch file for local debugging is:

REM Target computer is local
set _NT_DEBUG_PORT=com2
set _NT_DEBUG_BAUD_RATE=19200
set _NT_SYMBOL_PATH=e:\support\debug\i386\symbols
SET _NT_LOG_FILE_OPEN=c:\temp\debug.log
remote /s "i386kd -v" debug

The last line of the batch file uses the remote utility to start the host debugger. If you use this, users of Windows NT–based computers who are networked to the host computer (and who have a copy of the remote utility) can connect to the debug session by using the command:

remote /c computername debug

where computername is the name of the host computer.

To allow remote debugging, which requires the use of a modem, begin with the batch file in the previous example. Change the baud rate to 9600, and add the -m switch to the last line. The result is as follows:

REM Target computer is remote from the host
set _NT_DEBUG_PORT=com2
set _NT_DEBUG_BAUD_RATE=9600
set _NT_SYMBOL_PATH=e:\support\debug\i386\symbols
SET _NT_LOG_FILE_OPEN=c:\temp\debug.log
remote /s "i386kd -v -m" debug

You run the batch file from the directory that contains the debugger files.

When you start the debugger, one of two screens appears, depending upon whether you are doing local debugging or remote debugging.

When doing local debugging, the following screen appears:

**************************************
***********   REMOTE  ************
***********   SERVER  ************
**************************************
To Connect: Remote /C BANSIDHE debug

Microsoft(R) Windows NT Kernel Debugger
Version 3.51
(C) 1991-1995 Microsoft Corp.

Symbol search path is:
KD: waiting to connect...

At this screen, you can press ctrl+c to gain access to the target computer, if it is still running. If the target is currently stopped at a blue screen, you will probably gain access automatically. If you have any problems, press ctrl+r to force a resync between the host computer and the target computer.

If you are doing remote debugging, the same screen as shown for local debugging appears, with the following extra line:

KD: No carrier detect - in terminal mode

In this case, the debugger is in terminal mode, and you can issue any of the standard AT commands to your modem. Begin by sending commands to disable hardware compression, flow control, and error correction. These commands will vary from modem to modem, so consult your modem documentation. Once you connect to the target system and have a carrier detect (CD) signal, you are returned to the debugger.