How to Set Up a Remote Debug Session Using a Modem

• Microsoft Windows NT Workstation versions 3.5, 3.51, and 4.0
• Microsoft Windows NT Server versions 3.5, 3.51, and 4.0 -------------------------------------------------------------------------

SUMMARY

This article describes the procedures for preparing an X86 Intel-processor- based and RISC-processor-based computer running Windows NT Workstation or Server for a remote debug session.

MORE INFORMATION

• What Is the Need For the Kernel Debugger
• What Is the Process of Remote Debugging
• Configuring a System for Remote Debugging
• Debugger Options
• Editing the BOOT.INI file for Intel x86 Computers
• Changing Firmware Boot Options for RISC Computers
• Preparing the Modem

What Is the Need For the Kernel Debugger

The kernel debugger that comes with Windows NT will produce information about the system that is normally not available without the aid of the debugger. You can enable the kernel debugger from the Windows NT start-up menu. Enabling the kernel debugger will allow the Windows NT debugger to output debugging information to a remote debugger for a remote user to analyze. This is normally done at the request of a Microsoft Customer Support Engineer for analyzing a fatal error in Windows NT that cannot be diagnosed from the MEMORY.DMP file or when a MEMORY.DMP file is not produced.

What Is the Process of Remote Debugging

The process of Remote debugging happens when two computers are connected together via modems over a phone line. The target computer and the host computer are running the Windows NT kernel debuggers which communicate using a special debug API and Protocol.

Configuring a System for Remote Debugging

Configuring a system for remote debugging involves changing the boot options to tell Windows NT to load the kernel debugger. On an Intel x86 based platform this is done by editing the BOOT.INI file. On a RISC based system (DEC Alpha, MIPS and PPC based processors) this involves changing the boot options in the firmware menu. You will also need to have a modem and connect it to the appropriate COM port on the target computer and connect an inbound phone line to the modem.

Debugger Options

There are several boot options which can be used in configuring the system for debugging. These options are the same on Intel x86 and RISC platforms, although when used on a RISC platform the / is not required:

/DEBUG      - When this option is used, the kernel debugger will be loaded
              during boot and kept in memory at all times. This means that
              a support engineer can dial into the system being debugged
              and break into the debugger, even when the system is not
              suspended at a Kernel STOP screen.

              debugger. If no serial port is specified the debugger will
              default to COM2 on Intel x86 based computers and to
              COM1 on RISC computers.

              during boot but swapped out to the pagefile after boot.
              As a result, a support engineer will not be able to
              break into the debugger unless Windows NT is suspended
              at a Kernel STOP screen.

/BAUDRATE   - Sets the Baud rate that the kernel debugger will use.
              The default baud rate is 19200. A baud of 9600 is the
              normal rate for remote debugging over a modem.

Editing the BOOT.INI file for Intel x86 Computers

To set up the target computer on an Intel x86 based computer, simply edit the BOOT.INI file in the boot partition and add the appropriate debugger options. The BOOT.INI is edited using a standard ASCII text editor. The BOOT.INI file is a file found on the root of the system partition (generally the C drive) and will have the Hidden, System, and Read-Only attributes set. These attributes must be changed.

To Change the Attributes of the BOOT.INI File

To change the attributes of the BOOT.INI file, follow Procedure I or II:

Procedure I:

1. Type the following at a command prompt:

         attrib -s -h -r c:\boot.ini
   

2. Edit BOOT.INI.

   attrib +h +r +s c:\boot.ini \

1. In File Manager, choose By File Type in the View menu, and then select the Show Hidden/System Files box in the By File Type dialog box, and choose the OK button.

2. In the File Manager window, select the BOOT.INI file, and then choose Properties from the File menu.

3. In the Properties dialog box, clear the Read Only, Hidden, and System attribute box and choose the OK button.

4. Edit the BOOT.INI using Notepad or another text editor and add the debug options as described below.

Configuring the Boot Options in the BOOT.INI

To configure a computer for debugging you must add one of the previous options to the BOOT.INI. Generally you will want to add /DEBUG and /BAUDRATE, to lower the baud rate to 9600 and put the computer in debug mode. You should use /DEBUGPORT=COMX if you cannot use the default COM port (COM2) for debugging. The following example shows how to add these options.

The following is an example of a typical BOOT.INI. Each entry in the [operating systems] section should correspond to the options listed in the boot menu during a normal system startup:

   [boot loader]
   timeout=30
   default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
   [operating systems]
   multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows NT Version 3.5"
   multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows NT Version 3.5"
   [VGA mode]  /BASEVIDEO
   C:\="MS-DOS"

1. Select the startup option that you normally use and add the /DEBUG switch at the end of the line.

2. To specify the communications port, add the switch /DEBUGPORT=COM1.

3. To specify the baud rate, add the switch /BAUDRATE=9600.

   [boot loader]
   timeout=30
   default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
   [operating systems]
   multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows NT Version 3.5"
   /debug /debugport=com1 /baudrate=9600
   multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows NT Version 3.5"
   [VGA mode] /BASEVIDEO
   C:\="MS-DOS"

Changing Firmware Boot Options for RISC Computers

Unlike the Intel x86 computers, which use a BOOT.INI file, boot options on RISC computers are configured through the firmware. The following steps walk you through configuring the boot options on MIPS-, PPC- and DEC Alpha- based computers. Be sure to carry out step 10 for all DEC Alpha computers before a support engineer attempts to dial in and debug the system.

1. Restart the computer. The ARC System screen appears, displaying the main menu from which you can select an action.

2. On a MIPS and PPC RISC-based system, choose Run Setup to display the Setup menu and then choose Manage Startup to display a menu of the boot options. On a Digital Alpha AXP RISC-based system, choose Supplementary Menu, Set Up the System, and Manage Boot Selection Menu to display a menu of the boot options.

3. Choose Change a Boot Selection to display a list of the operating systems that are installed on this computer.

4. Choose the Windows NT operating system. If you have more than one version of Windows NT installed, select the one that you want to debug. A two-part screen appears for changing the current settings of the environment variables used to start the RISC-based computer. The environment variable that controls whether or not the RISC-based computer starts up in debug mode is the OSLOADOPTIONS variable.

5. To edit the value for the OSLOADOPTIONS variable, use the arrow keys to select it from the list of variables. Once selected, it appears in the Name box at the top of the screen.

6. Press ENTER to display the Value box.

7. Type the options you want to add in the Value box, separated by spaces and press ENTER to save them. For example, if you want to turn on DEBUG mode and set the COM port to COM2, you type the following:

         OSLOADOPTIONS  debug  debugport=com2
   

   If you do not specify the debug port, the default debug port is set to COM1.

   Since RISC-based computers allow only a default baud rate of 19200, you do not need to specify the baud rate.

8. Press ESC to stop editing.

9. Based on the type of computer you have, do one of the following:

       - On a MIPS or PPC RISC-based system, choose Return To Main Menu and
         then Exit to return to the ARC System screen.
   

         -or-
   

       - On a Digital Alpha AXP RISC-based system, choose the option
         Supplementary Menu, save your changes and then choose Boot Menu to
         return to the ARC System screen.
   

10. If this is the first time you debug this system you need to carry out

        additional configuration steps. For these steps, consult the Windows NT
        Resource kit Version 3.51 Update or Update 2, Appendix A.
    

11. Restart the computer to run under Windows NT.

Preparing the Modem

To connect to the remote debugger successfully you must connect an external modem with auto-answer capabilities to the desired COM port. Most modems have this feature, but it is generally disabled. The modem should be set to 9600 baud. For a proper connection, the kernel debugger requires that hardware compression as well as error detection and flow control be turned off. Make sure the modem works correctly before proceeding any further. In most cases you will need to configure the modem by connecting to a spare COM port on the target machine or another machine, using TERMINAL.EXE or some other communication program to send configuration strings to the modem, then moving the modem to the target machine and COM port without turning off the modem. This is why an internal modem will not work, as rebooting the system will generally reset any of the changes you have made to the modem settings.

To configure the modem:

   NOTE: You can carry out all these steps on the target computer, using
   the COM port you want to configure as the debug port provided you carry
   them out before configuring the debugger on the target system.