How to Set Up a Remote Debug Session Using a Null Modem Cable

• Microsoft Windows NT Workstation version 3.50 and 3.51
• Microsoft Windows NT Server version 3.50 and 3.51

SUMMARY

This article is intended to describe the procedures for setting up the debugger on a Windows NT system using a Null modem cable attached to another Windows NT system.

MORE INFORMATION

The process of setting up the debugger with a Null modem cable is broken down into the following topics:

• What is the need for the kernel debugger?
• What is the process of remote debugging?
• Why use a Null modem cable?
• Changing the attributes on the Boot.ini file
• Debugger Options
• Configuring the debugger for the system
• Setting up the Null modem cable
• Configuring the Host machine for a Remote.exe session

What is the need for the kernel debugger?

The kernel debugger that comes with Microsoft Windows NT provides information about the system that is normally not available without the aid of the debugger. Windows NT has the ability to enable the kernel debugger from the Windows NT start-up menu (Boot.ini). Enabling the kernel debugger allows the Windows NT debugger to output debugging information to a remote debugger for a remote user to analyze. This is normally done at the request of a Microsoft Support Engineer for analyzing a fatal error in Windows NT that cannot be diagnosed from the Memory.dmp file or when a Memory.dmp file is not produced.

What is the process of remote debugging?

For our discussion the target machine will be the system that is being debugged and the host machine will be the system doing the debugging. The process of remote debugging happens when two computers are connected by the serial communication ports. The target computer and the host computer are running the Windows NT kernel debuggers, which communicate using a special debug API and Protocol.

Why use a Null modem cable?

Connecting the Null modem cable to a serial port allows the debugger to run at its current top speed of 19200, compared to the speed of 9600 over a modem. It is generally considered more reliable than using a modem for the following reasons: it does not involve configuring a modem, there is no possibility of a modem or line failure, a system can be debugged locally, and finally it provides faster debugging.

The most common method is to set up a target computer for debugging and the host computer for doing the debugging using a Remote.exe session. This configuration allows a technical support person to connect to the Remote.exe session by means of a RAS connection and debug the target computer without physically being present at the target computer.

Changing the attributes on the Boot.ini file

To set up the target computer, simply edit the Boot.ini file in the boot partition and add the appropriate debugger options. The Boot.ini is edited using a standard ASCII text editor. The Boot.ini is hidden in the boot partition directory. The Boot.ini attributes are hidden, read-only and system. These attributes must be changed to edit the Boot.ini file.

To change the attributes of the Boot.ini file:

At the command prompt, type attrib -h -r -s c:\boot.ini and press Enter, and then edit the file. To restore the read-only, hidden, and system attributes when you finish editing the file, type attrib +h +r +s c:\boot.ini at the command prompt.

- Or -

1. In File Manager, choose By File Type on the View menu. Select the Show Hidden/System Files box in the By File Type dialog box, and click OK.

2. In the File Manager window, select the Boot.ini file, and then choose Properties on the File menu.

3. In the Properties dialog box, clear the Read Only, Hidden, and System Attribute box and click OK.

4. Edit the Boot.ini using Notepad or another text editor.

Debugger Options

Once the Boot.ini on the target computer is opened for editing, several options can be added to the Boot.ini file to configure the debugger environment on the target computer. The various options available are listed and described below.

/DEBUG      - This option is used to enable the kernel debugger

              debugger. If no serial port is specified the debugger will
              default to COM2.

              fatal error has occurred.

/BAUDRATE   - Sets the Baud rate that the kernel debugger will use. The
              default baud rate is 19200. A baud of 9600 is the normal
              rate for remote debugging over a modem.

Configuring the debugger for the system

There are several options available for configuring the debugger on the target computer. To add any one of the previous options to the Boot.ini, use the following example.

The following is a typical Boot.ini. Each entry in the [operating systems] section should correspond to the options listed in the boot menu during a normal system startup.

[boot loader] timeout=30 default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS [operating systems] multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows NT" Version 3.5" multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows NT" Version 3.5 [VGA mode]" /BASEVIDEO C:\="MS-DOS"

To enable the debugger, use COM1 to do the following.

1. Select the startup option that you normally use and add either the /debug switch at the end of the line.

2. To specify the communications port, add the switch /debugport=com1

Setting up the Null modem cable

A Null modem cable is a serial cable that has been configured to allow data to be sent between two serial ports. Normally you connect a serial port to a serial device. In this case we need to configure the target computer to accept a serial-port-to-serial-port connection. It is important that a Null modem cable be used because if a normal serial cable is used the target system expects a serial device, not another serial port, and the system cannot be debugged.

When you use a NULL modem to make a direct serial connection between two computers, your cable must be wired as shown in the following tables.

9-Pin NULL Modem Cabling

Remote host serial port connector Host system serial port connector

Signal 3 2 Transmit Data 2 3 Receive Data 7 8 Request to Send 8 7 Clear to Send 6, 1 4 Data Set Ready and Carrier Detect 5 5 Signal Ground 4 6, 1 Data Terminal Ready 25-Pin NULL Modem Cabling

Configuring the Host computer for a Remote.exe session

The concept here is that the host computer is being set up to do a debug of the target computer as if there were a technical support person sitting at the host computer. The Remote.exe will be run to allow this debug session to be accessed remotely through a RAS connection.

The Remote.exe is a command line utility which allows you to run command-line programs on remote computers. Remote.exe uses two parts, the server component and the client component. To use Remote.exe, you must first start the server end on the computer where you are debugging from. This allows other users to connect to your debugging session using the client portion of Remote. The Remote.exe comes with the Resource Kit. For more details and the command syntax of this utility, refer to the Resource Kit online help.

The Windows NT installation compact disc and the Windows NT Resource Kit compact disc should be available to accomplish this setup.

1. The first thing should be done is to set up a RAS account on the network for the technical support person. This information, including the name of the host system, should be documented and ready to give to the support person.

2. The host computer should be configured as described in the following

   Article-ID: Q148659
   TITLE     : How to Set Up Windows NT Debug Symbols