Marc Adler
Many PC applications have a command that permits you to escape to the operating system without having to terminate the application. While working on a document with a word processor, you might suddenly realize that one of the files you need to import into your document is archived. This means you have to go to the operating system, dearchive the file, and return to the point where you left off in the word processor. For another example, many program editors allow you to compile the file that you are editing. To do this, the editor must load the compiler into memory, run the compiler, and return you to the point in your file where you were editing.
The Microsoft C run-time library provides functions that allow you to suspend your program temporarily, load another program into the memory area that the current application does not occupy, and run the new program. To run a new program, simply make the following call:
system("myapp");
The system call will load a copy of COMMAND.COM above your application. The system will pass its string argument to COMMAND.COM to be used as the command line. COMMAND.COM will then attempt to execute this command.
However, if myapp is a large program, then COMMAND.COM will probably be unable to execute it. This is usually because there is not enough contiguous unallocated memory left in your system to load the new program. Resident in low memory are the ROM BIOS, DOS device drivers and buffers, and the resident portion of COMMAND.COM, which can be from 20Kb to 60Kb in size depending on which version of DOS you are running. Any Terminate-and-Stay-Resident (TSR) programs will also take up memory. Last is the code and data space taken up by the current application. Most applications allocate and free chunks of memory while running, causing a problem known as fragmentation. (For more on memory management and fragmentation, see "Creating a Virtual Memory Manager to Handle More Data in Your Applications," MSJ Vol. 4 No. 3.) Memory is fragmented when free blocks of memory are scattered among allocated blocks. Unfortunately, DOS1 does not perform garbage collection and compaction on its own, and neither do most applications. The result is that DOS is sometimes unable to load a program not because of a lack of free memory, but because of a lack of contiguous free memory.
One solution is to take a complete snapshot of the application's memory image, store it somewhere, free all of the memory taken up by the application, and run the new program. When the new program is finished, the memory snapshot is loaded back into RAM. Then the old application should be exactly as it was before it was swapped out-byte for byte.
This approach has been implemented in software developed by Thomas Wagner of Ferrari Electronic GmbH in Berlin, West Germany. (The code can be downloaded from any MSJ bulletin board.) One of the nice features of this code is that it permits you to swap your program to either a hard disk or expanded memory. If you have plenty of expanded memory free, the swapping is very fast, and the user perceives almost no delay. When a program is swapped out, it leaves only about 1Kb of code in memory.
Swap API
Applications must make a single call to swap a program out and load another one in. The function is defined as follows:
int pascal do_exec(char *pszCommand,
char *pszArgs,
int iMode,
unsigned uMemNeeded,
char **pEnviron)
The name of the file to execute is pszCommand. For instance, using the string "cl" would invoke the C compiler. If an absolute pathname is not given, this function will search for the program in the current directory, and then search the directories specified in the DOS PATH environment variable. Do_exec will try to locate a file with a COM extension first and then a file with the EXE extension. The function will not attempt to execute batch files.
If an empty string (that is, a string with \0 in the first byte) is passed as the first argument to do_exec, a copy of COMMAND.COM (or whatever command interpreter is specified in the COMSPEC environment variable) will be executed.
The second argument to do_exec, pszArgs, is a string containing the arguments to the program. If you're invoking the C compiler, pszArgs might be a string like this:
/c /AL foo.c
The third argument, iMode, tells do_exec how to swap out the old application. If iMode is 0, you will be returned to the DOS level instead of the old application when the new application is terminated. If iMode is 1, EMS will not be used for swapping. If iMode is 1 (which it should be in most cases), the function will return to the old application after the spawned program terminates.
The fourth argument, uMemNeeded, specifies the maximum amount of free memory required by the new application. If this amount is greater than the amount of contiguous free memory in the system at the time of the swapping, the old application will be swapped out to disk or to EMS. However, if there is enough free memory, the old application will not be swapped. This means swapping will never take place for a uMemNeeded of 0, while a value of FFFFH for uMemNeeded will always cause swapping to take place, regardless of the amount of free memory available.
The last argument, pEnviron, specifies the environment to pass to the new program. If this argument is NULL, a copy of the parent application's environment will be passed to the new program. Otherwise, pEnviron must point to an array of pointers to environment strings, with the final member being NULL.
The sample call to do_exec, shown below, swaps out the current application and spawns a copy of the C compiler:
int rc;
rc = do_exec("cl", "/c /AL foo.c", 1, 0xFFFF, NULL);
Now let's examine the C function do_exec (see Figure 1) and the lower level assembly language routine do_spawn (see Figure 2). For the most part, these routines prepare data structures for the DOS EXEC function, which is interrupt 21H, service 4BH. (For more on this function, see "Everything You Always Wanted to Know about the MS-DOS EXEC Function. . . ," MSJ, Vol. 4 No. 1.)
The first thing that do_exec does is copy the argument string into a separate buffer and prefix the buffer with a 1-byte length indicator. The DOS EXEC service expects to see the parameter list in this form. Unfortunately, DOS puts a 128-byte restriction on the length of the parameter string. In addition, a length specifier is placed before the name of the command to execute.
If the calling program specifies a custom environment to pass to the new program, you must allocate a buffer large enough to hold all the environment strings. Each NULL-terminated environment string is copied in sequence into the buffer; the final environment string is terminated with two NULL bytes.
The amount of free memory available can be determined by calling the DOS memory allocation function interrupt 21H, service 48H. If you pass the value FFFFH as the number of memory paragraphs to allocate, this function will always fail (since FFFFH x 16 bytes = 1Mb of conventional memory). However, as a result of this, the function instead returns the amount of memory available in the system. If this amount is less than what the new program needs, the old program will need to be swapped out.
While swapping the old program to disk, a temporary file with the unlikely name of $$AAAAAA.AAA will be created. The DOS environment is searched for a variable named TMP or TEMP; if found, the swap file is created there. These environment variables usually specify a directory that applications can use to store temporary information in (such as virtual memory swap files or spooler output) while they are running. If these environment variables are not found, the swap file is created in the current directory.
Next the assembly language routine do_spawn is called to perform the actual swapping. The do_spawn routine is called with the iMode argument, the name of the swap file, the name of the command (with the length-specifier byte), the parameter string (also with the length-specifier), a copy of the environment, and the length of the environment.
Here is an overview of what do_spawn does. First, do_spawn traverses the memory control blocks allocated to the current program, adding up the size of all the blocks in the process. If swapping to expanded memory, the routine determines how many 1Kb blocks of expanded memory to allocate and allocates the expanded memory pages. If instead you're swapping to disk, do_spawn creates the temporary disk file. Next, the routine traverses the memory control block chain, writing the contents of the memory allocated to the control block to disk or to expanded memory. The routine then moves its reload code to low memory and executes the new program. When control returns from the executed program, do_spawn reads the memory image back in from disk or from expanded memory and resumes execution of the original application.
Memory Control Blocks
To understand how do_spawn works, you must be familiar with memory control blocks. When a program asks DOS for some memory, DOS allocates the memory, then returns the address of the free block to the program. DOS places a 16-byte data structure, called a memory control block (MCB), before the allocated memory area. The MCB is used to record information about the allocated block. The structure of an MCB is shown in Figure 3.
The first byte in the MCB indicates whether the block is the last block in the MCB chain: if it is not the last block in the chain, it's the letter M; if it is the last block, it's the letter Z.
The next two bytes contain the program identifier of the program requesting the memory allocation. Every program DOS loads is given a 256-byte header called a Program Segment Prefix (PSP). The PSP contains information such as the length of the program, a copy of the program's command string, and so on (see Figure 4). The PSP is always located on a paragraph boundary. Since there is a unique PSP for each program that is running, it's convenient to use the segment where the PSP is located to identify a program.
The next two bytes of the MCB contain the number of 16-byte paragraphs allocated to this block. The MCB's remaining 11 bytes are not used by DOS. The do_spawn routine takes advantage of this, using the last 8 bytes of the MCB to store some of its own information. It is always dangerous to rely on the undocumented features and data structures of DOS, but do_spawn has been used successfully under DOS 2.1 through DOS 4.01.
As mentioned, do_spawn will move its reload code down to low memory, starting at the 92nd byte of the swapped program's PSP. Do_spawn can do this because the information that had been at PSP+92 is not needed while the spawned program is running. Therefore, do_spawn saves it in a special area. Then the reload code and data from the swapped program can be stored in PSP+92 while the other program is running.
Do_spawn traverses the program's MCB chain, gathering information about each memory block. As mentioned above, the last eight bytes of each MCB are used to store some of do_spawn's information. The revised structure of the MCB is shown in Figure 5.
The do_spawn routine starts at the first MCB attached to the program, located 16 bytes before the PSP. It then marches down the MCB chain, and for every MCB that has been allocated to this program, it records information about the MCB in the last eight bytes. It also determines the number of 1Kb blocks that must be allocated from expanded memory to store the old program in.
You might ask why do_spawn needs to create its own chaining information, given that DOS maintains the regular MCB chain. Assume that a TSR program is active in your system while this program is running. Since a TSR can ask DOS for a block of memory at any given moment, another program's memory blocks might be found in this chain. DOS could easily allocate a block of memory from your program's memory space and give it to the TSR. To ensure that your program swaps out memory that belongs to it and not to another program, do_spawn must create a separate chain of MCBs instead of using the full DOS chain of MCBs.
Once the MCBs are scanned, the swapping is performed (presuming that swapping was specified in the call to do_spawn). If do_spawn wants to use expanded memory to hold the swapped memory, it must see if an expanded memory manager is installed and ensure that there are enough expanded memory pages to hold the program's memory image. Otherwise, it has to create a temporary file on disk and swap the program image to that file.
EXEC Function
Now that the entire memory image of the program has been swapped to either expanded memory or to disk, you must prepare a data structure called a parameter block for the DOS EXEC function. This parameter block (see Figure 6) is passed as an argument to the EXEC function.
If you specified that the spawned program should use the first environment, segEnvironment will be set to 0. If not, do_spawn puts the segment of the buffer where the user-defined environment string is stored into segEnvironment and adds the size of the environment string to the number of bytes that will stay resident when the second program is spawned.
The do_spawn routine moves the argument string to a storage area in low memory and assigns the address to pszArguments. It also relocates the command string there. Then do_spawn fills the two dummy File Control Block (FCB) structures in the parameter block with zeros, parses the argument string into both FCBs, and assigns their addresses to pFCB1 and pFCB2.
Finally, We Swap
The memory has been swapped and the EXEC parameter block has been prepared. All that is left to do before the EXEC is to reduce the program to the smallest memory image possible. Things that must be preserved in memory include a small data area for do_spawn that contains information about the program that will be swapped, such as the EMS memory handle or the name of the temporary file, the saved code from PSP+92, and so on. The code that performs the actual EXEC must also be saved as well as the code that reloads the memory image. If the latter two items are not preserved, when the spawned program terminates, control will be returned possibly to garbage code. This information is stored starting at PSP+133; it takes up about 1Kb. (You can play with the PSP like this because the first thing that do_spawn did was save the contents of PSP+92 in a storage area.)
All systems are go for DOS to EXEC the new program. Do_spawn merely passes the EXEC function the name of the command to execute and a pointer to the EXEC parameter block, and off we go. We can only hope that some TSR has not fragmented memory too much and that the new program has enough contiguous memory to load.
When the spawned program terminates, it returns control to do_spawn's reloading code, which was previously placed in low memory. The reloading code determines whether the original program was swapped to disk or to expanded memory, and then opens the appropriate device. It reloads the memory image, retrieves the return code from the EXEC function, deletes the swap file, and returns control to the point in do_exec where do_spawn was called.
The possible return codes from do_spawn and do_exec are listed in Figure 7. If your application gets an error code from do_exec, you might want to call do_exec again, this time asking do_exec to load a copy of COMMAND.COM and letting COMMAND.COM try to execute the command. You would do this if you were trying to execute a BAT file, or if your command specified some input or output redirection or piping. COMMAND.COM must be used to interpret and execute the commands in a batch file or to handle redirection, which the DOS EXEC function does not recognize. Sample code to do this is shown in Figure 8.
Enhancements
The code presented here permits applications to load and run specialized programs, making the overall application environment more powerful. These routines could be the kernel of a more sophisticated task-switching system for specialized applications. You'd only have to add data structures that will keep track of program information associated with each swapped program and a user-interface mechanism to allow the user to switch between the various tasks easily.