Using Shell


The first thing you have to decide when running someone else’s program is whether to run synchronously or asynchronously. Generally, programs written for Windows want to run asynchronously. You send your children off to the video parlor and forget about them. Or you keep in touch by phone—COM, DDE, or the Clipboard, in Windows’ terms. Character-based programs (at least those that have survived the Windows onslaught) will usually prefer to operate synchronously. You send your children to the store to get some flour and eggs, but you can’t finish baking until they get back with the goods—usually
a modified file. Of course, nothing prevents character-based programs from work­ing independently or Windows-based programs from modifying data for their parents.



Figure 11-4. The Test Execute program.


The Shell function assumes that you want to run programs asynchronously. Running programs synchronously requires an extra effort. In olden times, hard­core 16-bit Visual Basic programmers used the GetModuleUsage API function to wait for programs to terminate. The 32-bit version works much differently.


The Shell function returns a process ID. Process IDs and handles were introduced in “The process list—Windows NT view” in Chapter 6. To summarize, every process has an ID number that uniquely identifies it, but most operations on processes require handles. You can get a handle from an ID with the Win32 OpenProcess function, which requires you to specify what you would like permission to do with the handle. You can open multiple handles with ­different permissions. The WaitOnProgram procedure hides some of the details. If you want to wait for a program to finish, you can call it like this:

Dim idProg As Long, iExit As Long
idProg = Shell("mktyplib shelllnk.odl", vbHide)
iExit = WaitOnProgram(idProg)
If iExit Then MsgBox "Compile failed"

WaitOnProgram returns the exit code of the shelled program. This assumes that the program chooses to return a meaningful exit code and isn’t written in Visual Basic. Most character-mode programs set exit codes out of habit, but many Windows programs don’t bother. Visual Basic itself returns an exit code when launched from the command line, but it doesn’t provide any means for you to set an exit code in your own programs. (If you’re thinking you could call the ExitProcess API yourself, forget it. I tried that, but the exit code you set doesn’t show up on the outside.)


The WaitOnProgram function takes an optional WaitDead parameter that determines which of two strategies will be used. The default (False in the example above) polls for termination in a DoEvents loop. If the WaitDead argument is True, the calling program is stopped dead in its tracks. Here’s the code:

Function WaitOnProgram(ByVal idProg As Long, _
                       Optional ByVal WaitDead As Boolean) As Long
    Dim cRead As Long, iExit As Long, hProg As Long
    ' Get process handle
    hProg = OpenProcess(PROCESS_ALL_ACCESS, False, idProg)
    If WaitDead Then
        ' Stop dead until process terminates
        Dim iResult As Long
        iResult = WaitForSingleObject(hProg, INFINITE)
        If iResult = WAIT_FAILED Then ErrRaise Err.LastDllError
        ' Get the return value
        GetExitCodeProcess hProg, iExit
    Else
        ' Get the return value
        GetExitCodeProcess hProg, iExit
        ' Wait, but allow painting and other processing
        Do While iExit = STILL_ACTIVE
            DoEvents
            GetExitCodeProcess hProg, iExit
        Loop
    End If
    CloseHandle hProg
    WaitOnProgram = iExit
End Function

If you take the dead wait route, the WaitForSingleObject API function puts the Test Execute program into hibernation. You can click away as much as you want, but nothing fazes the Test Execute program until the child terminates. This is very efficient for other programs in the system because you’re not using any resources while you wait. But it’s a pretty rude way to behave if the calling program has a user interface. The program won’t even be repainted. In fact, if you do this in the IDE, Visual Basic itself will hibernate because it’s the real program waiting for the shelled program to finish. Generally you’ll only want to use the dead wait option from programs that have no user interface.