Passing Function Pointers to DLL Procedures and Type Libraries

See Also

If you're familiar with the C programming language, function pointers may be familiar to you. If you're not, the concept merits some explanation. A function pointer is a convention that enables you to pass the address of a user-defined function as an argument to another function you've declared for use within your application. By using function pointers, you can now call functions like EnumWindows to list the open windows on the system, or EnumFontFamilies to catalog all of the current fonts. You can also use them to gain access to many other functions from the Win32 API that have not previously been supported in Visual Basic.

For Visual Basic, several limitations apply to the use of function pointers. For details, see "Limitations and Risks with Function Pointers" later in this topic.

Learning About Function Pointers

The use of function pointers is best illustrated with an example. To start, look at the EnumWindows function from the Win32 API:

Declare Function EnumWindows lib "user32" _
(ByVal lpEnumFunc as Long, _
ByVal lParam as Long ) As Long

EnumWindows is an enumeration function, which means that it can list the handle of every open window on your system. EnumWindows works by repeatedly calling the function you pass to its first argument (lpEnumFunc). Each time EnumWindows calls the function, EnumWindows passes it the handle of an open window.

When you call EnumWindows from your code, you pass a user-defined function to this first argument to handle the stream of values. For example, you might write a function to add the values to a list box, convert the hWnd values to window names, or take whatever action you choose.

To specify that you're passing a user-defined function as an argument, you precede the name of the function with the AddressOf keyword. Any suitable value can be passed to the second argument. For example, to pass the function MyProc as an argument, you might call the EnumWindows procedure as follows:

x = EnumWindows(AddressOf MyProc, 5)

The user-defined function you specify when you call the procedure is referred to as the callback function. Callback functions (or "callbacks," as they are commonly called) can perform any action you specify with the data supplied by the procedure.

A callback function must have a specific set of arguments, as determined by the API from which the callback is referenced. Refer to your API documentation for information on the necessary arguments and how to call them.

Using the AddressOf Keyword

Any code you write to call a function pointer from Visual Basic must be placed in a standard .BAS module — you can't put the code in a class module or attach it to a form. When you call a declared function using the AddressOf keyword, you should be aware of the following conditions:

• AddressOf can only be used immediately preceding an argument in an argument list; that argument can be the name of a user-defined sub, function, or property.
  
  
• The sub, function, or property you call with AddressOf must be in the same project as the related declarations and procedures.
  
  
• You can only use AddressOf with user-defined subs, functions, or properties — you cannot use it with external functions declared with the Declare statement, or with functions referenced from type libraries.
  
  
• You can pass a function pointer to an argument that is typed As Any or As Long in a declared Sub, Function, or user-defined type definition.

Note   You can create your own call-back function prototypes in DLLs compiled with Visual C++ (or similar tools). To work with AddressOf, your prototype must use the __stdcall calling convention. The default calling convention (_cdecl) will not work with AddressOf.

Storing a Function Pointer in a Variable

At times, you may need to store a function pointer in an intermediate variable before passing it to the DLL. This is useful if you want to pass function pointers from one Visual Basic function to another. It's required if you are calling a function like RegisterClass, where you need to pass the pointer through an argument to a structure (WndClass), which contains a function pointer as one of its elements.

To assign a function pointer to an element in a structure, you write a wrapper function. For example, the following code creates the wrapper function FnPtrToLong, which can be used to put a function pointer in any structure:

Function FnPtrToLong (ByVal lngFnPtr As Long) As Long
   FnPtrToLong = lngFnPtr
End Function

To use the function, you first declare the type, then call FnPtrToLong. You pass AddressOf plus your callback function name for the second argument.

Dim mt as MyType
mt.MyPtr = FnPtrToLong(AddressOf MyCallBackFunction)

Subclassing

Subclassing is a technique that enables you to intercept Windows messages being sent to a form or control. By intercepting these messages, you can then write your own code to change or extend the behavior of the object. Subclassing can be complex, and a thorough discussion of it is beyond the scope of this book. The following example offers a brief illustration of the technique.

Important   When Visual Basic is in break mode, you can't call vtable methods or AddressOf functions. As a safety mechanism, Visual Basic simply returns 0 to the caller of an AddressOf function without calling the function. In the case of subclassing, this means that 0 is returned to Windows from the WindowProc. Windows requires nonzero return values from many of its messages, so the constant 0 return may create a deadlock situation between Windows and the Visual Basic, forcing you to end the process.

This application consists of a simple form with two command buttons. The code is designed to intercept Windows messages being sent to the form and to print the values of those messages in the Immediate window.

The first part of the code consists of declarations for the API functions, constant values, and variables:

Declare Function CallWindowProc Lib "user32" Alias _
"CallWindowProcA" (ByVal lpPrevWndFunc As Long, _
   ByVal hwnd As Long, ByVal Msg As Long, _
   ByVal wParam As Long, ByVal lParam As Long) As Long

Declare Function SetWindowLong Lib "user32" Alias _
"SetWindowLongA" (ByVal hwnd As Long, _
ByVal nIndex As Long, ByVal dwNewLong As Long) As Long

Public Const GWL_WNDPROC = -4
Global lpPrevWndProc As Long
Global gHW As Long

Next, two subroutines enable the code to hook into the stream of messages. The first procedure (Hook) calls the SetWindowLong function with the GWL_WNDPROC index to create a subclass of the window class that was used to create the window. It then uses the AddressOf keyword with a callback function (WindowProc) to intercept the messages and print their values in the Immediate window. The second procedure (Unhook) turns off subclassing by replacing the callback with the original Windows procedure.

Public Sub Hook()
   lpPrevWndProc = SetWindowLong(gHW, GWL_WNDPROC, _
   AddressOf WindowProc)
End Sub

Public Sub Unhook()
   Dim temp As Long
   temp = SetWindowLong(gHW, GWL_WNDPROC, _
   lpPrevWndProc)
End Sub

Function WindowProc(ByVal hw As Long, ByVal uMsg As _
Long, ByVal wParam As Long, ByVal lParam As Long) As _
Long
   Debug.Print "Message: "; hw, uMsg, wParam, lParam
   WindowProc = CallWindowProc(lpPrevWndProc, hw, _
   uMsg, wParam, lParam)
End Function

Finally, the code for the form sets the initial hWnd value, and the code for the buttons simply calls the two subroutines:

Private Sub Form_Load()
   gHW = Me.hwnd
End Sub

Private Sub Command1_Click()
   Hook
End Sub

Private Sub Command2_Click()
   Unhook
End Sub

Limitations and Risks with Function Pointers

Working with function pointers can be unforgiving. You lose the stability of Visual Basic's development environment any time you call a DLL, but when working with function pointers, it can be especially easy to cause the application to fail and to lose your work. Save often and back up your work as necessary. Following are notes on some areas that require special attention when working with function pointers:

• Debugging. If your application fires a callback function while in break mode, the code will be executed, but any breaks or steps will be ignored. If the callback function generates an exception, you can catch it and return the current value. Resets are prohibited in break mode when a callback function is on the stack.
  
  
• Thunks. Thunking is the way that Windows enables relocatable code. If you delete a callback function in break mode, its thunk is modified to return 0. This value will be correct most of the time — but not all of the time. If you delete a callback function in break mode and then type it again, it's possible that some callees will not know about the new address. Thunks aren't used in the .exe — the pointer is passed directly to the entry point.
  
  
• Passing a function with the wrong signature. If you pass a callback function that takes a different number of arguments than the caller expects, or mistakenly calls an argument with ByRef or ByVal, your application may fail. Be careful to pass a function with the correct signature.
  
  
• Passing a function to a Windows procedure that no longer exists. When subclassing a window, you pass a function pointer to Windows as the Windows procedure (WindowProc). When running your application in the IDE, however, it's possible that the WindowProc will be called after the underlying function has already been destroyed. This will likely cause a general protection fault and may bring down the Visual Basic development environment.
  
  
• "Basic to Basic" function pointers are not supported. Pointers to Visual Basic functions cannot be passed within Visual Basic itself. Currently, only pointers from Visual Basic to a DLL function are supported.
  
  
• Containing errors within a callback procedure. It is important that any errors within a callback procedure not be propagated back to the external procedure that initially called it. You can accomplish this by place the On Error Resume Next statement at the beginning of the callback procedure.