January 2000

Hook Up Custom Callback Functions to Create Menu Bar Help

by Deborah L. Cooper

Download 200001.zip

Often, menu bar commands can be confusing to a first-time user of your application. While standard Window commands, such as File, Open, and Close are familiar, other commands may seem a little obscure, to say the least. Fortunately, with aide from a simple subclassing technique and a few API functions, you can provide a mini Help feature for menu bar commands. Using these tools, your application can display Help text when a user selects a menu item. For instance, the current window's caption area or a Status Bar control might briefly explain the selected menu item, like the message shown in Figure A. In this article, we'll show you how to create a callback function to build this feature.

Figure A: We'll hook our custom callback procedure in place of the standard window procedure to create custom menu bar help.
[ Figure A ]

Addressing the nature of callbacks and subclasses

Visual Basic's AddressOf operator, introduced in version 5, lets you pass a pointer for a user-defined sub, function, or property to an API procedure. The API function can then pass various data to the callback procedure, which the callback can manipulate as it deems appropriate. To send this pointer to an API function, you use AddressOf procedurename as in OldWndProc = SetWindowLong (gWH, _ GWL_WNDPROC, AddressOf WindowProc)where WindowProc refers to a custom procedure.

Subclassing refers to a technique by which the callback function intercepts messages sent to the operating system. Because our code modifies an existing object, that is, exchanges a new procedure for the window's original procedure, the new window is said to be a subclass of the original object.

Warning: In general, callback functions must contain a specific set of arguments, which are determined by the calling API function. For a brief review of other callback requirements, read "Callback cautions" in this issue.

Hooking a callback function into a window procedure

To create a mini-Help system for our example's menu bar, we'll use the SetWindowLong API function to exchange, or hook, our callback function into the application's window. The SetWindowLong function lets you change the attribute of a specified window. This API function uses the following declaration statement: 
Declare Function SetWindowLong Lib _
"user32" Alias "SetWindowLongA" _
(ByVal hwnd As Long, ByVal nIndex _
As Long, ByVal dwNewLong As Long) _
As Long
To change the standard window procedure, we'll set the nIndex argument to GWL_WNDPROC, and dwNewLong to the address of our new custom function. If the function succeeds, it returns a 32-bit integer that points to the previous window procedure. Table A lists some of the other attributes you can change with this function. Because each window has only one window procedure that handles all messages sent to it, our custom function will become the sole arbiter of those messages.

Table A: Attributes changed with SetWindowLong
Value Description
GWL_EXSTYLE Changes extended window style
GWL_HINSTANCE Changes application instance handle
GWL_ID Changes window identifier
GWL_STYLE Changes window style
GWL_USERDATA Specifies 32-bit value that parent application uses to identify the window
GWL_WNDPROC Changes address for the window procedure

Stringing along the original window procedure

Once firmly hooked in place, our custom function will determine if the application has passed the WM_MENUSELECT message, which occurs when you select a menu item. If it has, then the function will display the appropriate Help text in the window's caption. In all cases, however the procedure will pass messages on to the parent window for normal processing. Doing so ensures that all other window operations remain the same.

To maintain the normal window message chain, we'll call the CallWindowProc API function. This function's sole purpose is to pass message information to a specified window procedure. This function uses the following declaration: 

Declare Function CallWindowProc Lib -
"user32" Alias "CallWindowProcA" _
(ByVal lpPrevWndFunc As Long, ByVal _
hwnd As Long, ByVal Msg As Long, _
ByVal wParam As Long, ByVal lParam _
As Long) As Long
where lpPrevWndFunc contains a pointer to the previous window procedure. In our example, we'll store the pointer to the original window procedure, and then pass this pointer value in this argument. Because we're not going to alter the existing window behavior, our custom function will pass along the other arguments as is.

Unhooking a procedure

Finally, to reassign the original window procedure to the window, we'll once more use the SetWindowLong function along with the GWL_WNDPROC setting. This time, we'll pass the original window procedure's address in the dwNewLong argument. Now that we've covered the basic API functions we'll use in our technique, let's put it all together in a demonstration program.

Menu bar Help with the click of a mouse button

Figure A shows our example. As you can see when you hover the mouse over, or click on, a menu item, the application displays Help text in the form's caption. To begin, create a new Visual Basic project and save it as Minihelp. Also, name the standard form frmCallbacks. Next, from Visual Basic's Projects menu, select Add Module. When VB adds Module1.BAS, add the API declarations in Listing A to it.

Listing A: SetWindowLong and CallWindowProc API declarations
Declare Function SetWindowLong Lib "user32" Alias _
"SetWindowLongA" (ByVal hwnd As Long, ByVal nIndex _
As Long, ByVal dwNewLong As Long) As Long
Declare Function CallWindowProc Lib "user32" Alias _
"CallWindowProcA" (ByVal lpPrevWndFunc As Long, _
ByVal hwnd As Long, ByVal Msg As Long, ByVal wParam _
As Long, ByVal lParam As Long) As Long
Public Const GWL_WNDPROC = -4
Private Const WM_MENUSELECT = & H11F

Now, let's add the procedure that will intercept the windows messages. Listing B shows the WindowProc procedure that we created. Enter it in the module directly below the API declarations. Notice that it also contains the LowOrd function, which we'll explain later.

Listing B: The WindProc and LowOrd procedures
Function WindowProc(ByVal hwnd As Long, ByVal Msg _
As Long, ByVal wParam As Long, ByVal lParam As _
Long) As Long

On Error Resume Next
Dim X As Long
Dim MenuText(0 To 7) As String

X = CallWindowProc(OldWndProc, hwnd, Msg, wParam, lParam)
Select Case Msg
Case WM_MENUSELECT
MenuText(0) = "This helps catch a big one!"
MenuText(1) = "Help Options"
MenuText(2) = "Operatin a callbacks 'n rod"
MenuText(3) = "Findin a deep API pond"
MenuText(4) = "Creatin a good fish story"
MenuText(5) = "Gettin the proper bait"
MenuText(6) = "Unhook it now that you got it."
MenuText(7) = "Exit the program"
frmCallbacks.Caption = MenuText(LowOrd(wParam))
End Select
WindowProc = X
End Function

Function LowOrd(DbleWord As Long) As Integer
If DbleWord And & H8000& Then
LowOrd = & H8000 Or (DbleWord And & H7FFF&)
Else
LowOrd = DbleWord And & HFFFF&
End If
End Function

Create the form

For our first step, let's add the menu items. To do so, select Tools | Menu Editor. Table B shows the buttons we added to the menu bar. An arrow next to the name indicates that we made the command part of a submenu. So, for example, make the Open button a sub-item within the File menu item. Finally, we can add the code to hook the WindowProc procedure. Add the code in Listing C to the form's code window.

Table B: Our example application's menu items
Caption Name
&Fishin mnuFishin
=>&Operatin menuOperatin
=>F&indin mnuFindin
=>&Creatin mnuCreatin
&Help mnuHelp
=>Help Off mnuHelpOff
&Exitin mnuExitin

Listing C: The example form's code
Private Sub Form_Load()
gWH = Me.hwnd
OldWndProc = SetWindowLong(gWH, GWL_WNDPROC, _
AddressOf WindowProc)
End Sub

Private Sub mnuExitin_Click(Index As Integer)
Unload Me
End Sub

Private Sub mnuHelpOff_Click(Index As Integer)
SetWindowLong gWH, GWL_WNDPROC, OldWndProc
End Sub

Private Sub mnuOperatin_Click(Index As Integer)
MsgBox "Hook, line, and sinker."
End Sub

The callback function in action

Press [F5] to execute the demonstration program and select the Fishin menu option. When you do, Windows uses the WindowProc function to process the message. First, it passes along all the message information on to the operating system for normal processing. Next, since the incoming message contains the WM_MENUSELECT value, the function determines which text to place in the caption.

Whenever Windows processes a WM_MENUSELECT message, the wParam argument contains a word value that indicates the selected menu item's (or sub-menu item's) index. The function passes the word value to the LowOrd function, which strips out the low-order value-the real value we need. The function then uses this value to determine which Help text to display.

To close the form, either select the Exitin option or click the form's Close button. When you do, Visual Basic unhooks the WindowProc function from the window and restores the original window procedure.

Conclusion

In this article, we've shown you how to use the AddressOf operator to subclass Windows messages. In addition, we've shown you how to use a custom callback function along with the API. You'll also find this same technique useful for many other Visual Basic applications that you develop.

Copyright © 2000, ZD Inc. All rights reserved. ZD Journals and the ZD Journals logo are trademarks of ZD Inc. Reproduction in whole or in part in any form or medium without express written permission of ZD Inc. is prohibited. All other product names and logos are trademarks or registered trademarks of their respective owners.