One of the critical benefits for customers using MicrosoftÒ Windowstm for Pen Computing is the fact that customers can choose from a broad range of Windows development tools to create exciting new pen applications. Microsoft Visual Basictm with the Professional Toolkit is one example of a development tool customers can employ to create exciting pencentric applications. Visual Basic's extensible design enables customers to leverage custom controls. In the case of the Professional Toolkit two new pen controls, the BEdit and HEdit control, have been added to allow for quick development of pencentric applications.
The first part of this article describes the Windows for Pens custom controls and their properties. The second part describes a simple Visual Basic application and how, utilizing these controls, it becomes a pencentric application.
The BEdit and HEdit custom controls replace the standard input controls available in Visual Basic. The HEdit is a pen-enhanced version of the text box control. It supports most of the Visual Basic text box properties and provides specific handwriting and inking capabilities drawing upon the Windows for Pens API. The BEdit, or boxed edit, control provides the application with comb or box style guides that accept pen input. Each segment or box accepts only a single character of input. The BEdit is a superset of the HEdit and allows for additional manipulation of the writing area. The following illustration shows the typical BEdit and HEdit controls.
The HEdit control provides the primary input area for pen applications. After dragging and setting an HEdit on your form, it will immediately accept handwriting and present the recognized text in the defined box. Since many of the initial pen applications will be form-based, the HEdit has an additional BorderStyle setting—Underline. This setting applies only to single line controls.
Unlike typed text whose size is controlled by the application, handwriting size depends solely on the user's habit. Users tend to write larger due to the resolution and display quality typifying today's pen computers. Pen applications address this problem by allowing inking in areas that extend beyond the displayed input box. The size of this area, which is sometimes called the "gray area," is controlled via the "Inflate" properties. There are four such properties: InflateBottom, InflateLeft, InflateRight, and InflateTop. This larger area becomes active only when the associated control receives focus (which dictates that the user starts inking inside the visible boundary). The gray area is not active in delayed recognition mode and ink can be drawn only inside the visible boundary.
The "boxed edit" (BEdit) control is unique to Windows for Pens. The main reason for adding this control is to address the limitations of today's recognition algorithms. By breaking handwritten text into single characters and forcing them into box or comb guides, we drive the user to write more neatly. The structure of the BEdit provides important baseline and segmentation information to the recognizer. The BEdit applies to fields that accept a predefined text length. For example: phone numbers will typically not exceed 15 characters; by utilizing the BEdit rather than the HEdit we can significantly increase recognition accuracy.
A comprehensive set of properties controls the size of the BEdit cells, their shape (box, comb), and the number of rows and columns in a certain control.
It is no secret that present handwriting recognition engines, even though boasting accuracy levels as high as 95%, are far from perfect. Pencentric applications should be designed to minimize text entry and to utilize contextual information during the recognition process. One popular method to enhance recognition accuracy is by constraining the character set that applies to a certain area. For example, there is no point in allowing alphabetic characters in a social-security number field. By constraining the allowed character set in the social-security number field to numerical only, we avoid misrecognizing a "Z" for a "2" and "O" for a "0". Visual Basic provides the CharSet property, which accepts the following settings: ALC_DEFAULT, ALC_LCALPHA, ALC_UCALPHA, ALC_NUMERIC, ALC_PUNC, ALC_MATH, ALC_MONETARY, ALC_OTHER, and ALC_WHITE. These settings can be ORed together and are controlled either at design time via a custom dialog, or at run time.
Ink is one of the most exciting features and unique to pen applications. The ability to capture, store, manipulate, and display ink breaks ground for new application categories as well as enabling added functionality in existing "text-centric" applications.
Visual Basic supports inking via the DelayedRecog property. When set to True all writing remains as ink on the control until the property is set to False. When it is changed from True to False, the OnTap property is examined. If OnTap is True, recognition of the collected ink will take place when the user taps on the control with the pen.
Aside from allowing quick erasing of the ink via the EraseInk method, Visual Basic does not provide sophisticated ink handling methods. To add advanced functions such as ink selection, resizing, and editing, the developer has to access the underlying ink structures and utilize the Windows for Pens API. Visual Basic provides handles to the ink and the window structure via the hInk and hWnd methods.
The pen custom controls incorporate two new events: the RcResult and the Update. The RcResult occurs whenever the control receives recognition results from the recognizer. The returned result is a pointer to the RcResult structure (Recognition Context), which plays a central role in Windows for Pens applications. The RcResult structure contains a symbol graph that describes the possible results from the ink, a handle to the ink, and the best guess at the symbols. If DelayedRecog is True the RcResult structure does not contain any information about recognized symbols.
The Update event occurs before the control redraws the data. This differs from the Change event, which redraws the data before it occurs. Update can be used to format data so that flashes do not appear.
Pen-Checkbook is a simple Visual Basic application that was initially developed as a Windows application and converted into a fully functional pen application using the pen custom controls. The application was created utilizing a bitmap of a check drawn with Paintbrush. The bitmap was cut into the clipboard and pasted into a picture field in the main program form. The main form includes five Edit Controls: PayToCtrl, AmountCtrl, AmountStrCtrl, RemarksCtrl, and SignatureCtrl. These input fields accept user input and will be at the focus of the conversion process to the pen. Three other fields, DateLabel, CheckNumLabel, and CurrencyLabel, are display only fields. They display current date, a revolving check number, and the relevant currency, respectively. These fields are controlled by the application but do not accept user input. The following figure shows Checkbook prior to its conversion.
The File and Edit menus include standard functions such as Save, Save As, Open, Undo, and Clear. Most of these functions were not implemented because they do not carry direct relevance to the pen. However, the application includes a global structure that holds the last check information; in our conversion process we will discuss how to change this structure so that it can hold ink information and how to store and retrieve this information from a file.
The Options menu includes three functions: Balance, Hide, and Show Last. The Balance function loads a second form that displays the current balance and enables the user to modify it. The current balance is automatically updated when a check is signed by the user. If the user tries to write an amount that exceeds the current balance, the application warns the user and cancels the check. The menu bar can be removed via the Hide function. Tapping on the form brings back the menu bar. This can be useful in pen applications, which can be controlled via gestures rather than menu functions. It supports a less cluttered user interface while retaining the option to display additional controls for training purposes or for advanced functionality. Lastly, the Show Last function displays the last check that was signed by the user.
At this phase our goal is to allow Checkbook to accept pen input utilizing the pen custom controls. At the end of this phase, PayToCtrl, AmountStrCtrl, and RemarksCtrl will accept and recognize your handwriting and Windows for Pens standard gestures. The AmountCtrl field will be replaced by a boxed edit field and will accept and recognize numbers and gestures only. The SignatureCtrl field will accept your signature and retain it as ink. The BalanceCtrl field in the Balance form will be converted to a BEdit on the fly (when the user presses the Change button). It's interesting to note that this whole process will require adding less than ten lines of code.
The first step in converting Checkbook is replacing the standard edit controls with the pen controls. This is a rather tedious task as there is no automated way to do it. You will have to delete the old control, paint the new HEdit or BEdit control, and redefine the various properties such as BorderStyle, BackColor, CtlName, etc. During this process you will notice a number of new properties. The CharSet property enables you to define the characters that will be accepted by the input field. Restricting the character set substantially improves recognition. The following control panel is displayed when defining CharSet.
We will discuss other controls such as the "Inflate" properties and the ink attributes in the next section.
When changing SignatureCtrl we modify the DelayedRecog property and set it to True. This will create an ink field. The ink can be later recognized if we tap on it and the OnTap property is set to True. We will later show how this signature can be "locked" to a certain check so that users cannot change the amount on the check once it has been signed.
Changing the AmountCtrl and BalanceCtrl (second form) requires more changes. The single edit control is replaced by two Boxed Edit (BEdit) controls, for Dollars and Cents. You will need to adjust the "Cell" and "Comb" properties to achieve the best space utilization without compromising usability of the new field. The names of the new controls will now be DollarCtrl, CentCtrl, DollarBalanceCtrl, CentBalanceCtrl. Lastly, utilizing the CharSet property, we will constrain the accepted characters to gesture and numerical values.
At the end of this phase, Checkbook is easily controlled with the pen. You may write and sign a check exactly the same way you are doing it with your paper checkbook. You may also use Windows for Pens gestures to edit and manipulate the various fields. The following diagram shows Pen-Checkbook at the completion of the first phase.
After using the application for a while you may discover that some layout changes may make your life much easier. As mentioned before, handwriting is typically larger than printed text. The controls that were built for keyboard input may be too small and result in loss of ink when the pen exits the border of the control. Using the Inflate properties of Hedit and Bedits, you can adjust the size and location of the fields to fit the new paradigm. The Inflate properties do not change the visible borders of the controls; they affect the "gray" area that envelops them, thus enabling the user to write beyond the visible borders without losing ink. Other changes may be required in applications that are more cluttered than Checkbook. Pen applications should be easy to use and should entertain intuitive look-and-feel. The primary users of pen computers will be people on the move who are not in the position to manipulate multiple windows, complex tool bars, and chained menus.
Visual Basic provides several hooks that enable developers to tap into the full functionality supported by the Windows for Pens API. These include the RcResult and Change events and the hInk and hWnd handles.
The RcResult event occurs when a BEdit or HEdit control attempts to recognize the ink. The event can be used as a trigger to create locking form fields. Once the user stops inking in the signature field, all the input fields in the form will be locked and no changes can be made. This can be very practical in form-based applications where a signature validates the data in the form. In our case, once the user has signed the check the only way to "unlock" the input fields will be by pressing the "Pay" button or selecting "Clear" in the Edit menu. In both cases the input fields will be initialized. The following function was added to Pen-Checkbook to implement "locking" fields:
Sub SignatureCtrl_RcResult (RcResult As Long)
PayToCtrl.Enabled = False
DollarCtrl.Enabled = False
CentCtrl.Enabled = False
AmountStrCtrl.Enabled = False
RemarksCtrl.Enabled = False
SignatureCtrl.Enabled = False
PayButton.Enabled = True
End Sub
Since the RcResult event occurs before the recognition results affect the display, it can be used to trap custom gestures. The following sample declarations were taken from the global declarations file of Pen Checkbook. This file is basically the Visual Basic version of PENWIN.H. We will later refer to these declarations when processing the RcResult event.
Type RcResult ' RcResult structure
SYGraph As syg ' Symbol Graph element
wREsultsType As Integer ' Status of RcResult handling
cSyv As Integer
lpsyv As Long
hSyv As Integer
nBaseLine As Integer
nMidLine As Integer
hPenData As Integer
rectboundink As RECTSHORT
pntEnd As PointShort
lprc As Long
End Type
Type syg ' Symbol Graph structure
rgpntHotSpotsArray As PointArray ' "hot spot" array
cHotSpot As Integer
nFirstBox As Integer
lRecogVal As Long
lpSye As Long ' Pointer to first Symbol Element
cSye As Integer ' Number of Symbol elements
lpSyc As Long
csyc As Integer
End Type
Type SYE ' Symbol element structure
Syv As Long ' Symbol Value
lRecogVal As Long
cl As Integer
iSyc As Integer
End Type
Declare Sub VBTypeToCPointer Lib "pencntrl.vbx" (lpSrc As Any, ByVal
lpDest As Long, ByVal cb As Integer)
Declare Sub CPointerToVBType Lib "pencntrl.vbx" (ByVal lpSrc As
Long, lpDest As Any, ByVal cb As Integer)
'these two functions copy data to/from Visual Basic strings from/to
'memory pointed by long pointers
The following subroutine duplicates the functionality of the "Hide" function in the Options menu. If the user draws the "Circle H" gesture in the edit control, the menu bar will be hidden. The function Process_cGestures is defined in the global module and called by each edit control when the RcResult event occurs.
Sub Process_cGestures (ByVal RcResult As Long)
Dim VBrc As RcResult
Dim SyeTable() As SYE ' Array of Symbol elements
CPointerToVBType ByVal RcResult, VBrc, 80 ' Copy RcResult structure
' to VB var
NumOfSymbols% = VBrc.SYGraph.cSye ' Get number of Symbol elements
lpSye& = VBrc.SYGraph.lpSye ' Pointer to first Symbol Element
ReDim SyeTable(NumOfSymbols%) ' Redefine Symbol array
CPointerToVBType ByVal lpSye&, SyeTable(0), NumOfSymbols% * 12
' Copy Symbol elements
If (NumOfSymbols% = 1) Then ' Process only if single gesture
Syv& = SyeTable(0).Syv
SyvType& = Syv& \ &H10000 ' Get Symbol type
If (SyvType& = SYVHI_GESTURE) Then ' Is it a gesture?
If (Syv& = (SYV_CIRCLELOA + Asc("h") - Asc("a"))) Then
' Is it a "circle h" gesture
HideCmd_Click ' Hide menu-bar
VBrc.wREsultsType = VBrc.wREsultsType Or
RCRT_ALREADYPROCESSED
End If ' Mark to prevent further processing
End If
VBTypeToCPointer VBrc, ByVal RcResult, 80 ' Update RcResult
End If
End Sub
As previously mentioned, Checkbook enables you to store and display the last check that was paid. When SignatureCtrl becomes an ink field, the global structure "CheckRec" needs to be modified to hold a pointer to the ink structure rather than the signature text. The following declarations and code samples show how using the hInk, hWnd, and the Windows for Pens API, the ink is stored in memory and retrieved later on.
Declare Function DuplicatePenData Lib "penwin" (ByVal hPenData As
Integer, ByVal gMemFlags As Integer) As Integer
' DuplicatePenData takes a handle to ink (hpenData or
' hInk) and returns a new hInk/hPenData that has a copy
' of the ink. NOTE: Whenever this function is used, a
' GlobalFree() call must be issued to release the
' memory.
Declare Function SendMessage Lib "User" (ByVal hWnd As Integer,
ByVal wMsg As Integer, ByVal wParm As Integer, ByVal lParam
As Any) As Long
' General Message Sending function of Windows. Below are some
' messages that can be sent to HEdits or BEdits
Sub PayButton_Click ()
Amount! = Val(DollarCtrl.text) + Val(CentCtrl.text) / 100
balance = balance - Amount!
If balance < 0 Then
Beep ' Warn user if insufficient funds
MsgBox "Insufficient Funds - Check Cancelled", 0, "ERROR!"
balance = balance + Amount!
Else
check.NumField = CheckNumLabel.Caption ' Store last check
:
check.RemarksField = RemarksCtrl.text
check.SignatureField = DuplicatePenData(SignatureCtrl.hInk,
ByVal 0)
End If ' Store ink field
PayToCtrl.text = "" ' Initialize input fields
:
SignatureCtrl.EraseInk = True ' Erase signature
PayToCtrl.Enabled = True ' Unlock input fields
:
PayButton.Enabled = False
:
End Sub
Sub ShowLastCmd_Click () ' "Show Last" command function
Dim lRet As Long ' return val from Windows API
Dim lParam As Long
lParam = check.SignatureField ' Force integer into long
CheckNumLabel.Caption = check.NumField ' restore last check
:
lRet = SendMessage(SignatureCtrl.hWnd, WM_HEDITCTL,
HE_SETINKMODE, lParam)
' retrieve ink from memory
End Sub