Created: March 20, 1992
Abstract
Personal computers that use the pen as an input device (pen PCs) offer exciting new opportunities for both users and application developers. In this paper, we’ll look at the hardware advances that make pen computers possible and the impact they will have on our applications. We’ll study the MicrosoftÒ WindowsÔ for Pen Computing environment and see how comfortably the pen fits into the Windows architecture. We’ll explain how you can modify your application to use the new features, and we’ll examine some of the exciting ways the pen lets you improve your application’s user interface. When you’ve finished this paper, you’ll be ready to use the Windows for Pen Computing Software Development Kit (SDK) in conjunction with the Windows version 3.1 SDK to design your own applications to take advantage of the pen.
As personal computers become smaller and more mobile, they will be packaged into laptop, notebook, and even palmtop sizes. However, graphical user interface (GUI) applications also require a keyboard and a mouse. Keyboards simply don’t scale down well, and even trackballs scale poorly below laptop sizes. What we need is a device that fits in the user’s hand and provides all the input capabilities of a keyboard and a mouse. One such device is the pen.
We’re all familiar with pen-shaped devices and use them every day. From our first crayons, through the #2 lead pencil we used in school, to the finest Cross pen, we use this comfortably shaped tool to draw, to write, and to point. Some computers provide their users with pens (for example, CAD software is often used with light pens), so why haven’t pens been available for the rest of us? How will the pen computers we’ll all be using in five years change the way we work with computers? And how will pen PCs change the way programmers design the user interface of applications?
Personal computers won’t all come with pens for some time. Meanwhile, you can write applications that work perfectly well with the keyboard and the mouse and that take advantage of the pen when it’s present. In fact, thinking about how to take advantage of the pen will give you ideas for improving the user interface of your applications in all environments. The applications you’ve already written for the MicrosoftÒ WindowsÔ graphical environment provide the best foundation for moving to the pen platform.
Putting together a pen computer requires several pieces of hardware. First, we need a compact central processing unit (CPU) and support logic, along with a battery. Notebook computers are already a reality and will continue to get smaller. We also need a stylus input device and a stylus-sensitive display; these are just now becoming feasible. Together, the small personal computer and the pen will be the foundation for applications that make PCs even more personal and less computer.
Let’s examine the underlying PC hardware that will make this transition happen. In the decade since the PC was introduced, nearly every parameter in the PC has changed by an order of magnitude. By 1994, under a dozen integrated circuits (ICs) will be required to build a PC (compared to well over 100 ICs for the first PCs). Clock rate is up to 33 megahertz and power consumption has plummeted. Data and address busses are wider, and memory size has exploded. As a result of miniaturization, we can expect to run the same software on desktop systems and on any of today’s notebook or palmtop PCs. With all that power in a “go-anywhere” PC and with your operating environment (and even your applications) in read-only memory (ROM) on that PC, we can expect the personal in personal computers to finally win.
By itself, miniaturization of PCs doesn’t dictate moving to pen input devices. As PCs shrink below clamshell notebook size, the palm-sized mouse and six-palms-sized keyboard become unwieldy, and neither device scales down well. The stylus with stylus-sensitive surface and integral display becomes the best possible alternative. The shape and feel of a pen is familiar to us all, and the immediacy of writing directly onto the computer’s output device provides a compelling feeling of being in control.
The surface on which the pen operates is as important as the pen itself. To mimic paper, the surface has to provide output so that users can see the ink and the program objects that they work with. A thin, flat panel that is fast, legible, and sturdy is required. The display must be thin and flat to minimize parallax distortion. (The illusion of writing directly on the surface is weakened if the screen is as much as 3 millimeters below the tip of the pen and would be destroyed if that distance were much over 6 millimeters.) The display must be fast and legible to preserve the illusion that the pen is inking the surface. And it must be sturdy because users will pick up and take these devices everywhere, and the occasional “Oops!” is inevitable.
Today, several vendors offer integrated tablets that meet these requirements. The decreased cost of components makes it feasible to integrate tablets into a PC with only a $150–$300 premium over keyboard-and-mouse systems. As volume ramps up, we can expect to see the quality of the displays improve while the cost differential disappears. Several types of pen PCs will appear: clipboard-style devices with no keyboard, clamshell notebooks with a keyboard inside and a writing surface outside, and add-on devices for desktop PCs. Each of these “form factors” will serve different markets, and all will be doing pen computing. In fact, within the next 12–18 months, the dedicated palmtop appliance (for example, CasioÒ Boss, SharpÒ Wizard) will step up and become a full-fledged PC. So even palmtops will be doing pen computing soon.
Now let’s focus on what users will get for their dollars.
The pen is a superb mouse; it selects data and manipulates objects (such as the parts of a car in a claims estimate form) with complete accuracy. The pen is a terrific drawing tool; it is suitable for making sketches that are more permanent than napkin or back-of-the-envelope drawings and more detailed than any freehand drawing accomplished with a mouse. The pen is superb for writing—and a lot more people can write than can type. But the pen is also more than a mouse because it can directly specify not just what to change (the selection) but also how to change it (the operation). A simple one-stroke twist, loop, or curl gesture with the pen can be interpreted as a command and applied to the data directly under the gesture.
The pen is a direct input device. Instead of guiding a cursor with the mouse or with arrow keys, the user simply places the pen at the desired location. Instead of pressing a button on the mouse (or a function key on the keyboard) and using the mouse or arrow keys to extend the selection, the user simply places the pen on the writing surface, pulls it across the surface to the desired location, and picks up the pen.
The pen’s location is its own echo—it needs no artificial echo on the screen. In fact, although many of us who are familiar with the mouse will use a visible cursor for old time’s sake, new users may never bother with a cursor. Whether we see the cursor or not, we all benefit from the way the pen can point at objects and select them directly from the screen.
Once the pen is on the writing surface, the improvements over existing input devices become even more apparent. When a user draws with the mouse, Windows records no more than 20 locations per second to avoid data overload. Resolution approaches 400 points per inch. Pen input, on the other hand, is faithfully reproduced—over 200 locations per second at a resolution of 1000 points per inch. In later sections, we will examine the pen device driver, learn how pen input is handled, and understand how Windows can handle the vast amount of data from the pen without bogging down. For now, consider the benefits of a drawing instrument that isn’t constrained to 20 spots per second at a resolution of 1/400th of an inch.
Artists, even those of us who just doodle, find the familiar feel of the pen a tremendous improvement over the clumsy mouse or trackball for drawing. Drawings represent only one type of data for which the pen is a superior input device.
The pen is a natural replacement for keyboard data entry. Although some people can type faster than anyone can write, most of us feel more comfortable with a pen than with a typewriter. Whether the computer “ink” is retained as original data to be read later or translated into computer character codes representing the text, pen and ink are light-years ahead of the typewriter in flexibility.
When a pen is present in the system, users can choose between retaining their handwriting as ink or turning their handwriting into “typed” text. Your application can accommodate either choice easily by using Windows for Pen Computing.
The crowning achievement of the pen as an input device is its ability to directly specify both a selection (the “noun” for an operation) and an action (the “verb” for that operation). The use of position collapses the tedious “select object and then select operation from a menu” interface enforced by mouse and keyboard. With the pen, the user makes a single gesture at the object. Your application then determines which data to change and what operation to perform.
The use of gestures is the single most compelling aspect of the pen as an input device. Pointing, drawing, and writing are important, but applications and PCs today already support those functions. Gesturing is simply a new user interface rather than new functionality; it’s a natural use of the pen’s detail and accuracy. In fact, using a mouse to gesture would be difficult, and using a keyboard would be downright impossible. When Windows for Pen Computing sends a location and a gesture index to your application, you’ll be able to perform the desired action immediately on the indicated data. Gestures are so powerful that you’ll probably want to add some application-specific gestures of your own.
Windows for Pen Computing provides about a dozen predefined gestures, including gestures for selection (for example, to extend selection), gestures for operations (such as copy, paste, and undo), and gestures for noncharacter keys (BACKSPACE, SPACEBAR, TAB, and ENTER). If your application uses the standard Windows mouse and keyboard user interface for these items, it already handles the standard gestures.
For extensibility, users can define their own gestures by circling letters. Some circle-letter gestures are universal (for example, circle-C means copy), some are specific to a class of applications (for example, circle-B might mean bold to word processing applications), and others are application specific or user specific. If you want to provide more graphical gestures, you can implement an application-specific recognizer.
If you want your Windows-based application to run on the new pen PCs successfully, you’ll have to do more than simply take advantage of the pen when it’s present. Most of the impact on your applications is related to the small size and portable nature of pen PCs. In the following sections, we will review several design considerations.
Most computer screens adopt a horizontal (landscape) orientation. Clipboard-style computers can also be used in a vertical (portrait) orientation to imitate the way we traditionally hold clipboards. The impact on applications is that the clipboard’s display, which is 640x480 when held horizontally, suddenly becomes 480x640 when held vertically. Windows for Pen Computing supports portrait and landscape orientation, so you won’t be able to rely on 640 horizontal pixels. You should provide layouts that take both orientations into account. This is especially true for control bars (such as toolboxes and palettes) and dialog boxes.
A dialog box that relies on a display that can show over 40 standard characters horizontally may lie partially off-screen on a smaller display. The typical display has square pixels, so the system font size won’t change in portrait mode.
For at least the next year or so, clipboard PCs will be monochrome; almost all will provide 16 shades of gray. For this reason, you should ensure that your application looks good on monochrome displays and offer users a choice of color or gray-scale as a configuration option.
Clipboard pen PCs will often be used without a keyboard. You should be sure that your application can be driven entirely from its menus, dialog boxes, and control bars. The most important actions should be available directly through on-screen buttons or easily remembered gestures.
Use command completion techniques as much as possible: Offer the user choices based on available input, and update the choices as you get more input. As a result, your application will work better with less text input and will become easier to use even with the mouse and the keyboard.
Portable PCs are battery operated and present potential problems in power consumption. Windows version 3.1 and MS-DOSÒ version 5.0 both allow applications to participate in power management.
The heart of every Windows-based application is the message loop. The most important thing an application can do for power management is to wait in the loop whenever it waits for input. Many Windows-based applications are too clever by half—instead of entering a proper “wait” loop, they enter a “busy-wait” loop that prevents power management software and firmware from working.
The portability of the pen PC and the power of the pen to enter and manipulate data directly leads to new market opportunities.
When the PC achieves the simplicity of a three-ring binder, it will be ready to go with its owner to meet a customer or to take a walk on the factory floor. For people who work out of their vehicles, the size will be right. The worst thing about three-ring binders is that whatever you write or draw is stuck to the page you wrote it on. If you want to leave a price quote with a customer, you have to generate two copies. If information changes while you fill out the form, you have to mark the form and leave it with the customer or recopy the information to a clean sheet.
With a pen PC, the sheet you give your customer is clean and polished regardless of how many “what-if” scenarios you’ve tried out. You can even ask the customer to sign the computer-based form. And when you add a cellular telephone and modem to the configuration (which some clipboard PC manufacturers will provide), you’ll be able to negotiate using up-to-the-minute prices and availability.
Entire categories of software (such as daily planners, calendars, and contract-writing tools) that have been stalled in the marketplace on general-purpose computers will start to take off on these smaller pen computers.
Because pen computers are frequently used by people on the go, pen applications should minimize setup and startup time. Some of the features you can provide (such as on-screen buttons for frequently accessed commands) are almost certainly part of your designs today. You will need to think about how your application can learn from its users and become even more comfortable to use over time. Today’s applications require that the user do the learning. Tomorrow’s applications will need to learn from the user.
The simplest way an application can conform to the way it’s used is to remember user preferences for window location, layout, colors, and the like. However, this is merely cosmetic customizing. Other customizations can enhance both user productivity and the utility of your applications even more. For example, users should be able to access frequently used commands directly. A programmable toolbox offers this level of flexibility and should be your second line of customization.
You will also want to support users as they fill in fields and access data. As the first level of support, you can retain the most recent entry in a field as the default and remember defaults between invocations of your application. This will certainly help, but you can do more: If the user establishes a pattern of use (for example, by using a text editor to examine first .C and then .H and then .RC files), notice that fact and take advantage of it.
Usage patterns can also be captured by macro facilities. Macros are the power user’s defense against having to repeat long interactive sequences within an application. As typically provided, however, macro features are simply beyond the skills of the average user. Smart applications track user activity in the background, note repetitious sequences, and offer to help the user create a macro or tutor the user on a more direct command sequence. This doesn’t require artificial intelligence—you can simply research how users actually use your application, and build in responses to the typical novice’s usage patterns.
These improvements will limit the amount of user interface poking required to get a job done and will benefit all users, regardless of whether they’re using a pen or a mouse and a keyboard. Users working without a keyboard will appreciate these shortcuts even more than keyboard users.
Each time you design a form or present a user with choices, you probably have a clear idea of what sort of input you expect. When you enhance your applications to work with handwriting, you’ll want to take advantage of the expected type of input to tune the handwriting recognizer. You can embody these expectations in commands to the character recognizer.
You can filter input in even more powerful ways. The character recognizer returns a list of text strings, with the highest probability string presented first. Windows for Pen Computing provides a second-level mechanism, called the dictionary, for fine-tuning the output from the character recognizer. For example, a field intended to hold only months of the year might have an associated dictionary that performs word completion; on recognizing the first three letters, this dictionary would return the complete name of the month—capitalized to boot.
You can improve both handwriting recognition and voice recognition by providing contextual information to the recognizer. In fact, human speech has greater variations than handwriting, so contextual information is more helpful to a speech recognizer than it is to a handwriting recognizer. Every bit of work you put into dictionaries to improve handwriting recognition will apply to handling voice input in your Windows-based applications when Windows is extended with voice input.
Central to the Windows for Pen Computing architecture is the Recognition Context (RC) Manager. Other components include the device drivers for the pen and the display, recognizers that convert pen input into character sequences, and dictionaries that select the most likely character strings based on the recognizer’s output. These components blend pens into Windows. Before we examine each component, let’s survey the overall architecture.
Adding the pen to Windows involves enhancing both input and output. On the input side, the pen device driver provides pen data to the system. On the output side, pen data is echoed on the display as “ink” while drawing takes place. This ink layer is handled by the display driver through a new device-driver entry point and involves no changes to the graphics device interface (GDI). The presence of ink has no impact on Windows-based applications. At this lowest level in the system, the connection between pen input and ink on the display is immediate. This is identical to the mouse cursor location being updated at interrupt time.
Above the device driver level, pen input is interpreted and given meaning. The communication between the pen driver and the display driver guarantees that users see the pen as a writing tool that can leave ink freely on the display. The levels above supply the intelligence to interpret the ink as text, shapes, and gestures.
Windows-based applications interact with the pen through the RC Manager, which handles the pen driver, the recognizers, and the dictionaries on behalf of all applications. The RC Manager sends pen input to the appropriate window and coordinates pen input processing for that window.
Figure 1 illustrates the components of Windows for Pen Computing. The RC Manager is central to Windows for Pen Computing and lies in the system components layer. That layer also includes MS-DOS functionality, input handling (labeled Win to designate the functionality in the Windows KERNEL.EXE and USER.EXE libraries), and graphics output (handled by GDI.EXE).
Figure 1. Windows for Pen Computing Architecture
The RC Manager is the heart of the extensions provided in Windows for Pen Computing. It is the architectural component that orchestrates pen input handling. Code for the RC Manager is contained in PENWIN.DLL, so the terms RC Manager and pen DLL are synonymous.
Whenever an application wants to work with the Windows for Pen Computing extensions, it must obtain a recognition context (RC) data structure, fill in the fields of the RC appropriately, and hand the RC to the RC Manager. The fields in the RC data structure specify exactly how the pen data is to be handled. In this sense, the RC is patterned after the device context (DC) used with GDI.
In general, one RC is used with each inking “transaction” between the user and your application. Often, applications will find it convenient to allocate an RC permanently and control the recognition process by fine-tuning the RC parameters as needed (just like a private DC).
Some RC fields identify the recognizer to be applied to the ink and the dictionaries to be used to select the “best-guess” string. Other fields specify the conditions that end the inking transaction (for example, the pen going up, the pen going outside a specified rectangle, or time-out). The RC also provides parameters used by the recognizer and by the dictionaries.
At the end of each inking transaction, the RC Manager returns the results of ink processing performed by the specified recognizer and dictionaries to the window that initiated the transaction. If the window does not fill in an RC and initiate an inking transaction with the RC Manager, the pen functions exactly like a mouse.
Pens can be tethered or untethered to serve slightly different markets. Most pen computer manufacturers will choose the style that best serves their target markets. Although the two pen styles are supported by different hardware, they both act the same under Windows for Pen Computing. The pen works with computer electronics to encode (x,y) locations and other information. The pen must sense the difference between touching and not touching the display surface. This is typically accomplished with a tip-switch that acts just like the left mouse button. Many pens can report pressure levels at the surface, but this additional information is not required. Some pen systems can also detect the pen’s location even when the pen is not quite touching the surface (this is called proximity sensing).
In addition to the (x,y) location and “pen down” information, many pens provide a button on the barrel. A manufacturer may also choose to implement a pen that records the slant of the pen or the rotation of the pen barrel. Windows for Pen Computing does not require this information but provides a standard method for applications to determine whether the information is present and, if so, to access the information.
All pen devices must be capable of recording at least 120 events each second and must have a resolution of at least 200 dots per inch. Many pens will be capable of recording data at much higher resolution or rates. These requirements guarantee sufficient data for character recognition by the vector-based character recognizers used with Windows for Pen Computing.
The pen device driver reports all pen data to the system at 0.001-inch resolution. This standard resolution allows improvements in pen hardware. It also corresponds to the MM_HIENGLISH mapping mode in the Windows GDI and is therefore convenient for applications.
The pen SDK includes sample device drivers for WACOM and Scripsit tablets. These drivers can be modified easily to support any pen hardware that uses an interrupt to return data through I/O addresses.
The display driver under Windows for Pen Computing is an enhancement of the standard display driver for Windows version 3.1. The extensions support echoing ink on the display as it is drawn by the user and moving the origin of the screen to support landscape and portrait orientations. Ink support should be provided in such a way that removing the ink does not generate redraw messages to applications. The display driver must also provide a new pointer in the shape of a pen.
Inking is easy to implement; it is supported by the InkReady function in the display driver. Supporting portrait and landscape modes requires reading SYSTEM.INI for the desired mode and rewriting low-level routines to handle the reordered access to the frame buffer. The pen SDK includes a sample driver for VGA devices.
A recognizer is a dynamic link library (DLL) that collects ink from the display driver and interprets that ink as characters in one or more alphabets. When inking begins, the RC Manager calls the recognizer specified in the RC for the transaction. Applications do not directly interact with recognizers but may use functions provided by the RC Manager to load and access recognizers. This allows applications to access built-in recognizers, third-party recognizers, and custom recognizers with a single application programming interface (API).
Recognizers must handle an inking transaction according to the fields set in the RC for that transaction and end the transaction when the conditions specified in RC fields are met. The recognizer examines the ink for characters in the alphabets and character sets named in the RC. From the character sets named, it returns characters that are recognized at the specified level of confidence.
During recognition, the recognizer collects and analyzes the ink. The RC Manager returns the original ink to the calling application along with the results of analysis in the form of a symbol graph. A symbol graph is a sequence of 32-bit values, each of which is associated with a recognized character or a metacharacter. Metacharacters are used to group alternate possibilities if more than one character meets the confidence level set in the RC for the transaction. For example, the string “clea{r|n}” is a symbol graph representing two possible recognition results, “clear” and “clean”. In this example, ‘c’, ‘l’, ‘e’, ‘a’, ‘r’, and ‘n’ represent recognized characters, each of which is associated with a confidence level. The ‘{’, ‘|’, and ‘}’ symbols are metacharacters used to define the graph of alternatives.
All recognized characters in a symbol graph are associated with both the confidence the recognizer assigns to that character and the specific strokes in the ink that make up the character. If the recognizer cannot assign a character to some part of the ink with the required confidence, it must specify “I don’t know” at that position in the symbol graph.
When a recognizer completes its analysis, it returns results to the RC Manager, which arranges to return the results to the calling application. The recognizer returns ink, a symbol graph, symbol elements (containing the symbol value and confidence), and symbol correlations (which list the strokes for a recognized character). Recognizers may return partial results during an inking transaction.
Beyond the requirements for collecting ink and returning results, no constraints are placed on how recognizers work. Windows for Pen Computing was designed to accommodate recognizers for languages that use non-Roman alphabets. The recognizer interface is open and published, and third-party recognizers will be widely available. Some Windows-based applications may provide their own custom recognizers for special symbols.
At system startup, a default recognizer specified in PENWIN.INI is loaded and used by the default RC. Application-specific recognizers may be accessed using the InstallRecognizer and UninstallRecognizer functions. After a recognizer is installed, its handle may be used in place of the default recognizer by updating the proper field in the RC data structure for a transaction.
The default recognizer must recognize characters in the basic system language. For example, under Kanji Windows, the default recognizer recognizes Kanji characters. The default recognizer must also recognize the standard gestures (for example, paste and circle-R). Most recognizers also recognize additional character sets that support the base language. For example, Japanese writing includes Kanji, Katakana, and Hiragana characters. The default recognizer in Japan should therefore recognize all three character sets.
PCs running Windows for Pen Computing include a default recognizer selected by the computer manufacturer.
The Microsoft recognizer supports the basic Windows (ANSI) character set, which is sufficient for English and most Northern European languages. The Microsoft recognizer also supports standard gestures and delayed strokes (that is, it allows the user to dot an ‘i’ or cross a ‘t’ at any time). The recognizer can be trained by users. For example, the Microsoft recognizer’s database contains only a few of the 384 different orders for the five strokes that can be used to draw an ‘E’. Although the database was developed based on handwriting samples from several hundred people, users may occasionally need to “train” the recognizer. Training results are stored separately for each user, and the training database may reside anywhere in file space, so a user can access it from any PC on a network.
The Microsoft recognizer supports training for individual characters by individual users and adds the new information to a database for that user. Ink can be assigned to one, or even two or three, characters. This allows users to train the Microsoft recognizer to handle Boston-school style “cursive printing,” in which two or three characters are written together.
Different recognizers provide different methods to handle training, so there has been little standardization in this area. The first release of Windows for Pen Computing does not include a standard training system, but Microsoft is actively working with other vendors to define a standard user interface and a standard set of functions for training.
Microsoft also provides a shape recognizer that handles lines, circles, ellipses, squares, and rectangles. Applications must install and uninstall the shape recognizer separately and load the shape recognizer handle into the recognizer field in their RCs as needed. The shape recognizer returns characters from a “shapes” language and also returns symbol values that identify each shape.
The open architecture of Windows for Pen Computing allows pen applications to use application-specific recognizers. For example, a computer-aided engineering (CAE) application might include a recognizer for standard electrical symbols (such as capacitors and transistors) to let users enter drawings in freehand without having to select an item from a toolbox. Custom recognizers are not easy to build but add tremendous power to the interface of pen applications.
The recognizer’s job is to build a symbol graph. Even for relatively simple words and phrases, most symbol graphs include alternatives at one or more character positions. To minimize the alternatives simple applications must handle, Windows for Pen Computing includes another installable component: the dictionary function.
Dictionary functions filter a symbol graph and select a “best-guess” string. In general, a dictionary procedure examines the symbol graph and selects the most likely string from the alternatives indicated. For example, if a dictionary procedure using a list of English words is presented with the symbol graph “{lc|k}ind” (indicating “lcind” or “kind”), it would select “kind” as the best guess.
A dictionary can go even further by selecting a string that isn’t an alternative in the symbol graph as its best guess. This feature is especially useful when input is restricted to a small number of correct strings, such as names of months. A dictionary procedure is potentially far more powerful than a simple lookup program. For example, if a dictionary procedure designed to select months of the year is given the symbol graph “m{a|u}y”, it selects “May” as the best guess although ‘M’ does not appear in the symbol graph. The dictionary procedure takes advantage of the context (it must be a month of the year) and knowledge of English (months of the year are always capitalized) to return this result.
Dictionary procedures work hand in hand with the recognizer to improve the accuracy of the final result. With careful design, applications can combine ink recognition with an understanding of language syntax. The result will be increased accuracy in the interpretation of handwriting.
We will now examine information flow into both unmodified Windows-based applications and applications that take advantage of the pen.
Whenever the pen touches the tablet, the pen driver generates an interrupt and calls an entry in PENWIN.DLL. The RC Manager generates a mouse message (WM_LBUTTONDOWN) and sends it to the window procedure of the window that owns the pixel the pen touched. Because existing Windows-based applications treat the message as a mouse message, the RC Manager continues to send mouse messages to the window as the pen moves around. In this mode, the RC Manager ensures that the pen is a mouse, and no recognition contexts are used.
A user sometimes touches the tablet with the pen with the intention of writing or gesturing—that is, the user wants to use the pen as a pen and perform an inking transaction. Some inking transactions are as short as a single gesture; others may involve writing several lines of text or creating a complex drawing. Regardless of the duration of the inking transaction, your application works with the RC Manager to interpret the pen input.
Applications can be involved with the pen as a pen on several levels, each differing in the amount of code that specifically handles the pen and in the user interface improvements for that investment in new code.
With the “least new code” approach, you simply call the RC Manager ProcessWriting function when you get a WM_LBUTTONDOWN message. If ProcessWriting returns zero, the message came from the mouse and you’ll need to perform your standard mouse-handling routines. If ProcessWriting returns a nonzero value, the message came from the pen and the inking transaction has already been handled. ProcessWriting calls the recognizer and dictionaries and converts the result into mouse and keyboard messages that it sends to your application before it returns the nonzero value. Your program hardly changes at all.
This level of support provides a high degree of compatibility with keyboard and mouse systems. For this strategy to succeed, your application must use the standard Windows keyboard and mouse interfaces for operations such as selection, cut, copy, and paste. This approach is simple and adequate for applications that do not intend to take advantage of the new market opportunities presented by the pen. On pen computers, these applications will almost certainly be eclipsed by applications that are specifically designed for the pen.
A higher level of involvement with the pen requires that your application handle the analysis results from the recognizer and dictionaries directly. Like the previous approach, it begins with the WM_LBUTTONDOWN message, and the code handling the message is the same. You call ProcessWriting, and the RC Manager works with the components of Windows for Pen Computing to obtain a result. If the message came from the mouse, ProcessWriting returns zero; if it came from the pen, ProcessWriting returns nonzero.
However, when the RC Manager sends the results to your window in the WM_RCRESULT message, you handle the results directly. The lParam parameter in the WM_RCRESULT message points to an RCRESULT data structure created by the RC Manager for this transaction. This data structure contains the raw pen data, the symbol graph created by the recognizer, the best guess selected by the dictionaries, and other fields useful in analyzing the results.
When the WM_RCRESULT message is sent, you’ll examine the RCRESULT data structure and add the specified text or perform the specified action directly. When that’s done, simply return nonzero from the WM_RCRESULT message to let the RC Manager know that you’ve handled the transaction results and that it doesn’t need to convert the results into equivalent mouse and keyboard messages. Note that the “least new code” approach outlined earlier performs this conversion because DefWindowProc returns zero to the WM_RCRESULT message.
Doing the work directly gives you greater control over how the results are handled and improves performance. If you have a nonstandard user interface to a function, you must do the work yourself, because the RC Manager’s results-to-messages conversion fails for that function.
For ultimate control, your application can use the RC Manager’s Recognize function that sends the WM_RCRESULT message and then returns. Your application can do anything it wants with RCRESULT. It can accept the best guess directly, analyze the symbol graph itself, or even try out a different recognizer by using the RecognizeData function.
All the approaches we’ve described let you provide an RC for the transaction. ProcessWriting allows the rc parameter to be NULL, but both ProcessWriting and Recognize can work with your RC. At the beginning of a transaction, your application can allocate storage for an RC data structure, call InitRC(&rc) to set the RC fields to default values, and then hand the RC to the RC Manager. Your application may modify any of the RC fields to reflect almost any requirements you have for the transaction and to increase the probability that the best guess is the only result you’ll need to examine. For example, you might specify a dictionary procedure written for a specific control on a dialog box or limit the characters being recognized to numbers. By setting one or two fields in the RC before starting the RC Manager, you can improve the recognition accuracy significantly.
You should focus on the importance of the RC and the RC Manager when handling the pen as a pen. The RC Manager takes direction from applications through the RC, it works with the other components to produce a result, and it returns those results to the requesting application. Most of your programming for Windows for Pen Computing will involve modifying the way in which the RC Manager makes decisions about how to handle the user’s pen input.
Now that you’re familiar with the Windows for Pen Computing environment, you should be ready to modify an existing application or write a new program to take advantage of the pen. To design your application, use the Windows for Pen Computing SDK with the Windows version 3.1 SDK.