Controlling the RichTextBox Insert State


Every RichTextBox control has its own insert state that is independent of the global insert state as well as the insert state of all other RichTextBox controls. That would be fine if you had some way to read and change the state. But you don’t. The RichTextBox control has no property to check the state, and the underlying RichEdit control has no message to check the state.


This might seem like pretty dumb behavior, but it’s consistently dumb. The control always starts with insert mode on and then toggles every time the user presses the Insert key. Since we control creation of XEditor’s internal Rich­TextBox control, we can read the initial state (always insert on) and keep track of it. We can change it whenever we want by sending a fake insert keystroke.


The OverWrite property controls the insert state, and it, in turn, is controlled by the fOverWrite member variable. When the control is created, the OverWrite value set by the user at design time will be read into fOverwrite by UserControl­_ReadProperties. There are two possibilities, depending on whether the fOver­Write variable is False or True. If the fOverWrite variable is False (meaning insert mode is on), the RichEditBox insert mode is always on when the control is created, so the variable matches the real insert state. But if the OverWrite mode is True, the real state does not match the flag and must be changed.


The change happens in the UserControl_Load event, which occurs just once after all the properties have been read from the form.

If fOverWrite Then
    ' Make overwrite state match fOverWrite variable
    SendMessage txt.hWnd, WM_KEYDOWN, ByVal VK_INSERT, ByVal &H510001
    SendMessage txt.hWnd, WM_KEYUP, ByVal VK_INSERT, ByVal &HC0510001
End If

The WM_KEYDOWN and WM_KEYUP messages send an insert key to the internal control, changing its state. After you get the insert state in sync, you have to keep it that way. There are two ways that the state can change. One is if someone changes the OverWrite property. Here’s the code that makes that happen:

Property Get OverWrite() As Boolean
    OverWrite = fOverWrite
End Property

Property Let OverWrite(ByVal fOverWriteA As Boolean)
    ' Only change if value changed
    If fOverWriteA <> fOverWrite Then
        fOverWrite = fOverWriteA
        ' Change the keystate to match
        SendMessage txt.hWnd, WM_KEYDOWN, ByVal VK_INSERT, ByVal &H510001
        SendMessage txt.hWnd, WM_KEYUP, ByVal VK_INSERT, ByVal &HC0510001
        StatusEvent
        PropertyChanged "OverWrite"
    End If
End Property

Again you have to change the key state the hard way. Notice that you call StatusEvent to generate a StatusChanged event so that clients (such as Edwina) can update any insert state display.


The only other way the insert state can change is if a user presses the insert key, but you can monitor key presses in the KeyUp event of the XEditor control:

Sub txt_KeyUp(KeyCode As Integer, Shift As Integer)
    If KeyCode = vbKeyInsert Then
        ' Insert state changed, so change the variable to match
        fOverWrite = Not fOverWrite
        StatusEvent
    End If
    RaiseEvent KeyUp(KeyCode, Shift)
End Sub

The OverWrite property sent a keystroke to change the internal state. The KeyUp event changes the variable to match the state. Think for a minute about what would happen if sending an insert key message in the OverWrite property caused the KeyUp event to fire. This code wouldn’t work because the fOverWrite variable would be toggled one too many times. Fortunately, faking the key with SendMessage doesn’t fire the KeyUp event.

Handling the insert state is messy business, but fortunately XEditor handles the details so that Edwina and other clients can simply read or write the OverWrite property. Instead of using a real Insert panel in the status bar, Edwina uses a normal text panel with the letters INS on it. Edwina can update the insert display in the StatusChange event and toggle the OverWrite property when the user clicks on the insert panel.

The KeyUp event in XEditor is the perfect place to add new command keys or to change the editing behavior of the class in other ways. In your own enhanced version of XEditor you might want to add macro handling or key reassignment.