Dr. VBish

Reading Binary Data Files -- Even with Class -- and Playing Shell Games

Bill Shadish

I recently lost some valuable time on hoax virus warnings (grhhh). Perhaps I can save you some time by simply alerting you to HTTP://CIAC.LLNL.GOV and http://kumite.com/myths/. Also recommended: http://kumite.com/myths/home.htm and HTTP://SASSMAN.NET.

Hey, guess what, sports (or should I say, VB) fans? I'm happy to note that this column marks the beginning of my third year as your doctor. And surprise! So far, there have been no known deaths resulting from my doctoral scribings. However, our legal experts do tell me that I must mention that there have been some side effects on some of our readers. Known cases involve wheezing, gasping, and, occasionally, light chuckling. However, the virtual fine print say that this is known to affect less than one percent of you, so, as my helpful and esteemed (he's also a noted TV personality) colleague Dr. Nick has been known to say, "This is a good day to . . . operate."

And on that note, off we go.

Binary files?
I have a binary file that I need to open. Each record is 80 bytes in length. I need to open this file with VB5 and parse out the data in each record. Can you please suggest some code to get me started on the right track?

Gary Cassel (gcassel@home.com)
Baltimore, MD


I'll assume that your binary file is a collection of ANSI characters kept within a file where the actual record format is known. If you're using an ISAM (indexed sequential access method) file system such as BTRIEVE to create and house your binary data, then you might not be able to use the solution I propose below. That's because ISAM structures can keep a variety of information in their tightly coupled index files and might even store variable-length records. I also assume you know the contents of the file, and if you're working with a binary file whose structure is a mystery, well, you've got a different problem. You might, however, be able to use my approach as a tool to partially automate a brute-force trial-and-error approach to deciphering the contents . . . But, for our purposes, let's assume that this is just a single fixed record length structure file where you know what to expect within each record.

In that case, it's not too hard to get at the data. First, define your record format in a Type'd data structure, and then use the VB Read statement to pull it out. Given the "record layout" as described in the Employee_Record shown, the Read menu item in this month's example will cycle through the records in the binary file, one at a time. The employee's name for each record, in turn, is shown in a Msgbox that's displayed during the loop (see Listing 1).

Listing 1. Code to read 80-byte records in a binary file.
 
 Type Employee_Record
    SSN As String * 9
    FirstName As String * 15
    MiddleName As String * 15
    LastName As String * 15
    Dept As String * 5           ' Dept Key
    Mgr As String * 5            ' Mgr Key
     SecurityLevel As String * 1  ' Numeric Code
     Title As String * 15 
 End Type
 Dim EmpRec As Employee_Record
 Private Sub mnuRead_Click()
    Dim strMsg$
    Dim BinFile As Long    
    strPath$ = getPath()
    BinFile = FreeFile    
    Open strPath$ & "\" & "Temp.Bin" _
       For Binary Access Read As #BinFile Len = 80
    Dim lRecCount As Long   
    lRecCount = 1    
    While Not EOF(BinFile)            
       Get #BinFile, (lRecCount * Len(EmpRec) -_
          Len(EmpRec) + 1), EmpRec    
       MsgBox EmpRec.SSN & ", " & EmpRec.FirstName & _
          ", " & EmpRec.LastName        
        lRecCount = lRecCount + 1   
    Wend    
    Close #BinFile    
 End Sub


While there's plenty of file-specific hard-coding going on in this example -- for example, the record length and the fact that the empRec's variables must be known -- there are a couple of techniques I think you'll find useful. Given our assumptions (80-byte record), the Get method that we use to read the data must be positioned at the beginning of each record manually. That is, Get must be set to 1 the first time through, 81 the second time, and so on. Now, this isn't something you'll want to hard-code, especially if there are a number of different file lengths involved. So, the calculation (lRecCount * Len(EmpRec) - Len(EmpRec) + 1) handles this for us by using the size of the Employee_Record (formerly known as the EmpRec) to advance to the next record each time we travel through the loop. Also, we pull in the entire EmpRec in one read, rather than the more tedious approach of reading in each variable at a time, in order. Notice that it's a simple matter to address the individual variables within the EmpRec once the overall record is retrieved as illustrated in the MsgBox statement.

Read with class
Now, it might be a little more interesting to build a generic Class file to handle your binaries. This is one of those "write it better once up front and make it easier ever after" scenarios. We'll let our class handle Binary or ASCII files, but only the binary portions are built into the example in Listing 2. The properties for the Class (cFile) to handle your files might look something like the Listing.

Listing 2. Properties for a binary file-reading class. 
 
 Public Const EMPLOYEE = 1
 Public Const DEPARTMENT = 2
 Public Const DIVISION = 3
 Public Property Get RecSize() As Variant
    RecSize = mRecSize
 End Property
 Public Property Let RecSize(ByVal vNewValue _
    As Variant)
    If IsNumeric(vNewValue) Then
       mRecSize = vNewValue
    Else
       mRecSize = 1
    End If
 End Property
 Public Property Get FileType() As Variant
    FileType = mFileType
 End Property
 Public Property Let FileType(ByVal vNewValue _
    As Variant)
    If IsNumeric(vNewValue) Then mFileType = vNewValue
 End Property


Notice the FileType property, which allows you to have code handling binary, ASCII, or other types of files. Also, the RecSize property will let us feed in the record size for a file exactly one time (rather than having it hard-coded throughout the application code itself).

Then we'll add the generic routines to handle the normal Open, Get, and Close VB statements for processing a file (see Listing 3). Notice that we use the properties defined in the class to avoid hard-coding them throughout the application. This makes maintenance easier -- you won't go kicking and screaming if someone wants to up the record size to 81, since a change for this only needs to be made in two places now. Also, since the actual calls to manipulate the file are now found in one place, this more easily allows you to move away from a binary file structure in the future, if so desired. The RecPointer property will be used to automate away the runtime record-keeping of determining which record you're on and the calculations that are required to move you the correct number of bytes into the file to read or add new records. You'll find more routines to handle these basic tasks in the accompanying Download file.

Listing 3. The class methods.
 
 Public Sub bClose(Optional bFile)
    Close mFile
 End Sub
 Private Property Get RecPointer() As Variant
    RecPointer = mRecPointer
 End Property
 Private Property Let RecPointer(ByVal vNewValue _
    As Variant)
    mRecPointer = vNewValue
 End Property


Now we'll add functionality to handle things like parsing a record once we've read it. The Parse routine shown in Listing 4 will trim and separate fields within a record, using a definable character, which is # in our example. The mnuReadWithClass_Click routine is called from the menu, to use our cFile class to read and parse a record. Figure 1 shows the resulting parsed record.

Listing 4. The class extensions for the parsing routine.
 
 Private Const PARSECHAR = "#"
 Public Function ParseRecord(iRecType As Integer) As String
    Dim strBuf$
    Select Case iRecType
       Case EMPLOYEE
          strBuf$ = EmpRec.SSN & PARSECHAR & Trim$_
             (EmpRec.FirstName) & PARSECHAR & _
             Trim$(EmpRec.MiddleName) & PARSECHAR & _
             Trim$(EmpRec.LastName) & PARSECHAR & _
             Trim$(EmpRec.Dept) & PARSECHAR & Trim$_
             (EmpRec.Mgr) & PARSECHAR &_
             EmpRec.SecurityLevel & PARSECHAR & _
             Trim$(EmpRec.Title) & PARSECHAR
    End Select
    ParseRecord = strBuf$
 End Function
 Private Sub mnuReadWithClass_Click()
 '
 'This routine displays the employee record, parsed out
 ' with "#" characters between the fields. (The parse 
 'character can as easily be commas or whatever you'd 
 'like. We return the opened file handle into 
 'FileHandle on open. This syntax allows you to (later) 
 'have multiple files open at the same time. For our  
 'example, we just use the currently opened file,
 ' which is the employee info file.
     
 Dim objFile As New cFile
 Dim lFileHandle As Long
 Dim strBuf$
 objFile.FileType = BINARY_FILE
 objFile.RecSize = 80 
 lFileHandle = objFile.bOpen(READMODE, getPath() _
    & "\Temp.Bin")   
 clearRec EMPLOYEE    
 strBuf$ = objFile.bRead(lFileHandle, EMPLOYEE)
 MsgBox strBuf$, vbOKOnly + vbInformation, "Parsed Record"
 clearRec EMPLOYEE    
 MsgBox objFile.bRead(lFileHandle, EMPLOYEE), _
    vbOKOnly + vbInformation, "Parsed Record"    
 objFile.bClose
    
 End Sub


Ahhhh. Problem solved -- happiness is being able to read the binary data file. However, I advise you not to stop here but to write it to a SQL database structure -- for example, in Jet, Access, or SQL Server -- especially if the data is static. A DBMS will provide much more flexibility in terms of reporting, manipulating, and moving the data, as well as maintainability. The only exceptions I can imagine that might keep me from saving the original data out to a SQL database is if the data is frequently changed or if the overall size of your application files is critical.

Shell games
I've made an EXE file with VB5, and I want to use it in Word97. I use the command shell : 

 Sub do_it()
            Shell ("c:\myprogram.exe"), vbNormalFocus
            msgbox "done"
 End Sub


When I activate the Word macro do_it , myprogram.exe will start. The problem is that the statements after the shelled program are executed immediately. It does not wait for the shelled program to finish, even though the users should click on an OK button in the program (so the focus is in the shelled program). Is there any way to get around this?

René (jusho@wxs.nl),
Amsterdam, Netherlands


This is a fairly common problem -- and not only in the world of VBA. The simplest trick is to use a hard-coded semaphore to hold the Word macro up until the VB app runs. In this context, my hard-coded semaphore is going to be nothing more than a small ASCII file that the Word macro and the VB application will be aware of. This file will be used as a flag to signal which step in the process has control at any given point in time.

Using something along the logical lines of the following pseudo-code would work to solve your problem (see Figure 2).

1. Start the Word macro.
2. Create a control file (in a known location).
3. Start the VB app.
4. Fall into a While Loop, within the macro, that checks for the control file.
5. The VB app completes its work and deletes the control file as part of the shutdown process.
6. The Word macro then continues happily along.

Our example adds a parameter ("WAIT") to the VB call within the Word macro (see Listing 5). If that parameter is present when starting the VB app, then the Form Load routine will catch that and kick immediately into the suspense mode, awaiting a user OK click to continue on. The Word macro will wait for this file to disappear and then continue on its way.

Listing 5. The VB and Word macro code to have VB wait.
 
 ' VB code
 Private Sub Form_Load()
    If Left$(UCase$(Command$()), 4) = "WAIT" Then
       Call mnuWait_Click
    End If
 End Sub
 Private Sub mnuWait_Click()
    MsgBox "Let's pretend that this  
    If DoesFileExist("c:\test1.tst") Then Kill _
       "c:\test1.tst"
 End Sub
 
 'Word Macro Code
 Sub RunApp()
 '
 ' RunApp Macro
 ' Macro created 11/02/98 by Bill Shadish
 '
    Dim lControlFile As Long
    lControlFile = FreeFile   
    Open "c:\test1.tst" For Output Access Write As _
       #lControlFile
    Print #lControlFile, "XYZ"
    Close #lControlFile
    Shell "c:\drvb9901.exe WAIT", vbNormalFocus   
    While DoesFileExist("c:\test1.tst")
    'you may wish to add a timeout warning message here
    Wend
    MsgBox "There, that's better!", _
       vbInformation + vbOKOnly
 End Sub


This control file angle is the simplest approach to use. The only real drawback is that you have to create a separate physical file to use as the process controlling flag. As long as your application checks first for a previously created file and removes it when you're done, there are only a few ways that the file approach can fail. You might abort in the middle of running for some as yet unknown reason. This might leave an errant control file floating around where it isn't expected to be, and so on. Again, proper startup and cleanup code, which is completely reusable within other applications, prevents many of the possible problems.

Notice that the creation/deletion logic can be reversed. You can create the control file from either the Word macro side or the VB application -- use whichever piece is easier to do. Note that you might want to use the DoesFileExist routine from the preceding Binary File answer.

If you don't like the ASCII file approach, then you can also store similar control information into a database table. Using a database control table in this way also allows you to keep and track additional data, including the starting and stopping times of all involved components of the application. You can also store successful termination codes or error codes that can become part of an overall status log, available for the reading pleasure of your favorite systems administrator.

How to send me your questions
Send in your questions via the Dr. VB-ish site at HTTP://WWW.FO.COM. At the bottom of this page is a growing list of the very best links that have been discussed in this column so far.

Download drvb199.exe

Bill Shadish is a principal of Fundamental Objects, Inc., where he works with ActiveX controls and OLE server technology. (Why, he's even been known to do some Lotus Notes work from time to time!) Bill writes for a number of VB trade journals, including Visual Basic Developer, Inside Visual Basic, VBPJ, and Visual Programming++. bills@fo.com.