Going Native with J/Direct

DR. GUI Online
Part 1: November 17, 1997
Part 2: November 24, 1997
Part 3: December 1, 1997
Part 4: December 8, 1997

Dr. GUI
MSDN Online

Contents

Part 1: Dr. GUI Goes Native with J/Direct
Part 2: Going Native with J/Direct
Part 3: Still Native after All This Time
Part 4: The Conclusion, Finally, of the Column That Wouldn't Die

Part 1: Dr. GUI Goes Native with J/Direct

Java . . . Again?

As regular readers of Dr. GUI's columns know, Dr. GUI has had his ups and downs with Java. While, like all good programmers, Dr. GUI would love to be able to port code with no effort to other platforms, the good doctor has noticed many, many reports of cross-platform incompatibilities, bugs, and performance problems—sometimes quite severe performance problems.

So, as a cross-platform API, Java isn't all that the hype machine says it is. As Dr. GUI's Polish grandmother used to say, "Fooled me once, shame on you. Fooled me twice, shame on me." And Dr. GUI doesn't intend to be shamed by being fooled again.

A good language . . . 

As a language, Java's pretty cool: it gives you the tools you need to do serious object-oriented programming without all of the complication of C++'s "object-oriented assembly language" approach. Around the time Java came out, the good doctor was involved in a project to produce an interactive tutor for C++. Dr. GUI was quite envious of Java at that time: it would have been far simpler to teach Java than C++.

Many programmers like Java as a language—with good reason. In addition to being simpler than C++, Java also has cool features such as garbage collection. Not having to worry about when to free memory is a big relief to most C++ programmers, so garbage collection makes them more efficient.

 . . . especially for Win32 programming

So if you're programming for Win32, Java can still be a great choice—especially for programs where performance isn't critical. (Make sure before you commit: at least one company was so disappointed in Java performance that they rewrote their program in Visual Basic—and reaped a performance improvement of about two orders of magnitude.)

Beyond Java: Getting Native

You've heard it before: "Java language, good. Java API, lacking." But what if you want to use Java for real work? What if your Java program needs to get access to Windows functionality not supported by Java?

As you well know, any language needs to be able to use the native capabilities provided by the operating system. With Java, this need is especially acute since the built-in functionality is so incomplete. Even Java's creators anticipated this need and added the capability of writing native methods to Java.

But don't forget that as soon as you use native methods of any sort, you kiss whatever cross-platform pipe dreams you had goodbye. If you isolate your native calls into a small set of classes, you might be able to port most of your classes, but native anything is never a cross-platform solution. Native methods are, however, a great way to get the speed and functionality your customers demand.

Java virtual machines (VMs) provide some form or another of native method interface. These native method interfaces are one of the least consistent parts of Java. Just on Win32, there have been four methods so far of calling native methods more or less directly. Sun's and Microsoft's VMs have always differed in how their native method interface works.

The method supported by the Microsoft VM is called RNI, for Raw Native Interface. (Not to be confused with RMI—Remote Method Invocation.) A quick look at the documentation for this confirms that RNI is indeed "raw"—you have to worry about locking pointers, turning garbage collection on and off, and many other problems. dynamic-link libraries (DLLs) you call through RNI need to be specially written to work with RNI—you can't use RNI to call arbitrary DLLs, although you can write wrapper DLLs.

In exchange for this complexity, you get full access to all of the fields in every object. And you get speed—RNI is very fast. But RNI is more complex than most programmers need, so it's better reserved for systems-level programming tasks such as extending the capabilities of the VM. So leave it alone unless you need to do brain surgery.

Native code for the rest of us

Thankfully, there are methods of accessing native code that don't require brain surgery: Java/COM integration and J/Direct.

Microsoft's Java VM added the ability to treat COM objects as special Java classes. Java/COM integration allows any Java Bean to be an ActiveX control and any ActiveX control to be a JavaBean—all nearly automatically. COM integration is best for calling APIs that are wrapped in COM objects—or for ActiveX controls. (We'll discuss using COM objects in Java and using Java to write COM objects in a later column.)

About the only disadvantage to this method is that there's a translation layer between Java and COM, so the performance won't be as fast as if you were using C++ (and therefore had one less translation layer).

This integration is very powerful, but if you just want to call into a C DLL (such as the Windows API), you'd still have to write a wrapper COM object just to call your DLL.

Direct to you!

That's where J/Direct comes in. J/Direct allows you to call most any DLL "directly"—in other words, without having to write an RNI or COM wrapper DLL. The VM takes care of thorny issues such as mapping of data types for you. J/Direct is best for calling APIs (in Windows or your own DLLs) that are not wrapped in COM objects. This allows you to access almost the entire Windows API, for instance—and enables a new class of Java programs: Windows programs written in Java.

So how do I get direct?

All you do is declare a static native method preceded by a special directive in a comment, then call the method you've declared. The VM takes care of translating parameters and adjusting calling sequences—all you do is call. Note that, unlike other methods, there's no reason to write a wrapper DLL or a wrapper COM object—just call the functions you need directly. Take a look at the code below for an example of how to use MessageBox from Java. (Can you believe that Java still doesn't have an easy equivalent to MessageBox? The good doctor can't.)

class ShowMsgBox {
  public static void main(String
   args[])
  {
    MessageBox(0, "It worked!", 
        "This message box from Java", 0);
  }

   /** @dll.import("USER32") */
   private static native int
   MessageBox(int hwndOwner, String
     text, String title, int fuStyle);
} 

This class has a declaration of a function (MessageBox) and a definition of a function (main). The main function is the standard entry point for Java applications (not applets). It calls the Windows API MessageBox directly and easily.

The declaration of MessageBox itself couldn't be simpler. It's just a standard Java native method declaration—preceded by a special comment called a "directive" that tells the compiler what DLL it'll find the MessageBox entry point in. In this case, the @dll.import directive just says that MessageBox can be found in USER32.DLL.

Note that now any method in any class can call ShowMsgBox.MessageBox, as does the code in main above. In general, it really is that easy.

As you see, J/Direct is a great way for Java programs to access the full power of Windows—you can call almost any Windows API. (There are some restrictions when the data types manipulated by the API can't be translated to the Java world—but those restrictions are surprisingly few. Even pointers and callbacks are supported!)

Note that since code that uses J/Direct can do anything, it's considered untrusted—so applications can run it, but not applets. However, it is possible to wrap code that calls J/Direct in a Java wrapper object that you sign as safe in one or more categories.

More on @dll.import

The main directive you need to know is the @dll.import directive. It's this directive that provides a way to tell the Java VM the DLL to which it needs to link.

You can specify the DLL to use for static native methods in a whole class by putting the directive immediately above the class declaration. Or you can specify which DLL to use on a method-by-method basis, as shown in the MessageBox example above.

@dll.import also has parameters to allow you to link to ordinals, use an alias for the function name, and modify the calling convention. The good doctor will share more on these next week.

Making it easy to access the Windows API

You're probably looking at this and saying, "Nice—but I don't want to have to write these funky comment directives for each Windows function I want to use." Fortunately, you don't have to declare the entire Windows API yourself—Microsoft provides handy classes that contain these declarations. All you have to do is import the correct class(es).

All of these classes are available in the "com.ms.win32" package. So the MessageBox sample above could be simplified to:

import com.ms.win32.*;

class ShowMsgBox {
  public static void main(String
   args[])
  {
    MessageBox(0, "It worked!", 
        "This message box from Java", 0);
  }
}

The available APIs are declared in a set of classes—one for each Windows DLL:

For instance, MessageBox is contained in USER32.DLL and therefore declared in User32.class; just import this class (or the whole package) and you can call MessageBox directly.

Constants used in Windows are available in the "win" class. To access these, just import com.ms.win32.win and use win.CONSTANT_NAME, as shown below. Don't try to take a shortcut by extending or implementing com.ms.win32.win in your class; doing so will require that the entire class be loaded each time you load your class. (Talk about slow!) Write out "win.xxx" instead.

System.out.println("MAX_PATH = " + win.MAX_PATH);

Finally, the supported Windows structures are declared in a set of class files that have the structure name as the class name, such as RECT.class. Just import these and you can use the Windows structures as though they were Java classes.

Again, you get all of these classes when you import com.ms.win32.*. And note that the classes are an optional part of the Software Development Kit (SDK) for Java 2.0 install, so if you don't have them, just install again. (Since the classes contain only declarations, your users won't need to have them installed on their machines—but you'll need them in order to be able to compile.)

Try It Out!

Now you've got the basic knowledge you need to start playing with J/Direct. To get started, you'll need the SDK for Java 2.0. And having Internet Explorer 4.0 wouldn't hurt.

You won't be able to do everything just yet: we've not discussed how structures and callbacks work, so don't try to implement Petzold's GENERIC.C in Java—not just yet. There are a lot of APIs you can use to get experience in the meantime—or you can hop up to the Microsoft technologies for Java Web site (http://www.microsoft.com/java/default.htm) to read ahead. We'll talk about the remainder of the J/Direct topics in Part 2.

Part 2: Going Native with J/Direct

In Part 1, the good doctor shared some of the basics of using J/Direct to call into DLLs directly (including the DLLs that implement Windows). We broke off that conversation just before getting into some of the tricky and sticky parts of J/Direct.

The key issue is this: Java and Windows (primarily written in C and C++) don't deal with the world in exactly the same manner. The data types are sometimes different, the ways that memory is manipulated are different, and Windows has some concepts such as callbacks (requiring a pointer to a function) that Java just doesn't have. Also functions that are part of OLE have their own special rules.

As a result of these differences, J/Direct isn't really direct—it just looks direct. The VM has to do some level of translation (or marshalling) on just about every call. So while J/Direct will be far better than other methods for calling DLLs such as the Windows DLLs, it's still not going to be as fast as C++ because there's still a translation layer being used for each call and return; but we've already established that if you're using Java, performance isn't the most important issue for your application, anyway.

This time, Dr. GUI's going to discuss the issues of data marshalling. Next time, we'll discuss some of the odds 'n' ends, including callbacks and using dynamically loaded DLLs.

Translating Data with Data Marshalling

Although they're often close, the data types in Java don't map exactly to C/C++. Built-in data types map in a pretty natural manner, but there are some differences depending on whether the data element is a parameter to a function or a field embedded in a structure.

Strings, structures, and pointers all require special handling, as do callbacks and parameters that are polymorphic, or interpreted as different types in different situations.

Here is a table of the data mappings for parameters and return values:

Most of the data type mappings are what you'd expect—byte to char, int to int, long to _int64, boolean to BOOL, and so on. But a few are surprising and some are different depending on whether the data is a parameter or a member of a structure (declared with @dll.struct—we'll discuss this in a bit).

There are no unsigned data types (except char) in Java, so be careful when operating on unsigned values. And don't forget that long is 64 bits in Java, not 32 as in C/C++.

When passed as parameters, arrays map to pointers to the element type. For instance, byte [] in Java maps to CHAR* in C/C++. There are restrictions on return values, especially Strings and StringBuffers.

The Trouble with Structures

At first glance, you'd think that mapping structures from Java to C/C++ and back would be easy. After all, isn't a structure basically a class without member functions?

As it turns out, structures cause some special problems because they're stored very differently in C/C++ and in Java. In C/C++, a structure is stored in a contiguous block of memory, usually with some padding bytes between members to prevent unaligned accesses. (For instance, all members that are two bytes or longer would start at an even address, all members that are four bytes or longer would start at an address evenly divisible by four, and so on.)

In Java, the fields in a class are stored in some implementation-dependent fashion, but the fields are addressed (at least at first reference) by name, not by offset. This is required so that you can use updated versions of your class with old code, even if you add fields and methods.

In addition, the Java VM moves objects when it does garbage collection. This can cause a real problem if you pass the object by reference to a function in a DLL, so we have to insure that our special structures won't move.

The J/Direct solution to this problem is simple: you declare a class with a special "funky comment" @dll.struct directive and the VM creates an appropriate contiguous structure to hold the data memory (you can specify how to pack/pad the structure) and guarantees that the data won't move. For example:

/** @dll.struct() */
class SYSTEMTIME {
  public short wYear;
  public short wMonth;
  public short wDayOfWeek;
  public short wDay;
  public short wHour;
  public short wMinute;
  public short wSecond;
  public short wMilliseconds;
}

The following example uses the SYSTEMTIME structure in a DLL method call.

class ShowStruct {
    /** @dll.import("KERNEL32") */
    static native void GetSystemTime(SYSTEMTIME pst);
    public static void main(String args[])
    {
      SYSTEMTIME systemtime = new SYSTEMTIME();
      GetSystemTime(systemtime);
      System.out.println("Year is " + systemtime.wYear);
      System.out.println("Month is " + systemtime.wMonth);
      // etc.
    }
} 

As we've seen before, the VM takes care of all of the sticky details.

Arrays, strings, and other objects (such as structures) in structures pose special problems. In Java, all objects (including arrays and strings) are allocated on the heap, so when you have a member of a structure that's an object, it really is just a reference to that object. So if you declared an array of five ints in a class in Java, your objects would contain a reference to the array of five ints on the heap, not the actual array of five ints.

In C/C++, if you declare an array of five ints in a structure, you get the five ints right there. If you want a pointer, you declare a pointer. Java doesn't have this flexibility. But when you pass a structure from Java to a C/C++ DLL, it needs to conform to the way that C/C++ does things.

First, if you want to embed a structure inside a structure, then both must use the @dll.struct directive. But you may want to embed the fields of the embedded structure directly to save the VM time in marshalling from a structure reference to an embedded structure.

For embedded arrays and strings, we need yet one more directive: @dll.structmap. This directive specifies the size and type of the array that maps to the Java reference. For example the Windows LOGFONT structure is defined as follows in C/C++:

   typedef struct {
     LONG lfHeight;
     LONG lfWidth;
     /* <many other fields deleted for brevity> */
     TCHAR lfFaceName[32];
   } LOGFONT;

This structure can be expressed in Java using a directive to specify the size.

   /** @dll.struct() */
   class LOGFONT {
     int lfHeight;
     int lfWidth;
     /* <many other fields deleted for brevity> */
     /** @dll.structmap([type=TCHAR[32]]) */
     String   lfFaceName;
   } 

The @dll.structmap directive indicates the size of the fixed string as measured in characters (including space for the null terminator).

For embedded arrays, the directive is the same, but the syntax is a little different. To map from the C/C++ structure below:

struct EmbeddedArrays
{
  BYTE     b[4];
  __int64  l[4];
  doubl    d[4];
};

you would write:

/** @dll.struct() */
class EmbeddedArrays
{
    /** @dll.structmap([type=FIXEDARRAY, size=4]) */
    byte b[];
    /** @dll.structmap([type=FIXEDARRAY, size=4]) */
    long l[];
    /** @dll.structmap([type=FIXEDARRAY, size=4]) */
    double d[];
}

By the way, you specify the packing of structures using the "pack" modifier of the @dll.struct directive, as in /** @dll.struct(pack=4) */. The value can be 1, 2, 4, or 8; the default is 8. The effect of this modifier is the same as #pragma pack(4) in C/C++.

Strings

In general, you can just pass strings to DLL functions and the VM will do the necessary conversions automatically. For example:

class ShowCopyFile{ 
   public static void main(String args[]) { 
      CopyFile("old.txt", "new.txt", true); }
   /** @dll.import("KERNEL32") */
   private native static boolean CopyFile(String existingFile,
      String newFile, boolean f);
}

Strings are read-only in Java, so the Microsoft VM will only convert the String object for input to the DLL function only. If the DLL function might modify the string, pass a StringBuffer object instead. Make sure that the capacity of the StringBuffer is large enough (s = new StringBuffer(capacity);). For example:

class ShowGetTempPath{
   static final int MAX_PATH = 260;
   public static void main(String args[]) {
      StringBuffer temppath = new StringBuffer(MAX_PATH);
      GetTempPath(temppath.capacity()+1, temppath);
      System.out.println("Temppath = " + temppath);
   }
   /** @dll.import("KERNEL32") */
   private static native int GetTempPath(int sizeofbuffer, 
      StringBuffer buffer);
}

Do not use the default StringBuffer constructor; it will allocate space for only 16 characters. You may want to write a wrapper function to insure that a proper StringBuffer is always passed. For instance, you could write a new function:

public static String GetTempPath()
{
  StringBuffer temppath = new StringBuffer(MAX_PATH-1);
  int res = GetTempPath(MAX_PATH, temppath);
  if (res == 0 || res > MAX_PATH) {
    throw new RuntimeException("GetTempPath error!");
  }
  return temppath.toString(); // can't return a StringBuffer 
}

This method offers both convenience and safety and it maps the error return value of the function to a Java exception. Notice that you cannot return StringBuffer objects.

Strings are converted to ANSI unless the "Unicode" or "OLE" modifier is used in the @dll.import directive, in which case the string is passed in Unicode format. For example, if our code was running on NT, we could avoid the Unicode-to-ANSI and ANSI-to-Unicode conversions by using:

/** @dll.import("KERNEL32.DLL, unicode) */

instead of the code above. Or, better yet, use:

/** @dll.import("KERNEL32.DLL, auto) */

The VM will pick the faster method depending on whether it's running under Win95 or NT. (Dr. GUI recommends using "auto" always when calling Windows API functions.)

J/Direct automatically takes care of calling the appropriate "...A" or "...W" variant of the functions (MessageBoxA vs. MessageBoxW, for instance) based on the conversion it decides to use. It first tries to use the function name without appending an "A" or "W"; if that fails, it appends the appropriate character and tries again.

Strings cannot be declared as return type of a DLL function except in "OLE" mode, where the native return type is assumed to be a LPWSTR allocated using the CoTaskMemAlloc function.

What's Next?

"My," sighs the good doctor, "this has stretched on a bit, hasn't it?" But we're still not quite done: we need to look at the fascinating issues of how J/Direct deals with pointers, callbacks, dynamically loaded DLLs, polymorphic parameters, calling OLE functions, and getting information on errors.

Even if you never program in Java or never use J/Direct, Dr. GUI hopes you're finding this series helpful in deepening your understanding of C/C++ and the Windows API—just as learning Latin deepens your understanding of how your native language works.

Part 3: Still Native after All This Time

Dr. GUI welcomes you back to the penultimate segment of our journey through one of the Seven Wonders of the Microsoft Virtual Machine for Java, J/Direct.

First, we discussed the basics of J/Direct. Then we talked about how J/Direct marshals garden-variety data, including structures and strings. Now Dr. GUI will operate on harder topics: pointers, polymorphic parameters, and callbacks. Next, we'll wrap it all up with dynamically loaded DLLs, calling OLE functions, getting information on errors, and a comparison between J/Direct and RNI.

Pointers

As any beginning Java programmer can tell you, Java does not support a pointer data type. So with pointers so common in Windows, Dr. GUI muses, how in the world are we going to do direct calls to Windows?

Well, the answer varies depending on exactly what we're doing. If we just need a pointer to a single data object, we can pass a one-element array of the correct type. For other uses, we can store pointers in Java integers and, if needed, use those integers (which really are pointers) to access the data they need by using various methods of DllLib such as ptrToStruct and ptrToString.

Simple pointers

So let's say you're calling a function that requires you to pass a pointer to an int in which the function will store a value. In Java, there's only one way to pass by reference: pass an array. So, to pass a pointer to a single object, just pass a one-element array instead—in this case, an array of int.

For instance, the DLL function:

void foo(int * piStatus); // C/C++ function, piStatus -> one int

would correspond with the following Java function:

/* Java with J/Direct */
/** @dll.import("foo") */
static native void foo(int [] piStatus);

and you would call it as:

/* Java with J/Direct */
int piStatus[1];
piStatus[0] = inValue;
foo(piStatus);
   int outValue = piStatus[0];

Pointers from the land of C/C++: Pointers as dumb handles

So now we know how to use pointers as additional return values. But what about when a DLL, such as a Windows DLL, passes us a pointer? What will we do with it?

When you're passed a pointer from a DLL, you have a couple of choices. First, if you don't need to manipulate it or dereference it, just store it in an int. You can then pass this int back to the DLL when needed. It's almost as if you're treating it as a handle or a cookie, rather than a pointer. For instance, the DLL functions:

/* C/C++ functions */
foo * RetPtr();
void TakePtr(foo *inPtr);

correspond to the Java functions:

/* Java with J/Direct */
/** @dll.import("foo") */
static native int RetPtr();
/** @dll.import("foo") */
static native void TakePtr(int inPtr);

and would be called as:

/* Java with J/Direct */
int thePtr = RetPtr();
TakePtr(thePtr);

In essence, you're treating the pointer as a handle or a cookie—you're not using its value at all but just passing it back to the DLL. Because Java data moves in memory when garbage collection is done, please note that attempting to get and use the address of a regular Java data structure is, according to the Surgeon General, dangerous to your program's health.

Pointers to data

But what if the pointer points to data you need to access?

If the pointer points to a structure, you can create a reference to the structure from the pointer, then use the reference as you normally would. Note that the structure must be declared with @dll.struct.

/* Java code with J/Direct */
/* RECT declaration from com.ms.win32.RECT.class */
/* import com.ms.win32.*; */
RECT rect = (RECT)DllLib.ptrToStruct(RECT.class, rawPtr);
rect.left = 0;

There are other DllLib methods you can use to convert a pointer to characters to a Java String object: ptrToString, ptrToStringAnsi, and ptrToStringUni.

There's also an overloaded DllLib.copy method you can use to copy memory pointed to by a raw pointer into a Java array of various types.

Finally, for the really hard-core hackers and systems programmers out there, you can allocate and free global and task memory, get the address of a data structure, and even read and write bytes at a specific memory address with various DllLib methods. (Aren't you glad you can only run J/Direct code with security permissions?)

Polymorphic Parameters

The good doctor hears you now: "Polymorphic parameters? Huh? Since when does Windows do inheritance?"

Well, this is a little different. You might have noticed that in some cases a parameter to Windows might be interpreted as a different type depending on the value of some other parameter passed. For instance, the final DWORD parameter to the WinHelp API can be interpreted as a pointer to a string, a pointer to a MULTIKEYHELP structure, a pointer to a HELPWININFO, or a plain integer, depending on the value of the next-to-last parameter.

So what do you do? Pass a pointer as an int?

As it turns out, it's much simpler than that: to call such a function, just provide a separate overload of the function for each possible type of parameter. When you call the function, Java will pick the correct overload based on the type of the parameter you pass and J/Direct will convert the parameter correctly. It really couldn't be easier.

If the return type is polymorphic, you'll need a separately named method for each overload (since neither Java nor C++ allows defining overloads based on return value). You can make all of these differently named methods map to the same DLL function by using the entrypoint modifier on your @dll.import directive (shown later).

Aliases

You may want to name the method in Java with a different name than is in the DLL. To do this, just add an entrypoint modifier to the @dll.import directive, specifying the name of the DLL function after the equal sign of the modifier. For instance:

/** @dll.import("USER32", entrypoint="GetSysColor") */
static native int getSysColor(int nIndex); // note "g", not "G"

Use this feature to deal with polymorphic return types. For example:

/** @dll.import("foo", entrypoint="Func") */
static native int intFunc();
/** @dll.import("foo", entrypoint="Func") */
static native long longFunc();
// Java name different, DLL name same

To link by ordinal, make the name of the entrypoint #<num> where <num> is the number of the ordinal—for instance, #24. For example:

/** @dll.import("foo", entrypoint="#24") */
static native int meanOldNum24(); // Linked by ordinal

Callbacks

You can't get very far in Windows programming without needing a callback function. In fact, you can't even create a window class, since one of the members of the WNDCLASS structure is a pointer to a callback function (your window procedure, or WndProc!). There are also other places where callbacks are necessary, from dialog boxes to window enumeration. Without a way to handle callbacks, J/Direct wouldn't be very useful: how could you use it if you couldn't even translate Petzold's GENERIC.C into Java?

So how do you make callbacks work in a language that doesn't have pointers at all, let alone method pointers? Well, it takes a little trickery.

First, you have to put your callback function (method) in its own class. This class must be derived from com.ms.dll.Callback. You then override a function called "callback" that returns void or int and takes any number of int parameters. You can use various DllLib methods to convert parameters if necessary. Note that since you'll create a new callback object for each callback, you can store data related to that callback in the object rather than passing it as the extra parameter that many callbacks take.

For instance, you could set up a call to EnumWindows as follows:

class EnumWindowsProc extends Callback {
    public boolean callback(int hwnd, int lparam) {
      StringBuffer text = new StringBuffer(50);
      GetWindowText(hwnd, text, text.capacity()+1);
   
      if (text.length() != 0) {
        System.out.println("hwnd = " + 
                           Integer.toHexString(hwnd) +
                           "h: Text = " + text);
      } 
      return true;  // return TRUE to continue enumeration.
    }
  
  /** @dll.import("USER32") */
  private static native int GetWindowText(int hwnd, 
                                   StringBuffer text, int cch);
}

You can invoke EnumWindows with this callback in the following way:

  boolean result = EnumWindows(new EnumWindowsProc(), 0);

You must prevent garbage collection of the callback object until the function you call is done with it. If your callback is used by only one function, such as EnumWindows, you can just use the callback object directly, since garbage collection is turned off while you're in J/Direct functions.

But if the callback will be used after the function you pass it to returns (for example, timer callbacks) or you want to store the callback object in a native data structure (such as storing a WndProc callback in a WNDCLASS), you need to make sure that the callback object doesn't get garbage-collected. To do this, wrap the callback object in a root handle (com.ms.dll.Root) object.

In the case where you just want to store the callback across function calls, this can be quite simple:

EnumWindowsProc MyCallback = new EnumWindowsProc;
int rootHandle = Root.alloc(MyCallback);
// Use the Callback across function calls until you're done.
Root.free(rootHandle);

To store the address of the callback in a native structure, wrap the callback object in a root handle and use the DllLib.addrOf to get the address of the root object—then store that address in the structure. For instance, to set up the window procedure in a window class, you would write the WNDPROC class, overriding the callback method:

class WNDPROC extends Callback { 
   public int callback(int hwnd, int msg, int wparam, int lparam)
   { ... }
} 

and then write:

WNDCLASS wc = new WNDCLASS(); // com.ms.win32.WNDCLASS.class
int callbackroot = Root.alloc(new WNDPROC()); // your callback
wc.lpfnWndProc = DllLib.addrOf(callbackroot);

So Now What?

Well, now you've got a few more scalpels and forceps in your J/Direct doctor's bag—but in order to be able to fully operate, you need a few more tools. So in Part 4, we'll finish up this series by covering dynamically loaded DLLs, calling OLE functions, getting information on errors, and a comparison between J/Direct and RNI.

Part 4: The Conclusion, Finally, of the Column That Wouldn't Die

If you'd told the good doctor a couple of months ago that he'd end up spending four columns on J/Direct, he would have responded with utter disbelief. Perhaps Dr. GUI would have had you committed. (He can do that, you know. But only when he's really grumpy.)

But everything takes longer than expected, it seems. And the feedback you've been sending to the good doctor has been very good—many of you have found these columns useful and interesting. So, we'll continue on—at least for one more column. (Actually, we're done as of Part 4.)

In Part 1, we dissected the basics of J/Direct. In Part 2, we sliced into how J/Direct marshals garden variety data, including structures and strings. In Part 3, Dr. GUI operated on harder topics: pointers, polymorphic parameters, and callbacks. In Part 4, we'll suture it all up by dealing with calling OLE functions, dynamically loaded DLLs, getting information on errors, and a comparison between J/Direct and RNI.

So, without further ado (or doodoo), let's start!

OLE APIs

The world of OLE APIs brings a whole new set of conventions to function calls.

Because it's important to get error status information, especially in a remote server setting, OLE functions (in OLE32.DLL and OLEAUT32.DLL) almost always return an HRESULT that indicates whether the function succeeded or not. Since the single return value is used for error status, the "real" return value is passed by reference as the last parameter, by convention. This will cause us problems in Java because passing by reference is difficult (you have to use a one-element array).

In addition, all OLE strings use Unicode character encoding. Finally, OLE functions return strings by allocating the string using CoTaskMemAlloc; they expect the caller to free the string using CoTaskMemFree. J/Direct has to deal with all these things in order to make calling OLE APIs as easy as possible.

To illustrate the contrast, the code for a simple Add function would look like this in the Win32 style of coding:

  int sum;
  sum = Add(10, 20);

In OLE style, the Add function would be written in the following way:

  HRESULT hr;
  int sum;
  hr = Add(10, 20, &sum);
  if (FAILED(hr)) {
    ...handle error..
  } 

When you use the "OLE" modifier on the @dll.import directive, J/Direct takes care of all these differences in a way that's natural for Java. The return of the Java function is the actual return value. J/Direct takes care of checking the HRESULT for you. If the HRESULT indicates an error, J/Direct throws a ComFailException. So instead of checking the HRESULT, you handle an exception to deal with errors.

So in Java, using J/Direct, our OLE example above would be declared as follows:

/** dll.import("OLELIKEMATHDLL", ole) */
private native static int Add(int x, int y);
// Equivalent to HRESULT Add(int x, int y, int * sum); in OLE

and called like this:

int sum = Add(10, 20); 
// if we got here, Add succeeded—otherwise exception

Note, however, if the HRESULT is some nonfailure value other than S_OK (such as S_FALSE), J/Direct does not throw a ComSuccessException—this is different from how Java/COM integration works. If you need to check different success codes, such as S_FALSE, you'll have to call the OLE function as a regular DLL function (without the "OLE" modifier).

J/Direct also takes care of the differences in the way strings are handled. Strings are passed to and from OLE functions as LPCOLESTRs. When you return a string from an OLE function to Java, the VM passes a pointer to an uninitialized LPCOLESTR. The VM takes care of freeing the return value string after it converts it to a Java string.

GUIDs and VARIANTs are passed using the com.ms.com._Guid and com.ms.com.Variant classes. (Note the underscore on _Guid.) The style of passing is similar to strings except that no memory allocation occurs.

For example, OLE32 exports the functions CLSIDFromProgID and ProgIDFromCLSID to map between CLSIDs and the human-readable names used by the Visual Basic function CreateObject. These functions have these prototypes:

HRESULT CLSIDFromProgID(LPCOLESTR szProgID, LPCLSID pclsid);
HRESULT ProgIDFromCLSID(REFCLSID clsid, LPOLESTR *lpszProgId);

In Java, you would declare these methods in the following way:

  import com.ms.com._Guid;
  class OLE {
    /** @dll.import("OLE32", ole) */
    public static native _Guid CLSIDFromProgID(String szProgID);
       
    /** @dll.import("OLE32", ole) */
    public static native String ProgIDFromCLSID(_Guid clsid);
  } 

To pass a COM interface pointer, you must generate a Java/COM interface class using a tool such as jactivex.exe. You can then pass or receive COM interfaces by declaring a parameter to be of that interface type.

For example, the system class com.ms.com.IStream is a Java/COM interface that represents the Structured Storage IStream* interface. The OLE32 function CreateStreamOnHGlobal could be declared as follows:

import com.ms.com.*;
/** @dll.import("OLE32", ole) */
public static native IStream CreateStreamOnHGlobal(int hGlobal,
                                     boolean fDeleteOnRelease);

Run-Time Loading and Linking

Windows includes the ability to specify that DLLs load at run time, rather than at load time. In other words, you can ask Windows to load any arbitrary DLL by name. This is handy because you sometimes can't specify the exact DLL at compile time—you may need to calculate the name of the DLL at run time, for instance. Perhaps the DLL is different for Windows 95 and Windows NT. Or perhaps you have different DLLs for a family of similar devices.

Windows supports this with LoadLibrary and GetProcAddress. You use these functions via J/Direct to load the DLL and find the function you want to call.

Calling the function, however, is a problem: Java does not support method references. Instead, we'll use an int to hold the function pointer and then we'll call the function by using the call method exported by msjava.dll. It takes an int that contains the address of the function as the first parameter, and passes any number of parameters after that to the function it calls.

For example, to call a function prototyped in C as:

BOOL AFunction(LPCSTR argument);

you'd write (after declaring the Win32 API functions):

/** @dll.import("msjava") */
static native boolean call(int funcptr, String argument);

int hmod = LoadLibrary("...");
int funcaddr = GetProcAddress(hmod, "AFunction");
boolean result = call(funcaddr, "Hello");
FreeLibrary(hmod);

It almost couldn't be easier.

Finding out about Errors

Normally in Windows, you find out about errors by using the Windows API GetLastError to obtain an error code set by the previous API call. Because the Microsoft Win32 VM for Java may execute function calls of its own in the process of executing Java code, the error code may have been overwritten by the time you get to it. Because of this, J/Direct code should not call GetLastError directly.

To reliably access the error code set by a DLL function, you must do two things. First, use the SetLastError modifier to instruct the Microsoft VM to capture the error code immediately after invoking that method. (For performance reasons, this is not done by default.) Second, call the com.ms.dll.DllLib.getLastWin32Error method to retrieve the error code. Each Java thread maintains separate storage for this value.

For example, the FindNextFile function returns status information through the error code. FindNextFile would be declared as follows:

/** @dll.import("KERNEL32",SetLastError) */
static native boolean FindNextFile(int hFindFile,
    WIN32_FIND_DATA wfd);

A typical call would look like this:

import com.ms.dll.DllLib;
boolean f = FindNextFile(hFindFile, wfd);
if (!f) {
  int errorcode = DllLib.getLastWin32Error();
}

J/Direct vs. RNI

J/Direct and Raw Native Interface (RNI) are complementary technologies you can use to call native code. (The third method is calling COM objects using Java/COM integration.)

Using RNI requires that DLL functions adhere to a strict naming convention and it requires that the DLL functions work harmoniously with the Java garbage collector. RNI functions must be sure to call GCEnable and GCDisable around code that is time-consuming, code that could yield, code that could block on another thread, and code that blocks while waiting for user input. RNI functions must be specially designed to operate in the Java environment. In return, RNI functions benefit from fast access to Java object internals and the Java class loader. A good use for RNI would be extensions to the Java VM and API.

J/Direct links Java with existing code such as the Win32 API functions and/or DLLs that were written to be called from C/C++ or Visual Basic. These DLLs were not designed to deal with the Java garbage collector and the other subtleties of the Java runtime environment. So J/Direct automatically calls GCEnable on your behalf so that you can call functions that block or perform UI without having a detrimental effect on the garbage collector. In addition, J/Direct automatically translates common data types such as strings and structures to the forms that C functions normally expect. As a result, you don't need to write glue code and wrapper DLLs.

The tradeoff is that DLL functions called via J/Direct cannot access fields and methods of arbitrary Java objects. In this release, they can only access fields and methods of objects that are declared using the @dll.struct directive. Another limitation of J/Direct is that RNI functions cannot be invoked from DLL functions that have been called using J/Direct. The reason for this restriction is that garbage collection can run concurrently with your DLL functions. Therefore, any object handles returned by RNI functions or manipulated by DLL functions are inherently unstable.

So if you need more power than you can get by sticking with "100% Pure" Java, you can use RNI, J/Direct, Java/COM integration, or any combination thereof. The compiler and the Microsoft Win32 VM for Java allow you to mix and match J/Direct and RNI, even within the same class, as your needs dictate. And it's easy to generate Java classes that wrap most any COM class or ActiveX control using the jactivex tool.

Phew! That's Done!

Thanks for reading this far! Dr. GUI hopes you've found this as fun and instructive to read as it was to write. And the good doctor needs to thank the authors of the "About J/Direct" article in the SDK for Java 2.0. (See the Microsoft Technologies for Java Web site at http://www.microsoft.com/java/) for inspiration (and some leveraged code and text).