Greater Java
By Qusay H. Mahmoud
Java and a Global Computing Engine
The limits of parallelism seem to block further advances in process performance beyond the next 10,000 years. But a third alternative leads to the concept of an uncoordinated, globally distributed, parallel megacomputer. Such computers already exist in the form of asynchronous nodes on the Internet, but they have yet to be used to their fullest extent.
— Ted Lewis, Naval Postgraduate School
The World Wide Web has been successful in what it was designed for — a network-based hypermedia information system. The present model of the Web, however, has limited support for computing. The current computing models of the Web include server-side computing using CGI, and client-side computing using scripted HTML files or applets. These computing models are inherently limited.
This article discusses the implementation of a simple global Web-based computing engine to which users can upload code for execution. It also discusses the security issues involved when running code on a remote machine.
Why Global Computing?
In computing-intensive applications, there is always a need for more computing power and better performance. Having a global computing engine built of sub-components would solve many problems related to performance, fault tolerance, and resource limitation. One major advantage of a global computing engine is the ability to use as many idle machines on the Internet as possible to solve large, complex problems. An example of such a problem is the RSA129 factoring project, where more than 600 machines were used to factor an integer of 129 digits.
Why Web-Based?
The current simplified computing model of the Web allows users to execute server-side programs by calling a CGI script that, in turn, calls another program on the server. It also allows users to download and execute mini programs (applets) on their machine, which were initially stored on the Web server. In this case, the user requests the home page that contains the applet, and the applet will migrate to the client's machine and get executed.
It's apparent that one computing model that allows users to upload code to the server-side for execution is missing. In this model, the program on the client's machine is executed on a remote machine.
Why Not Use CGI?
Using conventional technologies, a simplified version of the global computing engine can be implemented, as shown in Figure 1. However, there are a number of disadvantages as well as limitations to using CGI and the File-Upload feature:
Limited I/O. CGI was developed for the purpose of form-based information processing. In form-based processing, once the user fills out the form and submits it, there isn't much interaction between the user and the script interpreting the form. While it isn't impossible to write another CGI script to request more information based on previous input, this is complex, and there is a substantial communication overhead.
Inconvenience. If the client's program consists of more than one class, the client is required to upload all the classes to the server's machine. This is tedious and inconvenient. The other option is to use a JAR file. However, this is a clumsy way to run code on a Web server.
Figure 1: A simplified version of the global computing engine.
Dynamic Class Loading
In this model of computing, we have two programs: a client and a server. The client program can either be a dynamic applet, or the browser can call a CGI script from the Web server that generates a form for the user to fill-out. The server process is the compute engine itself, which is responsible for receiving client's requests, executing their requests (loading classes) and sending the results back to the client.
Security Considerations
Security is an important issue when code may be running on remote machines. In general, there are two types of security problems: nuisances and security breaches. A nuisance attack simply prevents you from getting your work done; for example, clients' requests overload the compute engine and the computer may crash. Security breaches, on the other hand, are more serious; for example, your files may be deleted.
Having the compute engine load arbitrary classes into the system through a class loader mechanism would put the compute engine's integrity at risk. This is basically because of the power of the class loader mechanism. Thus, to be sure that untrusted code cannot perform any malicious actions, such as deleting files, the computing engine should run in a restricted environment. We'll create a custom security manager to protect the host's file system from a client's malicious code.
Implementation Details
Class loaders are a cornerstone of the Java Virtual Machine (JVM) architecture. They enable the JVM to load classes without knowing anything about the underlying file semantics.
Classes are introduced into the Java environment when they are referenced by name in a class that is already running. The first class to run is the one with the main(String argv[]) method declared in it as static. Once the main class is running, future attempts at loading classes are carried out by the class loader.
The abstract class, ClassLoader, is a subclass of Object and is contained in the java.lang package. Applications may inherit from the ClassLoader abstract class to extend its functionality in which the JVM dynamically loads classes. Normally, the JVM loads classes from the directory defined by the CLASSPATH environment variable on the local file system. The compute engine, however, should be able to load classes off the network.
For our computing engine, we implement a custom class loader, NetClassLoader, that is capable of loading classes from remote destinations off the network.
One hidden issue when working with class loaders is the inability to cast an object that was created from a loaded class into its original class. The object to be returned needs to be cast. A typical use of the NetClassLoader would take the form:
NetClassLoader ncl = new NetClassLoader();
Object obj;
Class c;
c = ncl.loadClass("someClass");
obj = c.newInstance();
((interface) obj).someClassMethod();Note that we cannot cast obj to someClass because only the class loader knows the new class it has loaded. This means that a custom class loader cannot simply run any class without any modifications. In fact, this is a limitation of any statically-typed language. For example, for a browser to load an applet, you must implement an applet by inheriting from the Applet class. This limitation can be solved in a couple of ways: either by having a main class that each user of the system has to extend, or by having an interface that users of the system will have to implement. In the global computing engine, we chose to have an interface. Our interface has the following definition:
public interface RemoteCompute {
public void run();
}If a client is interested in using the system, he or she must implement the above interface by providing an implementation for the run method, otherwise the compute engine would fail to load the client's class. The following is an example of a sample class:
public class SampleApp implements RemoteCompute {
public void run() {
System.out.println("This is a test...");
}
}The source for the NetClassLoader is shown in Listing X on page X.
Custom security manager. The built-in Java safety features ensure that the Java system isn't subverted by an invalid code. However, these features don't protect against malicious code. For example, imagine that a client is aware of a sensitive file (spy.txt) that exists on the computing engine's file system. The client may fool the computing engine into deleting that file, or even mailing it back to the client, by writing a class:
// A sample class to delete a file.
public class Spy implements RemoteCompute {
public void run() {
File f = new File("...../spy.txt");
if ((f.delete() == true) {
System.out.println("File:" + f + "deleted");
}
else { System.out.println("File cannot be deleted"); } }}The SecurityManager class, part of the java.lang package, provides the necessary mechanism for creating a custom security manager to define tasks that an application can and cannot do. For example, suppose we want to prevent deleting files from the computing engine's file system. The following snippet of code demonstrates how convenient this is in Java:
private boolean checkDelete = true;
// Prevent clients from deleting files.
public void checkDate(String f) {
if (checkDelete) {
throw new SecurityException("Can't delete: " + f);
}
}It's important to note that when defining a custom security manager, you must override some or all the permission checking methods, depending on the policies enforced by the security manager. By default, all the methods will simply throw SecurityException, meaning that the operation is not allowed.
In the next section, we'll discuss how our custom security manager can be installed and used by the computing engine. The source for RunnerSecurityManager is shown in Listing XX on page X.
Client-Server Model
In the client-server model, the client contacts the server directly. This means that whenever a client needs a computing resource, it must know the URL of the computing engine in advance. There are two programs in this model: a client and a server. And they are both implemented in Java using sockets.
Server. The server is the process running at all times, listening for requests from clients. For this process to support requests from multiple clients, we need some form of synchronization, as well as the ability to create new processes as needed — we'll use threads.
The Java programming language has built-in support for threads. There are two ways to create a multi-threaded servers in Java: by extending the Thread class or by implementing the Runnable interface. We inherited from the Thread class and provided implementation for the run method. (Note: The Runnable.run method is distinct from the RemoteComputer.run method.)
To achieve reliability in communication, we used a TCP-based stream network connection. The Socket (for clients) class and the ServerSocket (for servers) class of the java.net package, implement reliable stream connections. The server uses the ServerSocket class to accept connections from clients. Whenever a client connects to the port number on which the server is listening, ServerSocket allocates a new Socket object, connected to some new port, for the client to communicate through. The server, at this point, can go back to listening for more requests. As you will see from the source code for the server, the heart of the server is the custom class loader that will do most of the work.
The following is the main body of the server program:
public static void main(String argv[]) {
RunnerSecurityManager RSM;
try {
RSM = new RunnerSecurityManager();
System.setSecurityManager(RSM);
}
catch (SecurityException e) { e.printStackTrace(); } new ComputeEngine();}As can be noted from the above code snippet, the first thing we do in the main program is installing our RunnerSecurityManager by creating an instance of it and registering it using the instruction:
System.setSecurityManager(RSM); where RSM is an instance of RunnerSecurityManager. The call to the ComputeEngine constructor initializes ComputeEngine by having it listen on the default port number, 5555. Then it starts the thread by calling its start method. (Note: This server doesn't protect against denial-of-service attacks. A user can submit many jobs that go into compute-intensive loops. By repeatedly submitting such jobs, the attacker can effectively shut down the server.)
Client. The client program can run via a CGI script, an applet, or even a stand-alone program that can be used from the command prompt. The applet client can be a simple applet that sends input from a TextField component to the server and displays results from the server in a TextArea component. Because applets can be problematic to run, we'll briefly discuss the implementation of a stand-alone client program.
The stand-alone client program simply opens a connection to a computing engine and sends the URL for which code is to be executed. It then waits for the server to execute the code and sends the output. It receives the output, displays it on the screen, and exits.
Broker. For a client to use the compute engine described here, the client must know the URL of the compute engine. This was the case with locating homepages before search engines.
In the computing engine case, a broker can be developed to dynamically match clients' requests with the available (registered) computing engines. Such a broker can be a server with a database, as shown in Figure 2. Each time a computing engine is started, it will contact the broker to register its properties (when it's available, for how long, type of machine, etc.). Then, whenever a client needs a computing engine, it will send a request to the broker. The request may include the type of machine needed and how long it is needed. The broker will, in turn, search its database for a match, and if a match is found, it will be sent to the client and the client can use the computing engine.
Figure 2: A server with a database.
Conclusion
We've looked at how a global computing engine can be implemented in Java. Such a computing engine idea can be improved solving of large scale problems. The architectural-neutral nature of the Java language makes it an ideal candidate for implementing such a global computing engine.
References
The Next 10,000 Years: Part I and II by T. Lewis, Communications of the ACM, April 1996.
Design and Implementation of a Web-Based Distributed Computing Systems by Q. H. Mahmoud, Masters Thesis, 1996.
The author would like to thank his masters thesis supervisor, Dr. Weichang Du, University of New Brunswick, for his supervision and guidance through the preparation of his thesis, entitled Design and Implementation of a Web-Based Distributed Computing System. This article is based on the thesis.
Qusay H. Mahmoud is a Technical Instructor at Etisalat College of Engineering, United Arab Emirates. Previously, he worked as Senior Software Engineer at the School of Computer Science at Carleton University, and as Software Designer at Newbridge Networks. Qusay holds a B.Sc. in Data Analysis and a Masters degree in Computer Science, both from the University of New Brunswick, Canada. You can contact him at qmahmoud@ece.ac.ae.
Begin Listing X — NetClassLoader.java
import java.io.*;
import java.net.*;
import java.util.*;
/** * This class implements a custom network class loader.
* The class is capable of loading classes off the network.
* This class loader creates a cache of loaded classes so
* that classes that have been loaded before can just be
* fetched from the cache.
*/
public class NetClassLoader extends ClassLoader {
private Hashtable classes = new Hashtable();
public NetClassLoader() {
}
/**
* Loads a class with the specified name and return it.
*/
public class loadClass(String className)
throws ClassNotFoundException {
return (loadClass(className, true));
}
/**
* This method is called by the loadClass above.
*/
public synchronized Class loadClass(String className,
boolean resolveIt) throws ClassNotFoundException {
Class result;
byte classData[];
result = (Class) classes.get(className);
if (result == null) {
try {
result = findSystemClass(className);
if (result != null)
classes.put(className, result);
}
catch (Exception e) {
}
}
if (result == null) {
classData = null;
if (0 == className.indexOf("http://")) {
classData = loadnet(className);
}
if (classData != null) {
// This form of classData is deprecated. For
// security, the NetClassLoader should extract the
// real class name from className and pass it as
// the first argument to the new defineClass
// method.
result = defineClass(classData, 0,
classData.length);
if (resolveIt)
resolveClass(result);
if (result != null) {
classes.put(className, result);
}
}
}
return(result);
}
/**
* Loads a class with the specified name off the network.
*/
private byte[] loadnet(String name) {
URL url = null;
DataInputStream dis = null;
URLConnection urlc = null;
byte data[];
int filesize;
System.out.println("Loading " + name +
" from the network");
try {
url = new URL(name);
}
catch (MalformedURLException e) {
e.printStackTrace();
}
try {
urlc = url.openConnection();
dis = new DataInputStream(urlc.getInputStream());
}
catch (Exception e) {
e.printStackTrace();
}
filesize = urlc.getContentLength();
data = new byte[filesize];
try {
dis.readFully(data);
}
catch (IOException ioe) {
ioe.printStackTrace();
}
if (data == null)
System.out.println("Data = null");
return(data);
}
}End Listing X
Begin Listing XX — RunnerSecurityManager.java
import java.io.*;
import java.util.*;
/**
* A simple Custom Security Manager that protects against
* some malicious code to be executed by a class loader.
* In this Custom Security Manager, we disallow client's
* code (which will be loaded by our own Class Loader)
* from reading, writing, or deleting files. As well we
* quitting the JVM. Of course, the whole point of this
* is for demonstration only. There are other issues that
* should be implemented here; for example, client's code
* shouldn't be able to open socket connection to
* remote hosts.
*/
class RunnerSecurityManager extends SecurityManager {
private boolean silent = true;
private boolean checkRead = false;
private boolean checkWrite = false;
private boolean checkDelete = true;
private boolean checkExit = true;
private RandomAccessFile SecurityLog;
// Constructor.
RunnerSecurityManager() {
System.out.println("RunnerSecurityManager started");
try {
SecurityLog = new RandomAccessFile(
"security.log".replace('/',
File.separatorChar),"rw");
SecurityLog.writeChars(
"************************************\n");
SecurityLog.writeChars("SecurityLog started : " +
new Date().toString()+"\n");
SecurityLog.writeChars("************************\n");
}
catch (IOException ioe) {
System.out.println(
"RunnerSecurityManager: cannot open log file");
System.exit(0);
}
}
/**
* Writes an error message into the security log file
* @param message. The message @param classes If true
* the class that created the msg is printed.
*/
private synchronized void writeError(String error,
boolean classes) {
try {
SecurityLog.writeChars(
"** Security Error ** : " + error + "\n");
SecurityLog.writeChars(
"-- Occured on : " + new Date().toString() + "\n");
}
catch (IOException ioe) {
}
if (classes) {
Class[] arrClass;
arrClass=getClassContext();
try {
for (int i=2; i<arrClass.length; i++)
SecurityLog.writeChars(
" " + arrClass[i] + "\n");
}
catch (IOException ioe) {
}
}
}
/**
* Writes a message into the security log file @param
* message The message @param classes If true, the class
* that caused the msg is printed.
*/
private synchronized void writeMessage(String message,
boolean classes) {
try {
SecurityLog.writeChars("++ Message ++ : " +
message + "\n");
SecurityLog.writeChars("-- Occured on : " +
new Date().toString() + "\n");
}
catch (IOException ioe) {
}
if (classes) {
Class[] arrClass;
arrClass=getClassContext();
try {
for (int i=2; i<arrClass.length;i++)
SecurityLog.writeChars(
" " + arrClass[i] + "\n");
}
catch (IOException ioe) {
}
}
}
// The following operations are allowed. This is just
// hypothetical; more restricted access should be imposed
// especially when working with class loaders.
public void checkConnect(String host, int port) { };
public void checkCreateClassLoader() { };
public void checkAccess(Thread g) { };
public void checkListen(int port) { };
public void checkLink(String lib) { };
public void checkPropertyAccess(String key) { };
public void checkAccept(String host, int port) { };
public void checkAccess(ThreadGroup g) { };
public void checkExec(String cmd) { };
/**
* Check to see if a file with the specified file
* descriptor can be read.
*/
public void checkRead(FileDescriptor fd) {
if (checkRead) {
throw new SecurityException("Sorry, checkRead(" +
fd + ") not allowed");
}
if (!silent)
System.out.println("RunnerSecurityMan FD=" + fd +
" : checkRead");
}
/**
* Check to see if a file with the specified
* filename can be read.
*/
public void checkRead(String file) {
if (checkRead) {
throw new SecurityException("Sorry, checkRead(" +
file + ") not allowed.");
}
if (!silent)
System.out.println("RunnerSecurityMan FILE=" + file +
" : checkRead");
}
/**
* Check to see if a file with the specified file
* descriptor can be altered.
*/
public void checkWrite(FileDescriptor fd) {
if (checkWrite) {
throw new SecurityException(
"Sorry, not allowed to write " + fd);
}
else if (!silent)
System.out.println("RunnerSecurityMan FD=" + fd +
" : checkWrite");
}
/**
* Check to see if a file with the specified filename
* can be altered.
*/
public void checkWrite(String file) {
if (checkWrite) {
throw new SecurityException(
"Sorry, not allowed to write " + file);
}
else if (!silent)
System.out.println("RunnerSecurityManager FILE=" +
file + " : checkWrite");
}
/**
* Check to see if a file with the specified
* filename can be deleted.
*/
public void checkDelete(String file) {
if (checkDelete) {
writeError("checkDelete -> file=" + file, false);
throw new SecurityException(
"Sorry, not allowed to delete " + file);
}
else if (!silent)
System.out.println("RunnerSecurityManager FILE=" +
file + " : checkDelete");
}
/**
* Check to see if the system has exited the JVM.
*/
public void checkExit(int status) {
if (checkExit) {
writeError("checkExit -> status=" + status, true);
throw new SecurityException("Sorry, checkExit " +
status);
}
else if (!silent)
System.out.println("RunnerSecurityManager STATUS=" +
status + " : checkExit");
}
}End Listing XX
Begin Listing XXX — Compute Engine.java
import java.io.*;
import java.net.*;
import java.util.*;
/**
* This class implements a multi-threading compute engine.
* The compute engine has the ability to load classes off
* the network when instructed to do so.
*/
public class ComputeEngine extends Thread {
private static final int EXEC_PORT = 4444;
protected ServerSocket compute;
public ComputeEngine() {
try {
compute = new ServerSocket(EXEC_PORT);
}
catch (IOException ioe) {
System.out.println(
"Error in creating server socket: " + ioe);
} System.out.println(
"ComputeEngine listening on port: " + EXEC_PORT);
this.start();
}
/**
* Accepts a new connection in a separate thread.
*/
public void run() {
try {
while (true) {
Socket client = compute.accept();
Connection cc = new Connection(client);
}
}
catch (IOException e) {
System.out.println(
"Error listening for connection: " + e);
}
}
/** * This is the main method; execution starts here. Set
* our custom security manager and create an instance
* of the compute engine.
*/ public static void main(String argv[]) {
RunnerSecurityManager csm = null;
try {
csm = new RunnerSecurityManager();
}
catch (SecurityException se) {
System.out.println(
"RunnerSecurityManager is already running");
}
new ComputeEngine();
}
class Connection extends Thread {
Socket client;
DataInputStream is;
PrintStream os;
public Connection(Socket s) {
client = s;
try {
is = new DataInputStream(client.getInputStream());
os = new PrintStream(client.getOutputStream());
}
catch (IOException e) {
e.printStackTrace();
} this.start();
} public void run() {
String url = null;
try {
url = is.readUTF();
}
catch (IOException ex) {
ex.printStackTrace();
}
String className = url;
// Redirect the output stream to the client socket.
System.setOut(os);
// Use a class loader.
NetClassLoader ncs = new NetClassLoader();
try {
Object o;
o = (ncs.loadClass(className, true)).newInstance();
((RemoteCompute) o).run();
}
catch (Exception e) {
e.printStackTrace();
}
// Close the io streams and the connection to client.
try {
is.close();
os.close();
client.close();
}
catch (IOException ioe) {
ioe.printStackTrace();
}
}
}
}End Listing XXX
Begin Listing XXXX — client.java
import java.io.*;
import java.net.*;
/** * This class implements a simple client to be used with
* the Global Compute Engine. The client is simply run by
* the user of the system to load classes by the compute
* engine.
*/
public class client {
public final static int REMOTE_PORT = 4444;
// Even better: put the port number in a separate class
// that can be imported by the server and the client, to
// avoid this kind of mistake.
/**
* This is the main program of the client. @param host
* The host on which the Compute Engine is running on
* @param url The url of the class to be loaded.
*/
public static void main(String argv[]) throws Exception {
String host = argv[0];
String url = argv[1];
Socket cl = null, cl2=null;
BufferedReader is = null;
DataOutputStream os = null;
// Open connection to the compute engine on port 5555.
try {
cl = new Socket(host,REMOTE_PORT);
is = new BufferedReader(
new InputStreamReader(cl.getInputStream()));
os = new DataOutputStream(cl.getOutputStream());
System.out.println("Connection is fine...");
}
catch(UnknownHostException e1) {
System.out.println("Unknown Host: " + e1);
}
catch (IOException e2) {
System.out.println("Erorr io: " + e2);
}
// Write the url to the compute engine.
try {
os.writeUTF(url);
}
catch (IOException ex) {
System.out.println(
"error writing to server..." + ex);
}
// Receive results from the compute engine.
String outline;
try {
while((outline = is.readLine()) != null) {
System.out.println("Remote: " + outline);
}
}
catch (IOException cx) {
System.out.println("Error: " + cx);
}
// Close input stream, output stream and connection.
try {
is.close();
os.close();
cl.close();
}
catch (IOException x) {
System.out.println("Error writing..." + x);
}
}
}End Listing XXXX