Rewritten from scratch @ Friday, 25 May, about 16:00 GMT
(Code is cleaner now, bug can be reproduced and the question is more clear)
Original problem: I'm writing a server app that's required to accept files from clients over the net and process them with certain classes, which are loaded from locally stored .jar-files via URLClassLoader. Almost everything works correctly, but those jar-files are hot-swapped (without restarting the server app) from time to time to apply hotfixes, and if we're unlucky enough to update .jar-file at the same time class from it is being loaded, ClassFormatError is thrown, with remarks about "truncated class" or "excess bytes at the end". That's to be expected, but the whole application becomes unstable and starts to behave weird after that - those ClassFormatError exceptions keep happening when we try to load the class again from the same jar that was updated, even though we use new instance of URLClassLoader and it happens in different app thread.
The app is running and compiled on Debian Squeeze 6.0.3/Java 1.4.2, migration is not within my power.
Here's a simple code that mimics app behavior and roughly describes the problem:
1) Classes for main app and per-client threads:
package BugTest;
public class BugTest
{
//This is a stub of "client" class, which is created upon every connection in real app
public static class clientThread extends Thread
{
private JarLoader j = null;
public void run()
{
try
{
j = new JarLoader("1.jar","SamplePlugin.MyMyPlugin","SampleFileName");
j.start();
}
catch(Exception e)
{
e.printStackTrace();
}
}
}
//Main server thread; for test purposes we'll simply spawn new clients twice a second.
public static void main(String[] args)
{
BugTest bugTest = new BugTest();
long counter = 0;
while(counter < 500)
{
clientThread My = null;
try
{
System.out.print(counter+") "); counter++;
My = new clientThread();
My.start();
Thread.currentThread().sleep(500);
}
catch(Exception e)
{
e.printStackTrace();
}
}
}
}
2) JarLoader - a wrapper for loading classes from .jar, extends Thread. Here we load a class which implements a certain interface a:
package BugTest;
import JarPlugin.IJarPlugin;
import java.io.File;
import java.io.FileNotFoundException;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
public class JarLoader extends Thread
{
private String jarDirectory = "jar/";
private IJarPlugin Jar;
private String incomingFile = null;
public JarLoader(String JarFile, String JarClass, String File)
throws FileNotFoundException, MalformedURLException, ClassNotFoundException, InstantiationException, IllegalAccessException
{
File myjarfile = new File(jarDirectory);
myjarfile=new File(myjarfile,JarFile);
if (!myjarfile.exists())
throw new FileNotFoundException("Jar File Not Found!");
URLClassLoader ucl = new URLClassLoader(new URL[]{myjarfile.toURL()});
Class JarLoadedClass =ucl.loadClass(JarClass);
// ^^ The aforementioned ClassFormatError happens at that line ^^
Jar = (IJarPlugin) JarLoadedClass.newInstance();
this.setDaemon(false);
incomingFile = File
}
public void run()
{
Jar.SetLogFile("log-plug.txt");
Jar.StartPlugin("123",incomingFile);
}
}
3) IJarPlugin - a simple interface for pluggable .jars:
package JarPlugin;
public interface IJarPlugin
{
public void StartPlugin(String Id, String File);
public void SetLogFile(String LogFile);
}
4) the actual plugin(s):
package SamplePlugin;
import JarPlugin.IJarPlugin;
public class MyMyPlugin implements IJarPlugin
{
public void SetLogFile(String File)
{
System.out.print("This is the first plugin: ");
}
public void StartPlugin(String Id, String File)
{
System.out.println("SUCCESS!!! Id: "+Id+",File: "+File);
}
}
To reproduce the bug, we need to compile a few different .jars using same class name, whose only difference is number in "This is the Nth plugin: ". Then start the main application, and then rapidly replace the loaded plugin file named "1.jar" with other .jars, and back, mimicing the hotswap. Again, ClassFormatError is to be expected at some point, but it keeps happening even when the jar is completely copied (and is NOT corrupt in any way), effectively killing any client threads which try to load that file; the only way to get out from this cycle is to replace the plugin with another one. Seems really weird.
The actual cause:
It all became sort of clear once I simplified my code even more and got rid of clientThread class, simply instancing and starting the JarLoader inside the while loop in main. When ClassFormatError was thrown, it not just printed the stack trace out, but actually crashed the whole JVM (exit with code 1). The reason is not as obvious as it seems now (it wasn't for me, at least): ClassFormatError extends Error, not Exception. Hence it passes through catch(Exception E) and the JVM exits because of uncaught exception/error, BUT since I spawned thread which caused error from another spawned (client) thread, only that thread crashed. I guess it's because of the way Linux handles Java threads, but I'm not sure.
The (makeshift) solution:
Once uncaught error cause became clear, I tried to catch it inside the "clientThread". It sort of worked (I removed the stacktrace printout and printed my own message), but the main problem was still present: the ClassFormatError, even though caught properly, kept happening until I replace or remove the .jar in question. So I took a wild guess that some sort of caching might be a culprit, and forced URLClassLoader reference invalidation and Garbage Collection by adding this to clientThread try block:
catch(Error e)
{
System.out.println("Aw, an error happened.");
j=null;
System.gc();
}
Surprisingly, it seems to work! Now error only happens once, and then file class just loads normally, as it should. But since I just made an assumption, but not understood a real cause, I'm still worried - it works now, but there's no guarantee that it will work later, inside a much more complicated code.
So, could anyone with deeper understanding of Java enlighten me on what's the real cause, or at least try to give a direction? Maybe it's some known bug, or even expected behavior, but it's already way too complicated for me to understand on my own - I'm still a novice. And can I really rely on forcing GC?
