12

I maintain a JDBC driver that also has an embedded database server mode provided through a native library, which is accessed through JNA. The shutdown process, done as part of unloading the native library itself, runs into problems on Windows due to the order of unloading its dependencies. To avoid access violations or other issues, I need to explicitly shut down the embedded engine before unloading this library.

Due to the nature of its use, it is difficult to determine an appropriate moment to call for a shutdown. The only correct way to do this for a normal Java application is by registering a shutdown hook using

Runtime.getRuntime().addShutdownHook with a subclass of Thread that implements the shutdown logic.

This works fine for a normal Java application, but for web applications that include my library as part of the application (in the WEB-INF/lib of the WAR), this will cause a memory leak on undeploy as the shutdown hook will maintain a strong reference to my shutdown implementation and to the classloader of the web application.

What would be a suitable and appropriate way to address this? Options I'm looking into right now are:

  • Using java.sql.DriverAction.deregister() to do the cleanup.

    Not suitable as a driver will not be deregistered on a normal application exit.

  • Using java.sql.DriverAction.deregister() to remove the shutdown hook and execute the shutdown logic itself.

    Use of DriverAction is slightly problematic given the driver still supports Java 7, and this class was introduced in JDBC 4.2 (Java 8). This is technically not always the correct use of action (a JDBC driver can also be deregistered while existing connections remain valid and in use), and it is possible that the driver is used (through a javax.sql.DataSource) while the JDBC java.sql.Driver implementation is not registered.

  • Including a javax.servlet.ServletContextListener implementation annotated with @WebListener with the driver that will remove the shutdown hook and execute the shutdown logic itself.

    This option has complications if the driver is deployed to the server as a whole instead of to a specific web application (although those complications can be solved).

Is there a shutdown mechanism in Java I have overlooked that could be suitable for my needs?

Mark Rotteveel
  • 100,966
  • 191
  • 140
  • 197
  • 1
    Could you please explain what you mean by the driver being deployed to the server as a whole? I could not find anything about it, and I'm not aware of such functionality (at least not in Tomcat). I currently use the `@WebListener`-based solution in Tomcat to deregister a few `java.sql.Driver`s (using `DriverManager.deregisterDriver`) in a few containers (I always deregister exactly the driver that got registered by given container by storing `Class extends java.sql.Driver>`), and I'm wondering whether I haven't overlooked something. – Tomasz Linkowski Aug 14 '18 at 19:50
  • @TomaszLinkowski You can deploy a driver so the driver is globally available (possibly used as a server-wide data source). For example in Tomcat if you put it in `/lib` and define a data source in server.xml. Similar features exist in other application servers. Manually registering/deregistering the driver won't work for those situations (nor will this work for non-web applications), and I don't think it will solve my problem if the driver is globally available even if registered per WAR (`java.sql.Driver` implementations are (or should be) pretty light-weight). – Mark Rotteveel Aug 15 '18 at 07:41
  • @TomaszLinkowski re: light-weight: a driver itself doesn't actually 'hold' much, so the rest of the implementation could be shared, so deregistering my native library in that situations will probably be a bad idea (or require some additional reflection magic). Given the lack of responses so far, it looks like there is no mechanism that I can apply in all situations, so I will have to find some hybrid approach. – Mark Rotteveel Aug 15 '18 at 07:52
  • Thanks for the explanation :) Well, I understand you need to do more than I do (I just deregister the drivers so that Tomcat doesn't complain upon redeploys). As far as I understand, you want to shutdown the embedded engine *only if* no drivers are registered anymore, right? So e.g. if there's *no* global driver, and you have two containers, and each registered its driver, then if one container is destroyed, you do *not* want to shutdown the embedded engine until the second one is destroyed, right? And if there *is* a global engine, you want to shutdown only on app server shutdown, is that so? – Tomasz Linkowski Aug 15 '18 at 11:39
  • @TomaszLinkowski That sounds about right, except for the 'no global driver' situation, the embedded engine will need to be shutdown on container destruction, as the embedded engine in that case will be loaded per container. – Mark Rotteveel Aug 15 '18 at 12:20

1 Answers1

2

I tried to figure this out as this seems such an interesting case. I'm posting my findings here, although I feel I might still have misunderstood something, or made some too far-fetched simplifications. Actually, it's also possible that I totally misunderstood your case, and this answer is all useless (if so, I apologize).

What I assembled here is based on two notions:

  • application server global state (I use System.props, but it might not be the best choice - perhaps some temporary files would do better)
  • container-specific global state (which means all classes loaded by the container-specific ClassLoader)

I propose an EmbeddedEngineHandler.loadEmbeddedEngineIfNeeded method that would be called:

  • during your driver registration
  • in your javax.sql.DataSource implementation static initializer (if this whole DataSource-related thing works that way - I know little about it)

If I got it right, you won't need to call Runtime.removeShutdownHook at all.

The main thing that I'm uncertain about here is this - if the driver is deployed globally, would it be registered before any servlet is initialized? If not, then I got it wrong, and this won't work. But maybe inspecting the ClassLoader of EmbeddedEngineHandler could help then?


This is the EmbeddedEngineHandler:

final class EmbeddedEngineHandler {

    private static final String PREFIX = ""; // some ID for your library here
    private static final String IS_SERVLET_CONTEXT = PREFIX + "-is-servlet-context";
    private static final String GLOBAL_ENGINE_LOADED = PREFIX + "-global-engine-loaded";

    private static final String TRUE = "true";

    private static volatile boolean localEngineLoaded = false;

    // LOADING
    static void loadEmbeddedEngineIfNeeded() {
        if (isServletContext()) {
            // handles only engine per container case
            loadEmbeddedEngineInLocalContextIfNeeded();
        } else {
            // handles both normal Java application & global driver cases
            loadEmbeddedEngineInGlobalContextIfNeeded();
        }

    }

    private static void loadEmbeddedEngineInLocalContextIfNeeded() {
        if (!isGlobalEngineLoaded() && !isLocalEngineLoaded()) { // will not load if we have a global driver
            loadEmbeddedEngine();
            markLocalEngineAsLoaded();
        }
    }

    private static void loadEmbeddedEngineInGlobalContextIfNeeded() {
        if (!isGlobalEngineLoaded()) {
            loadEmbeddedEngine();
            markGlobalEngineAsLoaded();
            Runtime.getRuntime().addShutdownHook(new Thread(EmbeddedEngineHandler::unloadEmbeddedEngine));
        }
    }

    private static void loadEmbeddedEngine() {
    }

    static void unloadEmbeddedEngine() {
    }

    // SERVLET CONTEXT (state shared between containers)
    private static boolean isServletContext() {
        return TRUE.equals(System.getProperty(IS_SERVLET_CONTEXT));
    }

    static void markAsServletContext() {
        System.setProperty(IS_SERVLET_CONTEXT, TRUE);
    }

    // GLOBAL ENGINE (state shared between containers)
    private static boolean isGlobalEngineLoaded() {
        return TRUE.equals(System.getProperty(GLOBAL_ENGINE_LOADED));
    }

    private static void markGlobalEngineAsLoaded() {
        System.setProperty(GLOBAL_ENGINE_LOADED, TRUE);
    }

    // LOCAL ENGINE (container-specific state)
    static boolean isLocalEngineLoaded() {
        return localEngineLoaded;
    }

    private static void markLocalEngineAsLoaded() {
        localEngineLoaded = true;
    }
}

and this is the ServletContextListener:

@WebListener
final class YourServletContextListener implements ServletContextListener {

    @Override
    public void contextInitialized(ServletContextEvent sce) {
        EmbeddedEngineHandler.markAsServletContext();
    }

    @Override
    public void contextDestroyed(ServletContextEvent sce) {
        if (EmbeddedEngineHandler.isLocalEngineLoaded()) {
            EmbeddedEngineHandler.unloadEmbeddedEngine();
        }
    }
}
Tomasz Linkowski
  • 4,386
  • 23
  • 38
  • 1
    This looks like a promising solution, thanks. I can see some possible pain points to implement this, but I think this may work. – Mark Rotteveel Aug 16 '18 at 08:17
  • 1
    I took a while to get to this, but I have finally implemented it: https://github.com/FirebirdSQL/jaybird/commit/7e65ee051cbfb8d179777095c4a55f214c32a298 Thanks again for your help. – Mark Rotteveel Apr 20 '19 at 09:11
  • 1
    @MarkRotteveel I'm glad you managed to solve the problem. I see it's a bit different to what I proposed (e.g. you use `removeShutdownHook`, and you compare `ClassLoader`s to determine if the driver was loaded in a servlet context or not), but I trust you had good reasons for such an approach :) – Tomasz Linkowski Apr 20 '19 at 16:05
  • 1
    Yes, I modified your approach to solve issues when the driver is on the main classpath, but only first accessed from within the servlet context. With your approach that would lead to issues if the driver was used in multiple contexts, and one was stopped or redeployed. My current solution works in Tomcat, although I still have to test it in other application servers (checking the classloaders might be too naive). – Mark Rotteveel Apr 20 '19 at 16:14
  • I see, that's what I was afraid might be the case when I wrote: "if the driver is deployed globally, would it be registered before any servlet is initialized?". Nice that you managed to work around it! – Tomasz Linkowski Apr 20 '19 at 19:29
  • 1
    Yes, it took some trial and error to get it to work, but your answer gave me a good starting point. And in just in case I messed things up (or didn't consider certain cases), I also defined a system property that disables this cleanup/shutdown feature entirely. – Mark Rotteveel Apr 20 '19 at 20:05