0

I am trying to migrate a legacy application to current jee 7 standards.

During the application deployment/bootstrap, we initialize some caches through a startup servlet. Few caches are inturn used by other application components during the deployment process. These are initialized sequentially in the legacy application. I am trying to introduce multithreading to initialize all caches in parallel. So I am using ManagedExecutorService and submitting the tasks. After the tasks are submitted, the deployment is continuing for a while and stopping indefinitely. Below is my code

for(Cacheable cacheable: cacheableApps.values()) {
    mes.submit(new Runnable(){
        public void run() {
            cacheable.initialize();
        }
    });
}

I also tried using ManagedExecutorService.execute(Runnable) and also ManagedExecutorService.submit(Callable) with no luck.

Additionally, I tried calling ManagedExecutorService.invokeAll(Collection) and Future.get() but the server hung indefinitely at this point.

Then I replaced ManagedExecutorService with ManagedThreadFactory and the threads started initializing and the deployment completed with NPEs in other application components which uses the caches. This is because the caches are initializing at the end of the deployment.

The idea is to replace startup servlet with a startup singleton ejb, initialize my caches in multiple threads and halt the deployment process untill all threads return so that no NPEs are thrown and the deployment is finished smoothly.

While using ManagedThreadFactory, I tried halting the deployment like below

for(Cacheable cacheable: cacheableApps.values()) {
    RunnableTask task =new RunnableTask(cacheable);
    mtf.newThread(task).start();
    tasks.add(task);
}
int count = 0;
while(count!=tasks.size()) {
    count = 0;
    for(Task task: tasks) {
        if(task.isDone()) {
            count++;
        }
    }
}

public class RunnableTask implements Runnable {
    private boolean done;
    public(Cacheable cacheable) {
        this.cacheable = cacheable;
    }
    public void run() {
        cacheable.initialize();
        done = true;
    }
    public boolean isDone() {
        return done;
    }
}

But this code also hung up the server indefinitely. Because the threads are not started yet while the execution is in the while loop. The threads are only starting at the end of the deployment. So the above code is invalid.

To debug the problem, I created a dummy restful web service and added the below code

Future<String> future = mes.submit(new Callable<String>() {
    public String call() throws Exception {
        Cacheable.STUDENT.getApp().initialize();
        return "successful";
    }
});

return future.get();

This initialized the cache and the response of the web service is "successful".

Then I copied the exact code to my startup servlet and the deployment again hung while executing future.get()

I was wondering if there is some problem with the cache code and removed the call and simply returned "successful". Again the deployment hanged while calling future.get()

Krishna Chaitanya
  • 2,533
  • 4
  • 40
  • 74
  • I think I found out the issue. The same code is working fine in an ejb and not inside a startup servlet or servlet listener. – Krishna Chaitanya Apr 04 '17 at 09:33
  • I found out that the same code is working fine in a regular servlet or a restful web service when a http request is sent. Is it a bug in liberty or does the ee spec not allow the servlet container to access the concurrency features during initialization/deployment/bootstrap? – Krishna Chaitanya Apr 04 '17 at 09:42
  • then post this as answer so that others can utilize it. – Suncatcher Apr 04 '17 at 10:09

2 Answers2

1

It sounds like a possible bug. You should collect a dump of server threads for further diagnosis of the hang. When the server is hung, run the following command,

server dump <your-server-name>

When this completes, it will output the location of the dump file, for example,

wlp/usr/servers/your-server-name/your-server-name.dump-17.04.04_08.21.56.zip

Within the zip file, there will be a subfolder, such as, dump_17.04.04_08.21.56 with a text file called ThreadInfoIntrospector.txt

This file contains the stacks of all active threads in the server. Many of them may be idle. Post the stacks of any that are doing work (hung) for further analysis of the possible bug.

njr
  • 3,399
  • 9
  • 7
0

I think I found out the issue. The same code is working fine in an ejb and not inside a startup servlet or servlet listener.

Also, I found out that the same code is working fine in a regular servlet or a restful web service when a http request is sent. Is it a bug in liberty or does the ee spec not allow the servlet container to access the concurrency features during initialization/deployment/bootstrap?

Krishna Chaitanya
  • 2,533
  • 4
  • 40
  • 74