Jetty server has created content in temp directory that have consumed 47g disk space

Question

Our Web runs on the Jetty server in production. In one of the scenarios, we had an outage in the production, which was fixed. The outage did happen 2-3 times.

Now, after some time we observed jetty server has created war logs that have consumed 47g disk space and this has happened only on one of the VMs. We are not doing this in our code. Please note we have 2 war files that are deployed. Attaching a screenshot, hopefully, can help.

Any input or suggestion - on why would jetty create war logs files that are not needed or not wanted by the application occupying so much of the space will be a great help ! Thanks you in advance

Attaching the handler code for war file:

private HandlerCollection getWebAppHandlers() throws SQLException, NamingException{
  
    // Setting service war
        WebAppContext serviceWebapp = new WebAppContext();
        serviceWebapp.setWar(API_WAR_FILE_PATH);
        serviceWebapp.setContextPath(API_CONTEXT_PATH);
    
        //setting the war and context path for the UI layer: oaxui
        WebAppContext uiWebapp = new WebAppContext();
        uiWebapp.setWar(UI_WAR_FILE_PATH);
        uiWebapp.setContextPath(UI_CONTEXT_PATH);
        uiWebapp.setAllowNullPathInfo(true);
        uiWebapp.setInitParameter("org.eclipse.jetty.servlet.Default.dirAllowed", "false");
    
        //set error page handler for the UI context
        uiWebapp.setErrorHandler(new CustomErrorHandler());
    
        //handling the multiple war files using HandlerCollection.
        HandlerCollection handlerCollection = new HandlerCollection();
        handlerCollection.setHandlers(new Handler[]{serviceWebapp, uiWebapp});
        return handlerCollection;
    }

Joakim Erdfelt · Accepted Answer · 2022-03-21T14:25:38.890

Those are not log files.

They appear to be working directory / temp directory paths. Probably in your System temp directory.

Using the syntax ...

String defaultWorkingDirectoryName = 
  "jetty-" 
    + host + "-" 
    + port + "-" 
    + resourceBase + "-_" 
    + context + "-" 
    + virtualhost + "-" 
    + randomdigits;

The past answer on setting the temp directory for a webapp.

Jetty: How to change startup temp directory

For embedded Jetty, do this.

your uiWebapp and serviceWebapp should both have a temp directory specified, they should be unique to each other.

// MYBASEDIR comes from your application config
// suggest that it be MYBASEDIR=/var/run/myapp
uiWebapp.setTempDirectory(new File(MYBASEDIR + "/uiWebapp-temp-dir"));
serviceWebapp.setTempDirectory(new File(MYBASEDIR + "/serviceWebapp-temp-dir"));

You might also want to set the WebAppContext.setPersistTempDirectory(boolean) depending on your requirements.

That will delete the temp directory between restarts. (both on proper webapp shutdown, and again on webapp startup).

ALSO ...

Since this is a unix environment, you should have a system startup that cleans out the system temp directory of old files.

This can be done with various built in tooling that performs a periodic scan of the temp directory.

But be careful with long running apps, like your webapp and Jetty. Having the actively used directory be cleaned out from under it is a bad thing for stability.

It's usually a good idea with Embedded Jetty on Linux to be precise about your Jetty temp directories, not using the default, and put that working directory in the /var/run/<app> directory tree.

Thank you Joakim. When does this directory gets created, I assume at the docker/server startup. Do they must be created, I mean is it mandatory? Also what are we expecting in the method - getConfiguredBaseTempDir()? Is it an inbuilt method ? ? Please suggest, why working directory are needed and what is the ideal to cleanup ? — Harish Dalmia, Mar 21 '22 at 13:39
`getConfiguredBaseTempDir()` is just a stub-mock method, indicating that you have a configuration somewhere, and that it returns a Java `Path` object that you base your webapp specific directories from. - the temp/work directory is mandatory on the servlet spec, and will be used on every webapp deployment. — Joakim Erdfelt, Mar 21 '22 at 14:25

Jetty server has created content in temp directory that have consumed 47g disk space

1 Answers1