Mysqldump launched by tomcat cause OutOfMemoryError if tomcat uses datasource

Question

for some reason i've now decided to change my webapp connection method from simple managed connection to a pooled DataSource. All works fine, except for mysqldump that results in an OutOfMemoryError during the iteration on the BufferedReader. As you can see by the code pasted here, i try to make the dump by running an external process... so i can't figure out why this exception shows up.

can anyone help me?

/**
     * esegue il comando mysqldump e inserisce il risultato in un array di byte che poi verrÃ  restituito sotto forma di stringa
     * @return
     * @throws Exception
     */
    private String getData() throws Exception {

        Process run = null;
        InputStream inErr = null;
        BufferedReader brErr = null;
        StringBuffer tempErr = null;
        InputStream in = null;
        BufferedReader br = null;
        StringBuffer temp = null;

        try {
            int count;
            char[] cbuf = new char[BUFFER];

            // esegue il backup con mysqldump
            run = Runtime.getRuntime().exec("mysqldump --host=" + host + " --port=" + port + " --user=" + user + " --password=" + password + " " + db);

            // prima gestisce un eventuale errore
            inErr = run.getErrorStream();
            brErr = new BufferedReader(new InputStreamReader(inErr));
            tempErr = new StringBuffer();

            if (inErr.available() != 0) {
                while ((count = brErr.read(cbuf, 0, BUFFER)) != -1)
                    tempErr.append(cbuf, 0, count);
            }

            brErr.close();
            inErr.close();

            // scrive il testo dell'errore in un file
            if (tempErr.toString().length() != 0) {
                // scrive l'errore nel file di log
                scriviErrInLog(tempErr.toString());
                throw new Exception("Errore nel backup del database");
            }

            // ora gestisce un
            in = run.getInputStream();
            br = new BufferedReader(new InputStreamReader(in));
            temp = new StringBuffer();

            //System.out.println("creato input stream --- in.available() =  " + in.available());

            if ((count = br.read(cbuf, 0, BUFFER)) != -1) { 
                //System.out.println("sto per scrivere nel file");
                temp.append(cbuf, 0, count);

                br.mark(BUFFER);
                br.reset();

                while ((count = br.read(cbuf, 0, BUFFER)) != -1) {
                    temp.append(cbuf, 0, count);
                }

                //System.out.println("scritto");

                br.close();
                in.close();
                run.destroy();
            } else
                throw new BackupException("Errore nel backup del database: backup vuoto");

        } catch (Exception e) {

            br.close();
            in.close();

            brErr.close();
            inErr.close();

            run.destroy();

            throw e;
        }

        return temp.toString();
    }

thanks

ott 15, 2013 3:51:08 PM org.apache.tomcat.jdbc.pool.ConnectionPool abandon

WARNING: Connection has been abandoned PooledConnection[com.mysql.jdbc.JDBC4Connection@1c3e12b]:java.lang.Exception at org.apache.tomcat.jdbc.pool.ConnectionPool.getThreadDump(ConnectionPool.java:976) at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:730) at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:586) at org.apache.tomcat.jdbc.pool.ConnectionPool.getConnection(ConnectionPool.java:174) at org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:124) at qbnuovo.dao.DBManager.openConnection(DBManager.java:174) at qbnuovo.dao.DBManager.exeQuery(DBManager.java:60) at qbnuovo.dao.TurnoDAO.selectPianoLavoro(TurnoDAO.java:2116) at qbnuovo.dao.TurnoDAO.selectPianoLavoro(TurnoDAO.java:2072) at qbnuovo.service.TurnoService.selectPianoLavoro(TurnoService.java:577) at qbnuovo.command.VisualizzaConsuntivoCommand.execute(VisualizzaConsuntivoCommand.java:294) at qbnuovo.web.RequestProcessor.perform(RequestProcessor.java:49) at qbnuovo.web.EventController.processRequest(EventController.java:144) at qbnuovo.web.EventController.doGet(EventController.java:184) at javax.servlet.http.HttpServlet.service(HttpServlet.java:621) at javax.servlet.http.HttpServlet.service(HttpServlet.java:722) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:928) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:987) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:539) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:298) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)

MySqlBackup: prima di getData Exception in thread "http-bio-8080-exec-1" java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Unknown Source) at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source) at java.lang.AbstractStringBuilder.ensureCapacityInternal(Unknown Source) at java.lang.AbstractStringBuilder.append(Unknown Source) at java.lang.StringBuffer.append(Unknown Source) at qbnuovo.helper.MysqlBackup.getData(MysqlBackup.java:351) at qbnuovo.helper.MysqlBackup.eseguiBackup(MysqlBackup.java:272) at qbnuovo.dao.DBManager.eseguiBackupInterno(DBManager.java:710) at qbnuovo.dao.DBManager.eseguiBackupDatabase(DBManager.java:461) at qbnuovo.command.CambioDataCommand.execute(CambioDataCommand.java:124) at qbnuovo.web.RequestProcessor.perform(RequestProcessor.java:49) at qbnuovo.web.EventController.processRequest(EventController.java:144) at qbnuovo.web.EventController.doPost(EventController.java:193) at javax.servlet.http.HttpServlet.service(HttpServlet.java:641) at javax.servlet.http.HttpServlet.service(HttpServlet.java:722) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:928) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:987) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:539) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:300) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)

can u show logs for better understanding ? – Ashish Oct 15 '13 at 13:35 — Ashish, Oct 15 '13 at 13:35

score 0 · Answer 1 · answered Oct 15 '13 at 13:57

0

The problem is that you try to keep the whole mysql dump in memory, which is bound to end with that kind of error. It doesn't have anything to do with the pooled DataSource. Try to process the dump line by line instead, as you read from the input stream.

answered Oct 15 '13 at 13:57

coyote

169
4

do you mean to try to write line by line to the dump file instead of adding line by line to the StringBuffer? – firegloves Oct 15 '13 at 14:01
Yes that could be a solution. But if that's all you need to do, you can redirect the outpout when you run mysqldump, using > dumpfile.sql – coyote Oct 15 '13 at 14:17
that is a good idea, but executing this command with redirection i obtain an error that says to me that > is not a database table...?! – firegloves Oct 15 '13 at 15:00
the problem here is that runtime is not a shell, so it can't recognize redirection http://stackoverflow.com/questions/12380054/using-redirection-operators-with-java-runtime-exec – firegloves Oct 15 '13 at 15:19

Mysqldump launched by tomcat cause OutOfMemoryError if tomcat uses datasource

1 Answers1