Create 10000+ pages PDF with iText

Question

I need to generate really long PDF in iText (2.1.7), where pages are drawn by Graphics2D. The problem is, that each page requires ~2MB, so creating a 10.000 pages pdf needs ~20GB RAM. It there any possibility to flush a part of com.lowagie.Text.Document each ~100 pages on HDD? I have tried PdfWriter.flush() but it does not help.

Here is a little code. Running it with -Xmx200m results OutOfMemoryError.

import java.awt.Color;
import java.awt.Graphics2D;
import java.io.File;
import java.io.FileOutputStream;
import java.util.ArrayList;
import java.util.List;
import java.util.Random;

import com.lowagie.text.Document;
import com.lowagie.text.Rectangle;
import com.lowagie.text.pdf.PdfContentByte;
import com.lowagie.text.pdf.PdfTemplate;
import com.lowagie.text.pdf.PdfWriter;


public class Main {

    static String FILENAME = "/home/username/tmp/out.pdf";

    public static void main(String[] args) {
        try {
            System.out.println(printMem());

            Rectangle rectPage = new Rectangle(0, 0, 210, 297);

            FileOutputStream fos = new FileOutputStream(new File(FILENAME));
            Document d = new Document(rectPage, 0, 0, 0, 0);

            PdfWriter writer = PdfWriter.getInstance(d, fos);


            d.open();




            int pageNo = 200;

            for (int i = 0; i < pageNo; i++) {
                d.newPage();

                PdfContentByte cb;
                PdfTemplate tp;
                Graphics2D g2;


                cb = writer.getDirectContent();

                tp = cb.createTemplate(rectPage.getWidth(),
                        rectPage.getHeight());
                g2 = tp.createGraphicsShapes(rectPage.getWidth(),
                        rectPage.getHeight());

                paintRand(g2);

                g2.dispose();
                cb.addTemplate(tp, 0, 0);

                System.out.println(i + ") " + printMem());
                writer.flush();


            }


            d.close();

            System.out.println(printMem());
            System.out.println("done");






        } catch (Exception e) {
            System.err.println(e.getMessage());
            e.printStackTrace();
        } catch (OutOfMemoryError mem) {
            System.err.println("OUT OF MEMORY!!!!!!! " + printMem());
            mem.printStackTrace();
        }
    }






    private static String printMem() {
        final long usedMem = Runtime.getRuntime().totalMemory()
                - Runtime.getRuntime().freeMemory();
        String ram = "RAM: " + (usedMem / 1024 / 1024) + "MB / "
                + (Runtime.getRuntime().totalMemory() / 1024 / 1024) + "MB";
        if (Runtime.getRuntime().maxMemory() != Long.MAX_VALUE) {
            ram += " (max limit: " + Runtime.getRuntime().maxMemory() / 1024
                    / 1024 + "MB)";
        } else {
            ram += " (no max limit)";
        }

        return ram;
    }

    private static Random rand = new Random();

    private static List<Color> colors = new ArrayList<Color>()
    {
        {
            add(new Color(33, 33, 33));
            add(new Color(33, 33, 36));
            add(new Color(33, 33, 44));
            add(new Color(33, 33, 38));
        }
    };

    private static void paintRand(Graphics2D g2) {
        int count = 10000;

        for (int i = 0; i < count; i++) {
            g2.setColor(colors.get(rand.nextInt(colors.size())));
            g2.fillOval(rand.nextInt(210), rand.nextInt(210), rand.nextInt(55), rand.nextInt(55));
        }
    }
}

so i am getting

117) RAM: 239MB / 271MB (max limit: 271MB)
118) RAM: 246MB / 271MB (max limit: 271MB)
119) RAM: 244MB / 271MB (max limit: 271MB)
120) RAM: 245MB / 271MB (max limit: 271MB)
121) RAM: 247MB / 271MB (max limit: 271MB)
OUT OF MEMORY!!!!!!! RAM: 242MB / 271MB (max limit: 271MB)
java.lang.OutOfMemoryError: Java heap space
at com.lowagie.text.pdf.ByteBuffer.append_i(Unknown Source)
at com.lowagie.text.pdf.ByteBuffer.append(Unknown Source)
at com.lowagie.text.pdf.ByteBuffer.formatDouble(Unknown Source)
at com.lowagie.text.pdf.ByteBuffer.append(Unknown Source)
at com.lowagie.text.pdf.ByteBuffer.append(Unknown Source)
at com.lowagie.text.pdf.PdfContentByte.HelperRGB(Unknown Source)
at com.lowagie.text.pdf.PdfContentByte.setRGBColorFill(Unknown Source)
at com.lowagie.text.pdf.PdfContentByte.setColorFill(Unknown Source)
at com.lowagie.text.pdf.PdfGraphics2D.setPaint(Unknown Source)
at com.lowagie.text.pdf.PdfGraphics2D.setFillPaint(Unknown Source)
at com.lowagie.text.pdf.PdfGraphics2D.followPath(Unknown Source)
at com.lowagie.text.pdf.PdfGraphics2D.fill(Unknown Source)
at com.lowagie.text.pdf.PdfGraphics2D.fillOval(Unknown Source)
at Main.paintRand(Main.java:121)
at Main.main(Main.java:54)

Is there any option to write Document in peaces, like BufferedWriter does?

score 2 · Accepted Answer · edited May 23 '17 at 12:15

It is hard to understand why you would still use iText 2.1.7. The answer to the question https://stackoverflow.com/questions/25696851/can-itext-2-1-7-or-earlier-can-be-used-commercially is "No!" So please be a sport and upgrade.

As for your question: you should distinguish different problems.

Please check the answer to the following question: What is the size limit of pdf file when generated using c# code with images?

You will notice that iText 2.1.7 can not be used to create files with a size higher than 2GB. Knowing that each page requires 2MB and that you expect to have 10K pages, you should realize that you can not use iText 2.1.7 to achieve your goal.

You even try to make files that are larger than the limitation that is inherent to PDF 1.4, so you have to make sure that you use a compressed cross-reference table which is functionality that was introduced in PDF 1.5.

Also: if you have read the documentation, you should realize that iText flushes the content of each page to the OutputStream as soon as the page is ready. Hence your question "Is there any possibility to flush a part of Document each ~100 pages on HDD?" is strange.

Your diagnosis of the problem is wrong: you are creating a huge amount of PdfTemplate objects and you are keeping them in memory! You should use the releaseTemplate() method to write all the tp instances that are kept in memory inside the PdfWriter. That will free up plenty of memory.

But before you do that: please do the right thing and upgrade!

Create 10000+ pages PDF with iText

1 Answers1