3

I am using jMagick for java to process uploaded files (images and pdf files). Most images work fine but when I tried to convert large multiple-page pdf files to different size of images, it consumes very large amount of system memory (about 20G physical memory for a 200M pdf files), and failed eventually. I am setting the density to 200 since otherwise the image quality is very bad that I cannot even read the words in the output images. Below is the code I have:

  ImageInfo info = new ImageInfo(inputFilePath);
  info.setDensity(200);
  MagickImage image = new MagickImage(info);
  MagickImage[] imageFrames = image.breakFrames();
  for (MagickImage frame : imageFrames) {
    ImageInfo frameInfo = new ImageInfo();
    MagickImage frameDisplay = frame;
    frameDisplay.setFileName(outputFile);
    frameDisplay.writeImage(frameInfo);
  }

There is no error or exception in my java log, I just see the process died. I have tried to use policy.xml in /usr/share/ImageMagick-6.5.4/config/ to limit the memory usage as follows:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE policymap [
<!ELEMENT policymap (policy)+>
<!ELEMENT policy (#PCDATA)>
<!ATTLIST policy domain (delegate|coder|filter|path|resource) #IMPLIED>
<!ATTLIST policy name CDATA #IMPLIED>
<!ATTLIST policy rights CDATA #IMPLIED>
<!ATTLIST policy pattern CDATA #IMPLIED>
<!ATTLIST policy value CDATA #IMPLIED>
]>

<policymap>
    <policy domain="resource" name="memory" value="256MB"/>
</policymap>

But somehow it does not seem to work. I am using ImageMagick-6.5.4. Really appreciate any advice!

Andrea
  • 11,801
  • 17
  • 65
  • 72
Roy Cheng
  • 31
  • 2
  • "Does not work" meaning .... – thatidiotguy Dec 14 '12 at 20:13
  • The JVM restarted, not any image saved from the pdf which needs to be converted – Roy Cheng Dec 14 '12 at 20:15
  • "The JVM restarted"? What??? Do you mean your program crashed? If so, stacktrace please – thatidiotguy Dec 14 '12 at 20:16
  • And please update your question with the information requested :). You provided a fine description of what you are doing but you are missing the information what goes wrong. – Rutix Dec 14 '12 at 20:31
  • Sure sorry for the confusion. I are using weblogic, so the only thing we have in the log is the weblogic node for this particular JVM got restarted. At the meantime, when I tried to just run the convert command from the command line with same 200m file, the process was killed by the OS after a few minutes without any error messages, while I can see the 30G free memory dropped to about 100M during that processing time. – Roy Cheng Dec 14 '12 at 20:36
  • Can't you adjust your code so ImageMagick only considers one page per time instead of the whole file at once ? – mmgp Dec 14 '12 at 21:48

1 Answers1

2
<policymap>
    <policy domain="resource" name="memory" value="256MB"/>
</policymap>

Try updating this section to:

<policymap>
    <policy domain="resource" name="memory" value="1000MB"/>
</policymap>

and try again. I'm presuming when you said 'The JVM restarted', you mean that it crashed. Probably an OutOfMemory exception. No one can say for sure with the information you've provided.

Amir Afghani
  • 37,814
  • 16
  • 84
  • 124
  • 1
    Thx, let me try that. I was thinking it should be JVM OutOfMemory exception as well, which is why I am setting the JVM as: -Xms1024m -Xmx2048m But that seems even make the problem worse. The java process still took 30G memory and event crashed the linux server. That is why I want to limit the physical memory use even it means the image will be in VM so maybe gonne be slow quite a bit. – Roy Cheng Dec 14 '12 at 21:00
  • @Amir my suspicion as well – thatidiotguy Dec 14 '12 at 21:32