0

I am trying to read the mails from the MS Exchange by using camel and getting the attachments as DataHandler. A 10MB file takes around 3hrs to write into the location.

File outputFile = new File(someDirectory, someFileName);
DataHandler attachment_data = destination1Attachments.get("someFileName.txt"); 

        try (FileOutputStream fos = new FileOutputStream(outputFile)) {
            attachment_data.writeTo(fos);  
        }

I have also noticed that sometimes a 6 to 7Mb file takes around 2 to 3 minutes and when another mail comes just after that it takes more time than expected.

Because of GC ?

Trying to find the exact root cause or any other method to write the data to the file.

Update 1 :
Tried using BufferedOutputStream around FileOutputSteam as mentioned by @user207421 in the comment. No much change could find (just 1sec or little more).

smilyface
  • 5,021
  • 8
  • 41
  • 57
  • 2
    Have you tried using any profiling tool to find out what exactly takes too much time? – Pavel Smirnov Sep 17 '19 at 09:12
  • No Pavel. We are thinking to implement one. Will update once I have done that. – smilyface Sep 17 '19 at 09:14
  • 2
    You don’t need to implement a profiling tool. Just *use* an existing one. – Holger Sep 17 '19 at 10:01
  • Oh okey. @holger do you have a suggestion ? – smilyface Sep 17 '19 at 10:23
  • 1
    Add a `BufferedOutputStream` around the `FileOutputStream`. – user207421 Sep 17 '19 at 10:49
  • [visualvm](https://visualvm.github.io/) for example. Up to Java 8, it is even integrated into the JDK. If you have a commercial JDK build from Oracle, there’s the [flightrecorder](https://docs.oracle.com/javacomponents/jmc-5-4/jfr-runtime-guide/about.htm#JFRUH170) included. Then, there are [jprofiler](https://www.ej-technologies.com/products/jprofiler/overview.html) and [eclipse mat](https://www.eclipse.org/mat/)… – Holger Sep 17 '19 at 10:49
  • Tried using `BufferedOutputStream` around `FileOutputSteam` as mentioned by @user207421 . No much change could find (just 1sec or little more). – smilyface Sep 18 '19 at 10:13

2 Answers2

1

This could be due to the default implementation of write mechanism.

attachment_data.writeTo(fos);  

If the DataHandler.getDataSource()!=null then this theory will work

In this method implementation 8 bytes are getting read at a time and writing into the stream. The number of read and writes are more and this might be causing the issue.

Try reading the on your own from DataHandler.getInputStream and write to file by increasing the read content from the input stream.

1

One must assume that the object is loaded in memory or writeTo very inefficient. Hence specify the DataFlavor and inspect attachment_data.getTransferDataFlavors().

DataFlavor flavor = new DataFlavor(InputStream.class, "application/octetstream");
try (InputStream in = (InputStream) attachment_data.getTransferData(flavor)) {

Some fiddling needed.

Joop Eggen
  • 107,315
  • 7
  • 83
  • 138