4

I am having over 20s 5000x30 double arrays and each of which will be written to text files using:

PrintWriter test = new PrintWriter(new BufferedWriter(new FileWriter("test.txt")));

The processing took me over 10 minutes and I'd like to see if there is any alternative way to speed up the process.

danielbeard
  • 9,120
  • 3
  • 44
  • 58
user1696229
  • 41
  • 1
  • 2
  • 1
    Out of curiosity, have you actually profiled your code to make sure that this is where your bottleneck is? – Brian Sep 25 '12 at 05:25
  • the profile shows that the buffer holds a large chunk of memory. as for execution time, I only used simple approach by removing and adding printwriter parts and see the difference. – user1696229 Sep 25 '12 at 06:03
  • Fair enough. To be sure, you could always use CPU profiling, but if the difference is significant enough after removing the I/O for you to notice, it's probably your problem :) Most IDEs offer CPU profiling for situations not so black and white. Just searching "Java CPU Profiling [_your IDE name_]" will help. Cheers! – Brian Sep 25 '12 at 06:08

3 Answers3

3

Instead of dealing with everything over numerous buffer, avoid flushing the buffer until absolutely necessary.

In practice this means, don't use println printf flush format or any other method that would flush the buffer. By avoiding this you delay and combine costly system calls that eat up your runtime.

Alternatively set autoFlush to false in the constructor for PrintWriter. Check out this question for some more info

Community
  • 1
  • 1
daniel gratzer
  • 52,833
  • 11
  • 94
  • 134
  • Thanks. I need to use println to print double array to file. is there any other way not to use println? by the way, thanks for autoFlush suggestion. It improves a lot. – user1696229 Sep 25 '12 at 06:05
  • Do you actually need to use it? Simply adding "\n" will add a new line to a regular print statement – daniel gratzer Sep 25 '12 at 10:11
0

Set the autoFlush variable of the PrintWriter to false.

PrintWriter test = new PrintWriter(new BufferedWriter(
                                   new FileWriter("test.txt")), **false**);

Once entire writing ( println ) is done, Call test.flush()

This way you can avoid intermediate flushing time.

Munesh
  • 1,509
  • 3
  • 20
  • 46
0

I would recommend using java nio for this. They are inherently faster than traditional java io. Please refer to these (1,2) examples to start with.

Santosh
  • 17,667
  • 4
  • 54
  • 79