Writing to file in Fortran do loop

Question

I am just starting to learn fortran and I have made a simple program, implementing Eulers method. I noticed that the time used by the do loop is fairly similar to, for example, the same program in Matlab.

In my Fortran program, I had a write to a file inside this do loop. When I removed it the speed drastically improved, as expected. Then I added a second do loop, where I place the writing and measured the time required by both loops combined:

open(unit=1, file='data.dat')
write(1,*) t(1),' ', y(1)
! Calculating
call cpu_time(start)
do i = 2,n
    t(i) = t(i-1) + dt
    y(i) = y(i-1) + dt*(t(i-1)*(y(i-1)))
    !write(1,*) t(i),' ', y(i)
end do
call cpu_time(finish)
print  '("Time = ",f7.5," seconds.")',finish-start
call cpu_time(start)
do i = 2,n
    write(1,*) t(i),' ',y(i)
end do
 call cpu_time(finish)
print  '("Time = ",f7.5," seconds.")',finish-start

The time for a given n would take approximately 10-15 times longer on the last do loop compared to the first one.

So, my question is, is there a better way to write my data to a file?

You could write all t at once and all y at once. However, that would still be fairly slow. The formatting is quite expensive. You get faster by unformatted IO, but then you can not read the file as a textfile anymore. — haraldkl, Sep 25 '15 at 08:46
How are you compiling? Certain optimization settings will change the way the `data.dat` file is buffered which may dramatically improve the speed. — Ross, Sep 25 '15 at 09:43
Two more things: 1) Beware of using `CPU_TIME` because it may hide some time spent doing io, see http://stackoverflow.com/questions/6878246/fortran-intrinsic-timing-routines-which-is-better-cpu-time-or-system-clock . I found `gfortran` gave a 30% difference between `CPU_TIME` and `SYSTEM_CLOCK` for a test case. — Ross, Sep 25 '15 at 10:02
2) I wouldn't worry too much about the time taken for something like this. When you later have a problem with a real amount of data you should migrate your restart file to unformatted or hdf5 or something. For your toy case I think you should just take the minor IO hit. — Ross, Sep 25 '15 at 10:06

Ian Bush · Answer 1 · 2015-09-25T11:58:22.590

Assuming t and y are real and depending on the quality of implementation by the compiler, the size of the arrays, the phase of the moon and the local neutron flux something akin to the following might be quicker

ian-standard@barleybarber ~
$ cat t.f90
Program test

  ! Corrected program

  Implicit None

  Real, Dimension( 1:10 ) :: t, y

  Integer :: i

  Call Random_number( t )
  Call Random_number( y )

  Do i = 1, 3
     Write( *, * ) t( i ), y( i )
  End Do
  Write( *, * )

  Write( *, '( f0.9, 1x, f0.9 )' ) ( t( i ), y( i ), i = 1, 3 )

  Open( 1, file = 'stuff.dat' )
  Write( 1, '( f0.9, 1x, f0.9 )' ) ( t( i ), y( i ), i = 1, &
       Min( Size( t ), Size( y ) ) )

End Program test

ian-standard@barleybarber ~
$ gfortran -O -Wall -Wextra -pedantic -std=f95 t.f90 -o test

ian-standard@barleybarber ~
$ ./test
  0.997559547      0.217951715    
  0.566824675      0.133160353    
  0.965915322      0.900524497    

.997559547 .217951715
.566824675 .133160353
.965915322 .900524497

ian-standard@barleybarber ~
$ head stuff.dat 
.997559547 .217951715
.566824675 .133160353
.965915322 .900524497
.747927666 .386765957
.367390871 .445482254
.480636895 .661932170
.073754251 .016108274
.005355179 .650854826
.347081244 .646408796
.342243791 .322987258

as the compiler might be clever enough to turn this all into one I/O transaction instead of many. But you can't be sure. Also to give it a better chance if you don't need all those decimal places adjust the format appropriately.

You don't need to waste the memory with the extra array - though I admit in history that would help compilers and I/O libraries optimise the operation. Nowadays my suspicion is clever OS buffering of I/O makes it all irrelevant, but stuff like the above might help. — Ian Bush, Sep 25 '15 at 12:00
Almost totally irrelevant: I suggest referring to the *neutrino* flux (neutrons decay with a halflife of 15 minutes) ;) — Andras Deak -- Слава Україні, Sep 25 '15 at 12:12
It does look like this is faster for big values of n, approx 0.6 - 0.7 of the do loop — ninjacowgirl, Sep 28 '15 at 07:57

score 1 · Answer 2 · answered Sep 25 '15 at 14:24

for completeness I'll post the 2d array copy approach:

integer,dimension(10)::x,y
integer,allocatable::tmp(:,:)
integer i
x=[(i, i=0,9)]
y=x+42
allocate(tmp(2,size(x)))
tmp(1,:)=x
tmp(2,:)=y
write(*,'(i0,'','',i0)')tmp
deallocate(tmp)
end

0,42
1,43
2,44
3,45
4,46
5,47
6,48
7,49
8,50
9,51

If performance is really critical I'd try both.

Note the best approach of all may be to simply use a 2d array (or a derived type) in the first place.

Writing to file in Fortran do loop

2 Answers2