0

I am porting an old mathematical model (between 1995 to 2000) to a current linux machine. For this, I adapted all makefiles as shown:

FORTRAN  = gfortran # f90 -f77 -ftrap=%none
OPTS     = -O -u -lgfortran -g -fconvert="big-endian" # -O -u
NOOPT    = 
LOADER   = gfortran #f90
LOADOPTS = #-lf77compat

and:

SYSFFLAGS   = -O0 -u -g -fconvert="big-endian" # -f77=input
SYSCFLAGS   = -DX_WCHAR
SYSLDFLAGS  =  
SYSCPPFLAGS = -DSYS_UNIX -DCODE_ASCII -DCODE_IEEE # -DSYS_Sun
SYSAUTODBL  = -fdefault-real-8 #-r8
SYSDEBUG    = -g
SYSCHECK    = -C 
LINKOPT =
CPPOPT  = 

SHELL       = /bin/sh
CC      = cc

FC      = gfortran # f90
LD      = gfortran # f90
AR      = ar vru
RM      = rm -f
CP      = cp
MV      = mv -f
LN      = ln -s

So I replaced all the outdated compilers/options to be able to compile the code. After that, It generates the binaries with no error. Note that all option behind the # symbol where the original options in the Makefiles.

So, when running the program, it is not possible to read the sample data. IMO those files were created as unformatted sequence mode on a Sun machine. The following hex dump belongs to the file that I need to read.

0000000: 0000 0400 2020 2020 2020 2020 2020 2020  ....
0000010: 3930 3130 7465 7374 2d63 3031 2020 2020  9010test-c01
0000020: 2020 2020 4741 5520 2020 2020 2020 2020      GAU
0000030: 2020 2020 2020 2020 2020 2020 2020 2020
0000040: 2020 2020 2020 2020 2020 2020 2020 2020
0000050: 2020 2020 2020 2020 2020 2020 2020 2020
...
...
0000390: 2020 2020 2020 2020 2020 2020 2020 2020
00003a0: 2020 2020 2020 2020 2020 2020 2020 2020
00003b0: 2020 2020 3139 3936 3037 3232 2032 3030      19960722 200
00003c0: 3434 3920 4147 434d 352e 3420 2020 2020  449 AGCM5.4
00003d0: 2020 2020 3230 3030 3036 3134 2031 3230      20000614 120
00003e0: 3831 3720 6869 726f 2020 2020 2020 2020  817 hiro
00003f0: 2020 2020 2020 2020 2020 2020 2020 2034                 4
0000400: 3039 3630 0000 0400 0002 8000 bef7 21f3  0960..........!.
0000410: bf3c 55ab bf7a 8f71 bf99 e26a bfb2 db4e  .<U..z.q...j...N
0000420: bfc7 425f bfd6 64b1 bfdf d44f bfe3 6a43  ..B_..d....O..jC

After analyzing the code, it is able to read until the line 0000410. It is not possible to continue after the mark 0002 8000. The source code shown below is actually reading this file.

...
*   [INPUT] 
INTEGER    IFILE
CHARACTER  HITEM  *(*)                    !! name for identify
CHARACTER  HDFMT  *(*)                    !! data format
*
*   [ENTRY INPUT] 
REAL * 8   TIME1                          !! time
REAL * 8   TIME2                          !! time
REAL*8     DMIN
REAL*8     DMAX
REAL*8     DIVS
REAL*8     DIVL
INTEGER    ISTYPE
INTEGER    JFILE                          !! output file No.
INTEGER    IMAXD
INTEGER    JMAXD
*
*   [WORK] 
REAL * 8   DDATA ( NGDWRK )
REAL * 4   SDATA ( NGDWRK )
*
*   [INTERNAL WORK] 
INTEGER    I, J, K, IJK, IJKNUM, IERR
...
...
READ ( IFILE, IOSTAT=IEOD ) HEAD
...
...
...
DO 2150 IJK = 1, IJKNUM              
    READ ( IFILE, END=2150 ) SDATA(IJK)
    WRITE (6,*) ' IGTIO::GTZZRD: iteration=', IJK, SDATA(IJK)
2150 CONTINUE

In order to easily debug the loop I replaced for the one above. The original one is implicit. 

READ ( IFILE, IOSTAT=IEOD)
&              (SDATA(IJK), IJK=1, IJKNUM )

And the output for the loop is:

IGTIO::GTZZRD: iteration=           1 -0.48268089    
IGTIO::GTZZRD: iteration=           2  1.35631564E-19
IGTIO::GTZZRD: iteration=           3 -0.48142704    
IGTIO::GTZZRD: iteration=           4  1.35631564E-19
IGTIO::GTZZRD: iteration=           5   244.25270    
IGTIO::GTZZRD: iteration=           6  1.35631564E-19
IGTIO::GTZZRD: iteration=           7   983.87988    
IGTIO::GTZZRD: iteration=           8  1.35631564E-19
IGTIO::GTZZRD: iteration=           9  1.59284362E-04
IGTIO::GTZZRD: iteration=          10  1.35631564E-19
IGTIO::GTZZRD: iteration=          11   0.0000000  
---error here---

I am definately lost on this, so any help is appreciated.

rgrun
  • 178
  • 3
  • 14
  • Iteration 2, `1.35631564E-19` still looks to be endian incorrect to me. As in still big on a little. Couple of ideas to explore. Are you able to `ntohl` those iterations. What is the size of SDATA, `real*4`, can you try real*8? Just ideas, good luck. – Timothy Brown Jan 14 '14 at 17:21
  • Do you know what the output is supposed to be? Is the initial output correct? The Sun might have used big-endian, while the Linux, if on Intel, will be little-endian. For a file in native format (Fortran unformatted), what matters more is hardware rather than the OS. Converting endian should be enough for integers. gfortran has an option for endian conversion: http://gcc.gnu.org/onlinedocs/gfortran/CONVERT-specifier.html. It might not be enough for floating point numbers, which might have additional differences in representation. – M. S. B. Jan 14 '14 at 17:21
  • @francescalus HEAD length is 1024 (64 * 16) – rgrun Jan 14 '14 at 17:44
  • As a really horrible hack, what happens if you (keeping a backup) remove the `0000 0400` from the start and `0000 0400 0002 8000` from `0000404` and use `access='stream'` on the open. [Yes, I'm guessing hopelessly, but that final part which could be a record header could also be a special Studio-ism, and there look to be plausible real*4 numbers after that part.] Is at least `-0.48268089` a reasonable input? – francescalus Jan 14 '14 at 17:56
  • It is a hack, but it may be necessary. Sequential unformatted files are not portable. – Vladimir F Героям слава Jan 14 '14 at 17:58
  • I would try `f77` from the Solaris Studio for Linux. It may be compatible with the solaris version. http://www.oracle.com/technetwork/server-storage/solarisstudio/downloads/index-jsp-141149.html – Vladimir F Героям слава Jan 14 '14 at 18:00
  • You may have to change the endianess of the data: Solaris is big endian. – cup Jan 14 '14 at 20:17
  • @francescalus As you suggested, I deleted those marks as well as moved access to 'stream'. Definatelly, the program has a different behaviour. But I am not sure the effect on other parts of the code. – rgrun Jan 15 '14 at 11:13
  • If you are reading with stream access now you need to add code to process (at least skip) the headers. Basically read one integer*4 first, then two integers between each record. – agentp Jan 15 '14 at 13:52

3 Answers3

2

Here is whats going on - first this This is definitely a Big Enfian file. The first 4 bytes

00000400

are the big end 4 byte integer 1024, which is the length of your first record. which agrees with the length of HEAD (per comment) Now note that 00000400 is repeated at byte position 1024+4 exactly (hex dump line 400) as you should expect for a fortran unformatted file...so far so good.

Now the next 4 bytes

0002 8000

begin the second record. (edit correcting mistake) This is 163840 (2*16^4+8*16^3) You should find that repeated at position 1024+8+163840+4 in the hex dump. (should be line 028400 i think..)

Here is the problem: in your code you are reading that 160 kilobyte record into a single 4 byte variable, then moving on to the next record. My guess you are seeing that alternating 10^-19 because every other record is of type character.

In unformatted fortran you must read a whole record in one shot - try reading the entire array (without the loop..)

READ ( IFILE )SDATA

assuming sdata is dimensioned to hold 160 kb of course. (eg. real*4 (40960) )

agentp
  • 6,849
  • 2
  • 19
  • 37
  • ijknum value is 40960. SDATA value is 279040. Are you sure 00028000 is 10240? I think it is 163840 ;) As you suggested, this change is internally doing 21 reads of 8192 bytes each for READ statement (strace confirmation). After that, it raises "Fortran runtime error: I/O past end of record on unformatted file". – rgrun Jan 15 '14 at 09:47
  • you are right, i droped a zero. the point remains you need to read the whole record with a single read statement – agentp Jan 15 '14 at 13:04
  • Sorry, I mislead you with because I change the implicit loop with "what I though" it was an equivalent. @francescalus pointed me in the right direction. Even though, your solution led me to the solution. – rgrun Jan 15 '14 at 16:05
  • Also, @george suggestion's should actually work except for the size of SDATA is bigger than the expected data. So his option, in this case, cause to raise an exception. For this reason, the programmer used an implicit loop. In order to control how many bytes read. – rgrun Jan 15 '14 at 17:20
1

The answer to your problem is in the edit that I missed. Thanks also to george's arithmetic---which I hadn't bothered to do.

We can safely say that the record headers are correct, and that you can solve your problem with the endian conversion.

So, the problem is: reads with an implied-do loop are not equivalent to reads inside a do loop.

That is: read(unit) (i(j), j=1,5) is not the same as

do j=1,5
  read(unit) i(j)
end do

In the first, reading of the five values is from one record, in the second each is read from a distinct record.

You should, then, revert your change. If you want to do the same diagnostics, however, you can do something like

READ ( IFILE, IOSTAT=IEOD) (SDATA(IJK), IJK=1, IJKNUM )
WRITE (6, '("IGTIO::GTZZRD: iteration='", I0.0, F12.8)') (IJK, SDATA(IJK), IJK=1,IJKNUM)
francescalus
  • 30,576
  • 16
  • 61
  • 96
  • You are right! I placed it back and then it worked after commenting some code out of this scope. – rgrun Jan 15 '14 at 16:06
0

Out of this scope, for some reason a deep method in charge of reading the file is called in the 1100 loop. That was causing to read the file more times than necessary. Find below the fixed code:

* 1100 CONTINUE 
     CALL   GDREDX         !! read data
 O         ( GDATA , IEOD  ,
 O           HITEMD, HTITL , HUNIT , HDSET , 
 O           TIME  , TDUR  , KLEVS ,
 I           IFILE , HITEM , HDFMT ,
 I           IMAXD , JMAXD , 
 I           IDIMD , JDIMD , KDIMD           )
     IF ( IEOD .EQ. 0  ) THEN
         WRITE (6,*) ' IRWGD.F::GDRDTS: TSEL0=', TSEL0
         WRITE (6,*) ' IRWGD.F::GDRDTS: TSEL1=', TSEL1
         WRITE (6,*) ' IRWGD.F::GDRDTS: TIME=', TIME
*            IF (    ((TSEL0.GE.0).AND.(TIME.LT.TSEL0)) 
*     &          .OR.((TSEL1.GE.0).AND.(TIME.GT.TSEL1)) ) THEN
*                     GOTO 1100
*            ENDIF
        ENDIF
*
  RETURN
  END

That mislead me in order to figure out what was going on.

rgrun
  • 178
  • 3
  • 14