Loop over partially numbered files in Fortran

Question

I want to loop over filenames in a directory ./dir1/. The files in this directory partially follow a numbered pattern, for example

data001_abjdfsd.dat
data002_dchuwe.dat
data003_jnvkfi.dat

and so on. The first 8 characters follow a numbered pattern, the formats are all ".dat", and the rest of the string is arbitrary. Note that the string length in the filename is not fixed. I can probably do it easily in shell script using wildcards. How do I loop over these files in Fortran 90?

Have a look at the intrinsic `execute_command_line`. You do a listing of the files and redirect to a file. Just read the filenames from the freshly created file. — RussF, Aug 02 '17 at 07:53
Another option is to use wildcarlds at the command line to pass filenames via the command-line. You can use the commands `command_argument_count` and `get_command_argument` to retrieve those arguments programmatically. This is standard since Fortran 2003. See the docs of gfortran for instance https://gcc.gnu.org/onlinedocs/gfortran/COMMAND_005fARGUMENT_005fCOUNT.html#COMMAND_005fARGUMENT_005fCOUNT and https://gcc.gnu.org/onlinedocs/gfortran/GET_005fCOMMAND_005fARGUMENT.html#GET_005fCOMMAND_005fARGUMENT — Pierre de Buyl, Aug 02 '17 at 12:14
If you are using Intel Fortran, `GETFILEINFOQQ` from the `IFPORT` module can return file info for file names matching a specified pattern, and allows wildcards. — Matt P, Aug 02 '17 at 15:12

Scientist · Accepted Answer · 2017-08-03T14:51:20.627

Here is a complete platform-independent (but not idiot-proof!) solution to the problem (should work in Linux/Windows/Mac). Note that you will need Fortran 2008 compiler. The function getFileList(searchText,order,excludeText) in the following module will give you a list of files in the current directory (or the directory that is given in searchText). Note that searchText will be passed to the command line environment to seach for the files. So, wildcards could be used for file search. The two other arguments are optional: order determines in what order the files should be listed. Only two orders are supported in this function: name and date. The third argument excludeText is a text that you don't want to exist in any of the files to be listed by the function. On the output, the function returns a structure of type dynamicString that contains a list of all the requested files, in the order requested, each of which may have a different name length. I have enclosed a test file at the bottom of this response.

module ModFileList

  implicit none

  integer, parameter, private :: maxFileRecordLength = 2047

  type DynamicString
    character (len=:), allocatable :: record
  end type DynamicString

contains

!*********************************************************************
!*********************************************************************

  function getFileList(searchText,order,excludeText)

    implicit none
    character(len=*)   , intent(in)           :: searchText
    character(len=*)   , intent(in), optional :: order
    character(len=*)   , intent(in), optional :: excludeText
    type(DynamicString), allocatable          :: getFileList(:)
    character(len=:)   , allocatable          :: command,filename,orderLowerCase
    character(len=maxFileRecordLength)        :: record
    integer                                   :: iunit,counter,iostat,nRecord,nskip
    character(8)                              :: date
    character(10)                             :: time
    logical                                   :: exist

    if (present(order)) then
      orderLowerCase = getLowerCase(order)
    else
      orderLowerCase = 'name'
    end if

    if (getSlash()=='\') then  ! it's Windows cmd
      if (orderLowerCase=='name') then  ! ascending in name
        command = 'dir /b /a-d ' // searchText
      elseif (orderLowerCase=='date') then   ! oldest will be first
        command = 'dir /b /a-d /o:d ' // searchText
      else
        write(*,*) '    FATAL: In Misc@getFileList()'
        write(*,*) '           The requested file order is not supported.'
        write(*,*) '           order = ', orderLowerCase
        write(*,*) 'Program aborted.'
      end if
      if (present(excludeText)) then
        command = command // " | findstr /v /i " // trim(adjustl(excludeText))
      end if
    else
      if (orderLowerCase=='name') then  ! ascending in name
        command = 'ls -1 ' // searchText
      elseif (orderLowerCase=='date') then   ! oldest will be first
        command = 'ls -tr ' // searchText
      else
        write(*,*) '    FATAL: In Misc@getFileList()'
        write(*,*) '           The requested file order is not supported.'
        write(*,*) '           order = ', orderLowerCase
        write(*,*) 'Program aborted.'
      end if
      if (present(excludeText)) then
        command = command // " --ignore=" // trim(adjustl(excludeText))
      end if
    end if

    ! generate a brand new, non-existing filename
    counter = 0
    do
      counter = counter + 1
      call date_and_time(date,time)
      filename = date // '_' // time // '_' // 'getFileList_' // int2str(counter) // '.temp'
      inquire(file=filename,exist=exist)    ! check if the file already exists
      if (exist) cycle
      exit
    end do
    call execute_command_line(command//' > '//filename)

    nRecord = getNumRecordInFile(filename)

    ! check filename is not among records
    nskip = 0
    open(newunit=iunit,file=filename,status='old')
    do counter = 1,nRecord
      read(iunit,'(A)',iostat=iostat) record
      if(iostat==0) then
        if(filename==trim(adjustl(record))) nskip = nskip + 1
      else
        write(*,*) '    FATAL (1): In Misc@getFileList()'
        write(*,*) '               Error occurred while reading file.'
        write(*,*) 'Program aborted.'
        stop
      end if
    end do
    close(iunit)

    allocate(getFileList(nRecord-nskip))
    open(newunit=iunit,file=filename,status='old')
    do counter = 1,nRecord
      read(iunit,'(A)',iostat=iostat) record
      if(iostat==0) then
        if (filename/=trim(adjustl(record))) getFileList(counter)%record = trim(adjustl(record))
      else
        write(*,*) '    FATAL (2): In Misc@getFileList()'
        write(*,*) '               Error occurred while reading file.'
        write(*,*) 'Program aborted.'
        stop
      end if
    end do
    close(iunit)

    if (getSlash()=='\') then  ! it's Windows cmd
      command = 'del '//filename
    else
      command = 'rm '//filename
    end if
    call execute_command_line(command)

  end function getFileList

!*********************************************************************
!*********************************************************************

  pure function getLowerCase(string)
    implicit None
    character(*), intent(in) :: string
    character(len(string))   :: getLowerCase
    character(26), parameter :: lowerCase = 'abcdefghijklmnopqrstuvwxyz', upperCase = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'
    integer                  :: ic, i
    ! capitalize each letter if it is lowercase
    getLowerCase = string
    do i = 1, len(string)
        ic = INDEX(upperCase, string(i:i))
        if (ic > 0) getLowerCase(i:i) = lowerCase(ic:ic)
    end do
  end function getLowerCase

!*********************************************************************
!*********************************************************************

  function getNumRecordInFile(filename)
    implicit none
    character(len=*), intent(in) :: filename
    character(len=8)             :: record
    integer                      :: getNumRecordInFile,iunit,iostat
    open(newunit=iunit,file=filename,status='old')
    getNumRecordInFile = 0
    do
      read(iunit,'(A)',iostat=iostat) record
      if(iostat==0) then
        getNumRecordInFile = getNumRecordInFile + 1
        cycle
      elseif(iostat<0) then
        exit
      else
        write(*,*) 'FATAL error occurred reading file in Misc in getNumRecordInFile().'
        write(*,*) 'Program aborted.'
        stop
      end if
    end do
    close(iunit)
  end function getNumRecordInFile

!*********************************************************************
!*********************************************************************
  character(len=1) function getSlash()
    implicit none
    character(len=7) :: os
    call get_environment_variable('OS',os)
    if (os=='Windows') then
      getSlash = '\'
    else
      getSlash = '/'
    end if
  end function getSlash

!*********************************************************************
!*********************************************************************

  pure function int2str(integerIn,formatIn)
    implicit none
    integer     , intent(in)           :: integerIn
    character(*), intent(in), optional :: formatIn
    character(:), allocatable          :: int2str
    integer                            :: i,length
    character(len=63)                  :: thisFormat
    if (present(formatIn)) then
      write(thisFormat,formatIn) integerIn
      int2str = trim(adjustl(thisFormat))
    else
      do i=1,63
        if(abs(integerIn)<10**i) then
          length = i
          if (integerIn<0) length = length + 1
          exit
        end if
      end do
      allocate(character(length) :: int2str)
      write(thisFormat,'(1I63)') length
      thisFormat = '(1I' // trim(adjustl(thisFormat)) // ')'
      write(int2str,thisFormat) integerIn
    end if
  end function int2str

!*********************************************************************
!*********************************************************************  

end module ModFileList

Here is test file for the above module:

include 'mod_FileList'

program main

use ModFileList, only: getFileList, dynamicString, int2str

implicit none

type(dynamicString), allocatable :: FileList(:), SystemInfo(:)
character(63) :: record
character(len=127)            :: searchString
character(len=:), allocatable :: dir,fileName,fileBase,FileExt,test
logical                       :: exist
integer                       :: i
real*16                       :: dummy

searchString = '*.f90'
FileList = getFileList(searchString)

if (size(FileList)==0) then
    write(*,*) 'no file detected'
else
    write(*,*) int2str(size(FileList)), " files detected containing ", trim(adjustl(searchString)), " :"
    do i = 1,size(FileList)
    write(*,*) FileList(i)%record
    end do
end if

end program main

If you compile this test file and run it, the output should be something similar to the following:

2 files detected containing *.f90 :
main.f90
mod_FileList.f90

Are we hoping the person doesn't want to search for ";rm -rf *"? — francescalus, Aug 02 '17 at 17:36
Why do you include the `include 'mod_FileList'` line in the main program? I got an error without removing this line even after changing the included filename to "ModFileList", or is this part of the "not-idiot-proof" nature of your code? I use gfortran. — nougako, Aug 03 '17 at 00:58
I just tested the test file, it works perfectly fine with gfortran. remove include 'mod_FileList' from the test program and replace it by the module given above (insert the module straight on top of the test program. you can test it here: it works just fine: http://www.tutorialspoint.com/compile_fortran_online.php Note that you will need to change the compiler default options to support F2008 — Scientist, Aug 03 '17 at 14:38
the idiot-proof part refers to @francescalus important comment above. — Scientist, Aug 03 '17 at 14:40

Loop over partially numbered files in Fortran

1 Answers1