0

Objective: The moment multiple.csv files are uploaded to the folder, code should check each filename, if appropriate filename, file should be further used by sqlloader to get data uploaded in the database. Once file is uploaded, code should delete the file processed. Next time, same process repeats.

I have some parts of the code working but some are creating problem, especially related to inotifywait. Please help.

In first loop, I am trying to monitor the /uploads folder, the moment it finds the .csv file, it checks if the filename has space. If yes, it wants to change the space to underscore in the filename. I have been trying to find a way to find "space, () or ," in the filename but only could do the 'space' part change. This is giving me an error that file cannot be moved, no such file or directory.

Second loop works separately but not when incorporated with first loop as there are errors which I have not been able to debug. If I run second loop separately, it is working correctly. But if there is a way to optimize the code better in one loop, I would be happy to know. Thanks!

Example: folder name: /../../upload

filenames: abc_123.csv (code should not make any change) , pqr(12 Apr).csv (code should change it to pqr_12_Apr.csv), May 12.csv (code should change it to May_12.csv) etc. Once these 3 files have proper naming, it should be ready to be uploaded through sql loader and once files are processed, they get deleted.
My code is: 

#!bin/bash
inotifywait -mqe create /../../upload | while read file; do
    if [[ $file = '* *'.csv]]; then
        mv "$file" ${file// /_}
    fi
done

for file in /../..upload/*.csv
do
       sqlcommand="sqlldr user/pwd control="/../xxx.ctl" data=$file silent=feedback, header"
       $sqlcommand
       rm $file
done

Thank you!

iotaa
  • 15
  • 3
  • What happens If you quote `${file// /_}` ? Also you need a space between `.csv` and `]]` – Jetchisel May 12 '20 at 20:51
  • @Jetchisel : I tried it, it still says "no such file or directory". I did echo $file before starting the if loop and it prints CREATE_/../../upload. Hence its unable to process it. – iotaa May 12 '20 at 21:05

1 Answers1

0

I have modified your script to this,

#!/usr/bin/env bash

while IFS= read -r file; do
  filename=${file#* CREATE }
  pathname=${file%/*}
  if [[ $pathname/$filename = *\ *.csv ]]; then
    echo mv -v "$pathname/$filename" "$pathname/${filename// /_}"
  fi
done < <(inotifywait -mqe create /../../upload)
  • Remove the echo if you think the output is correct.
  • I just don't know how you can integrate the other parts of your script with that, probably create a separate script or remove the -m (which you don't want to do most probably). Well you could use a named pipe if mkfifo is available.

EDIT: as per OP's message add another parameter expansion for another string removal.

Add the code below the if [[ ... ]]; then

newfilename=${filename//\(\)}

Then change "${filename// /_}" to "${newfilename// /_}"

Jetchisel
  • 7,493
  • 2
  • 19
  • 18
  • Thank you, it worked well for the files with spaces in it. Additionally For files with () I want to remove them too. Example, abc(May 12).csv should be abcMay_12.csv. I am able to remove () by using ${file//[\(\) ]} but now im trying to replace space with underscore to achieve the output. – iotaa May 13 '20 at 18:55
  • Do them separate, use another parameter expansion. one for the space and one for the brackets, – Jetchisel May 13 '20 at 19:41
  • Yes that's what I ended up doing. Thank you so much! – iotaa May 13 '20 at 19:52