Bash script save csv in array and search for part of string

Question

after editing my script I would shortly like to explain what i want to do:

Check if files are in Folder
Look at begin of file name
search for file less than 1 hour old
take the file and do sqlldr ..if this succeeds move file to an other folder ...if not send mail

This is my Script, can someone please tell me if this is going to work? I am not sure about the syntax and also not sure if nr. 3 and 4. send mail works like this.

    #!/bin/sh

    #check if files are in folder
    declare -a arrCSV   #create array
    for file in *.csv
    do
    arrCSV=("${CSV[@]}" "$file")
    done

    shopt -s nullglob
    for file in read*.csv; do
    #run on all files starting with "read" and ending with ".csv" 
  for find $LOCATION -name $file -type f -mmin -60 do
    if
    sqlldr read*.csv 
then mv "$file" "$HOME/fail/" ;
else{ echo "Failed to load" | mail -s "FAIL" email@email.com}
done
    done

    for file in write*.csv; do
    #run on all files starting with "write" and ending with ".csv" 
      for find $LOCATION -name $file -type f -mmin -60 do
 if
sqlldr write*.csv 
then mv "$filen" "$HOME/unisem/fail/" ;
else { echo "Failed to load 2" | mail -s "FAIL" email@email.com}
done
    done

for file in *.csv will look for all .csv files in root of the directory where script is run, it will not look in subfolders or folders above, as for only finding files with starting with read and write, you don't need arrays, as mentioned in Socowi answer — ralz, Nov 22 '18 at 13:34
Shameless self plug, but I think you'll benefit from watching these videos on variable expansion and brace expansion: https://www.youtube.com/watch?v=yTijxqjZhRo and https://www.youtube.com/watch?v=82ESpisUh3Q — Conner, Nov 22 '18 at 13:36

score 1 · Accepted Answer · answered Nov 22 '18 at 13:32

1

You don't need an array if the read... and write... files can be processed in any order:

shopt -s nullglob
for file in read*.csv; do
    # run on all files starting with "read" and ending with ".csv" 
    sqldr ...
done
for file in write*.csv; do
    # run on all files starting with "write" and ending with ".csv" 
    sqldr ...
done

answered Nov 22 '18 at 13:32

Socowi

25,550
3
32
54

Thank you that helped a lot. Could you please also tell me how to insert an exception handling if the sqlldr doesnt work? – Swoop Dec 18 '18 at 08:22
@Swoop Use `sqldr ... || exceptionHandling`. The latter part will only be executed if `sqldr` fails. You could for instance use `{ echo "error while processing $file" >&2; exit 1; }`. – Socowi Dec 18 '18 at 08:31
so now mei scrip looks like this: for file in read*.csv; do sqlldr read*csv || { echo "error while processing $file" >&2; exit 1; } rm read*.csv done Will this work? I would like to delete the file after the sqlldr hase been done. And if the sqlld doesnt work just stop the sqlldr and dont delete the file – Swoop Dec 18 '18 at 08:42
@Swoop. Why don't you try it yourself? But no, it will probably not work. You did not use `"$file"` in your `sqlldr` command. To delete only on success use `if sqlldr ...; then rm "$file"; else echo ...; fi` but your control flow should work too, because the script exits on error before removing. – Socowi Dec 18 '18 at 09:06
I am just writing the script. i have no access to the unix. but thank you very much for helping.But what does $file exactly mean? And why would this work if i have sqlldr with read*csv and then rm $file Thank you!! – Swoop Dec 18 '18 at 09:34
@Swoop »*I'm a beginner*« seems a bit exaggerated when you don't even have access to a bash shell. Use bash on windows (WSL) or a VM to test your bash scripts. `file` is a variable that changes its content in each loop iteration. `"$file"` expands to the current content of that variable. – Socowi Dec 18 '18 at 11:07
yes i'll try to test it myself so thanks for the help first of all. but what if the csv is corrupt and the sqlldr doesnt work properly what do you suggest to do except for the echo command. is their any other or better opportunity to maybe stop the sqlldr or get back the data the sqlldr has loaded till it stopped. and what exactly does the >&2; exit 1 mean? – Swoop Dec 18 '18 at 12:39
There is no need to stop `sqlldr`. Bash executes the successor commands only after `sqlldr` stopped. `echo ... >&2` prints `...` to stderr. `exit 1` stops the whole script with error code `1`. *»get back the data the sqlldr has loaded till it stopped«* I'm not sure what you mean by that. I don't know anything about `sqlldr`. The path to the processed file would still be available as `$file`. – Socowi Dec 18 '18 at 12:58
So but if exit 1 stops the whole script the second for wont be looked at. That would be a problem because maybe there will be 2 files and theirfor all two for's and sqlldr should start, so would it be a problem to delete the exit 1 command ? – Swoop Dec 18 '18 at 15:03
"*Would it be a problem*" always depends on what exactly you want to do. I don't want to offend you, but I'm getting tired of this conversation. Learn the basics of bash and specify *exactly* what you want to do, then you might get more help. Please don't add any more comments. Edit your question if you know exactly what you need. – Socowi Dec 18 '18 at 16:34
i now changed and edited the question. I also tried it by myself. I hope you can give one last help on this. Thanks!! – Swoop Dec 19 '18 at 10:28

Bash script save csv in array and search for part of string

1 Answers1