Bash Shellscript Column Check Error Handling

Question

I am writing a Bash Shellscript. I need to check a file for if $value1 contains $value2. $value1 is the column number (1, 4, 5 as an example) and $value2 ($value2 can be '03', '04' , '09' etc) is the String I am looking for. If the column contains the $value2 then perform a move of the file to an error directory. I was wondering what is the best approach to this. I was thinking awk or is there another way?

$value1 and $value2 are stored in a config file. I have control over what format I can use. Here's an example. The file separator is Octal \036. I just depicted with | below.

Example

$value1=5
$value2=04

Input example1.txt

 example|42|udajha|llama|04
 example|22|udajha|llama|02

Input example2.txt

example|22|udajha|llama|02

Result move example1.txt to /home/user/error_directory and example2.txt stays in current directory (nothing happens)

JNevill · Accepted Answer · 2018-09-18T20:42:09.187

2

awk can report out which files meet this condition:

awk -F"|" -v columnToSearch=$value1 -v valueToFind=$value2 '$columnToSearch==valueToFind{print FILENAME}' example1.txt example2.txt

Then you can do your mv based on that.

Example using a pipe to xargs (with smaller variable names since you get the idea by now):

awk -F"|" -v c=$value1 -v v=$value2 '$c==v{print FILENAME}' example1.txt example2.txt | xargs -I{} mv -i {} /home/user/error_directory

edited Sep 18 '18 at 20:42

answered Sep 18 '18 at 20:36

JNevill

46,980
4
38
63

Your solution with xargs works perfectly. Was wondering if you had any ideas on how to log the files with errors to a error_log.txt while using xargs? – Defcon Sep 24 '18 at 20:39
Without putting a lot of thought into it, I think you could stick a pipe to `tee` in the middle there like `awk | tee -a error_log.txt | xargs ` `tee` is cool because it redirects stdin to a file and ALSO to stdout so you can use it again downstream. – JNevill Sep 24 '18 at 20:42
I see that is a very good solution. I have this function called log_error that takes 4 parameters. I wanted to use the incorrect file names from awk as the 4th parameter. I was wondering if that could be done with your solution? – Defcon Sep 25 '18 at 19:29

score 0 · Answer 2 · answered Sep 18 '18 at 21:15

If you're writing a bash shell script then you can break it down by column using cut. There are really so many options that it depends on what you want to get done.

In my experience with data I'd use a colon rather than pipe because it allows me to avoid the escape with the 'cut' command.

Changing the data files to:

cat example1.txt
example:42:udajha:llama:04
example:22:udajha:llama:02

I'd write it like this: (adding -x so that you can see the processing, but in your code you'd not need to do that.)

[root@]# cat mysript.sh
    #!/bin/sh -x
one=`cat example1.txt | cut -d: -f5`
two=`cat example2.txt | cut -d: -f5`

for i in $one
do
if [ $i -eq $two ]
     then
      movethis=`grep $two example1.txt`
      echo $movethis >> /home/me/error.txt
fi
done

cat /home/me/error.txt
[root@]# ./mysript.sh
++ cat example1.txt
++ cut -d: -f5
+ one='04
02 '
++ cat example2.txt
++ cut -d: -f5
+ two=02
+ for i in '$one'
+ '[' 04 -eq 02 ']'
+ for i in '$one'
+ '[' 02 -eq 02 ']'
++ grep 02 example1.txt
+ movethis='example:22:udajha:llama:02 '
+ echo example:22:udajha:llama:02
+ cat /home/me/error.txt
example:22:udajha:llama:02

You can use any command you live to move your content. Touch, cp, mv, what ever you want to use there.

This contains all of the most common shell mistakes: UUOC, deprecated backticks, `for` looping on file names, and unquoted variables. — Ed Morton, Sep 18 '18 at 21:54
You can use cut on any character: `cut -d'|' -f5` will work as well. — Nic3500, Sep 19 '18 at 15:44

Bash Shellscript Column Check Error Handling

2 Answers2