awk to store and reset variable from file

Question

Trying to use awk to lookup the string in file1 (which is always just 1 field) in the same line of file2. That is if row 1 is being used in file1 then only row 1 is used in file2. Since it is possible for the value to be missing this is a check done to ensure it is there. This is just an idea so there probably is a better way, but I just wanted to see. Thank you :).

file1

R_2017_01_13_12_11_56_user_S5-00580-24-Medexome
R_2017_01_13_14_46_04_user_S5-00580-25-Medexome

file2

The oldest folder is R_2017_01_13_12_11_56_user_S5-00580-24-Medexome, created on 2017-01-17+11:31:02.5035483130 and analysis done using v1.4 by cmccabe at 01/17/17 12:41:03 PM

desired output for $filename

R_2017_01_13_12_11_56_user_S5-00580-24-Medexome

After a bunch of processes are run using $filename, I need to reset that variable with a new one.

file1 (after rerunning some process)

R_2017_01_13_12_11_56_user_S5-00580-24-Medexome
R_2017_01_13_14_46_04_user_S5-00580-25-Medexome

file2 (after rerunning some process)

The oldest folder is R_2017_01_13_12_11_56_user_S5-00580-24-Medexome, created on 2017-01-17+11:31:02.5035483130 and analysis done using v1.4 by cmccabe at 01/17/17 12:41:03 PM
The oldest folder is R_2017_01_13_14_46_04_user_S5-00580-25-Medexome, created on 2017-01-17+06:53:07.3194950000 and analysis done using v1.4 by cmccabe at 01/18/17 06:59:08 AM

desired output for $filename now is (since this is value is new)

R_2017_01_13_14_46_04_user_S5-00580-25-Medexome

awk

filename=$(awk 'NR==1{print $1}' file1 file2)

so you want to print the last item from file1 that occurs in the same line as in file2? — fedorqui, Jan 18 '17 at 14:15
Other way around I think, since `file1` can have multiple values in it, but `file2` will only have one until it is updated by the new process. Thank you :). — justaguy, Jan 18 '17 at 14:23

score 2 · Accepted Answer · edited May 23 '17 at 10:29

2

You want to check if the last line of file2 contains a string given in file1.

For this, you just have to read that last line and then see if it matches with any of the words in file1.

$ awk 'ENDFILE {line=$0} FNR<NR && line ~ $1' file2 file1
R_2017_01_13_14_46_04_user_S5-00580-25-Medexome

This uses:

ENDFILE {line=$0}
after reading a file, $0 contains the last line that was read (well, not always, but I assume you have a modern version of awk, since ENDFILE is a GNU awk extension). With this, we store this last line into line, so that we can use it when reading the next file.
FNR<NR && line ~ $1
while reading the file1, check if the given word is present in the stored line. If so, print is automatically triggered.
This uses the FNR<NR trick, where FNR holds the number of the line in the current file, while NR the number of line in general. This way, FNR==NR is only true while reading the first file and FNR<NR from the second on.

edited May 23 '17 at 10:29

Community

1
1

answered Jan 18 '17 at 14:59

fedorqui

275,237
103
548
598

1

NIce [this is a filler]. – James Brown Jan 18 '17 at 15:03
how can this value be stored in `$filename` variable or if a file is created that stores this line can that be used by the subsequent scripts? Thank you :). – justaguy Jan 18 '17 at 15:27
1

@Chris of course, just say `filename=$(awk 'ENDFILE {line=$0} FNR – fedorqui Jan 18 '17 at 15:28
Thank you both very much :). – justaguy Jan 18 '17 at 15:36

score 1 · Answer 2 · answered Jan 18 '17 at 14:57

If you only need to check the last line of file2 continuously, you could:

$ awk 'NR==FNR{a[$1];next}{for(i in a)if(i ~ $0) print i}' file1 <(tail -f file2)

Explained:

NR==FNR{a[$1];next} read into a array the search terms from file1
file2 is tail -f'd into awk using process substitution, ie. it reads a record from the end of file2, goes thru all search words in a and looks them from the record, printing search word if there is a match

awk to store and reset variable from file

2 Answers2