Replacing a specific field using awk, sed or any POSIX tool

Question

I have a huge file1, which has values as follows:

a 1
b 2
c 3
d 4
e 5

I have another huge file2, which is colon delimited with seven fields as follows:

a:2543:2524:2542:252:536365:54654
c:5454:5454:654:54:87:54
d:87:65:1:98:32:87

I want to search the lines for the variables of file1 and replace its value in the 7th column in file2 so the output should be as follows:

a:2543:2524:2542:252:536365:1
c:5454:5454:654:54:87:3
d:87:65:1:98:32:4

What code/script have you tried? Is really the `file1` as simple as it seems? You do not expect us to write the code for you, do you? — Jdamian, Aug 16 '16 at 16:59
Replace with what? Is `file1` a mapping from `a` to `1` etc., and if so, why is `54654` replaced by `1`? — Benjamin W., Aug 16 '16 at 17:12

justaguy · Answer 1 · 2016-08-16T21:09:00.090

0

Maybe this awk will work, assuming the files are as posted

awk -F: 'NR==FNR{a[$1]=$0;next;}a[$1]{$0=a[$1]}1' file1 file2
a:2543:2524:2542:252:536365:1
c:5454:5454:654:54:87:3
d:87:65:1:98:32:4

edited Aug 16 '16 at 21:09

answered Aug 16 '16 at 21:01

justaguy

2,908
4
17
36

Unfortunately that didnt work for me, but I came up with a solution which I posted below. – Imran Rentia Aug 17 '16 at 20:49

Imran Rentia · Accepted Answer · 2016-08-17T21:04:13.563

0

So I came up with a solution; ended up with a couple of lines of code. Maybe there is a better way to do it.But this works !

while read line ; do 
  var1=`echo $line| awk '{print $1}'`
  var2=`echo $line| awk '{print $2}'`
  awk -v var1="$var1" -v var2="$var2" -F ':' 'BEGIN { OFS = ":"} $1==var1 {sub(".*",var2,$7)}{print}' file2 > file2.tmp
  mv file2.tmp file2
done < file1
cat file2

edited Aug 17 '16 at 21:04

answered Aug 17 '16 at 20:14

Imran Rentia

13
3

Much more efficient to replace `read line` with `read -r var1 var2 _`. (Of course, it would be better than *that* to have just one `awk` doing all the work, instead of one invocation per line, but that's a separate issue). – Charles Duffy Aug 21 '16 at 02:33

pakistanprogrammerclub · Answer 3 · 2016-08-21T02:34:21.903

0

This should work - it assumes both files are sorted on the first column - I'd be very interested in any performance comparisons with your solution for very large files - both speed and memory - --complement is linux-specific but easily replaced if not available

file0=$1 #file with single value
file1=$2 #file with 6th value to be replaced
# normalize on colon delimiter
tr ' ' : <$file0|
# join on first field
join -t: $file1 -|
# delete column 7
cut --complement -d: -f7

edited Aug 21 '16 at 02:34

answered Aug 21 '16 at 02:27

pakistanprogrammerclub

807
4
6

I'd consider quoting your expansions -- and you might consider using `<(sort "$file1")` and `< <(sort "$file0")` to guarantee the sort constraint. – Charles Duffy Aug 21 '16 at 02:35

Replacing a specific field using awk, sed or any POSIX tool

3 Answers3