Shell Scripting -awk option help required

Question

The script should read each file in the path and replace the string in each single row.How to create temp file and mv replace while i am iterating 10 diff input files name in the same path Pls advice

SunOS 5.10

FILES=/export/home/*.txt
for f in $FILES
do
echo "Processing $f file..."
cat $f | awk 'BEGIN {FS="|"; OFS="|"} {$8=substr($8, 1, 6)"XXXXXXXXXX\""; print}'
done

input file

"2013-04-30"|"X"|"0000628"|"15000231"|"1999-12-05"|"ST"|"2455525445552000"|"1111-11-11"|75.00|"XXE11111"|"224425" 
"2013-04-30"|"Y"|"0000928"|"95000232"|"1999-12-05"|"VT"|"2455525445552000"|"1111-11-11"|95.00|"VVE11111"|"224425"

output file

"2013-04-30"|"X"|"0000628"|"15000231"|"1999-12-05"|"ST"|"245552xxxxxxxxxx"|"1111-11-11"|75.00|"XXE11111"|"224425" 
"2013-04-30"|"Y"|"0000928"|"95000232"|"1999-12-05"|"VT"|"245552xxxxxxxxxx"|"1111-11-11"|95.00|"VVE11111"|"224425"

Not sure how use this

cat $f | awk 'BEGIN {FS="|"; OFS="|"} {$8=substr($8, 1, 6)"XXXXXXXXXX\""; print}' $f > tmp.txt && mv tmp.txt $f

When you do `FILES=/export/home/*.txt`, you're actually storing the glob pattern itself, not a list of filenames. It's better to either not do that at all, or use a shell where you can store the result as an array. — Charles Duffy, Feb 14 '14 at 16:33

score 0 · Answer 1 · edited May 23 '17 at 11:49

0

To achieve what appears to be your desired end result you can use ed instead of awk.

FILES=/export/home/*.txt
for f in $FILES; do
    echo "Processing ${f} file..."
    ed  "${f}" <<EOF
% s/\([0-9]\{6\}\)[0-9]\{10\}/\1xxxxxxxxxx/
w
q
EOF
done

This requires fewer steps (ie command calls) because you're not creating a temp file and moving it. Instead you're editing the file in place with the desired changes and then closing it.

% means "operate on every line" (don't worry about lines that don't match)
s means "perform a substitution" -- /[pattern]/[replacement]/
w means write
q means quit
EOF closes out the "here document"

Hope that helps.

Edit Note: Charles Duffy pointed out that ed ${f} would fail on files names with spaces in them and ed "${f}" would not suffer from that particular deficiency. This is true. It's also the case, however, that the for loop above would likely split on any spaces in the file names. You can set IFS (IFS='\n') to get around this limitation on KSH, BASH, MKSH, ASH, and DASH. In ZSH (depending on your version) you may need to set SH_WORD_SPLIT. As an alternative you can change from a for loop to a while loop with read:

FILES=/export/home/*.txt
ls ${FILES} | while read f; do
    echo "Processing ${f} file..."
    ed  "${f}" <<-EOF
% s/\([0-9]\{6\}\)[0-9]\{10\}/\1xxxxxxxxxx/
w
q
EOF
done

Edit Note: My erroneous statements above stricken but kept for historical purposes. See comments from Charles Duffy (below) for clarification.

edited May 23 '17 at 11:49

Community

1
1

answered Feb 16 '14 at 06:03

G. Clifford Williams

136
4

`ed "$f"` -- without the quotes, this won't work correctly on filenames with spaces. – Charles Duffy Feb 16 '14 at 06:35
...err. Using `ls` adds its own, platform-specific problems with unprintable filenames, and using `read` without `-r` or a cleared `IFS` expands backslash sequences and clears trailing whitespace. Much safer to use `printf '%s\0' $FILES | while IFS='' read -r -d '' f`, if for some reason you couldn't do the Right Thing and use `for f in /export/home/*.txt` – Charles Duffy Feb 16 '14 at 14:43
...and no, the for loop above **doesn't** split on spaces, because `FILES=/export/home/*.txt` doesn't expand the glob expression; it just stores it literally. – Charles Duffy Feb 16 '14 at 14:44
Again sir, you are correct. – G. Clifford Williams Feb 16 '14 at 15:50
I tried running the script FILES=/export/home/*.txt for f in $FILES; do echo "Processing ${f} file..." ed "${f}" < – Sen Feb 16 '14 at 17:54
@Sen Does your previous script modify the files appropriately? You can test the action by specifying just one file with the **ed** command `ed [[your filename]]` Then paste in the commands between ***< – G. Clifford Williams Feb 16 '14 at 20:25
@Cliff/Charles -I tried the script in bash/linux which is working fine but when i try the same with sunos/unix it is not working .Not seeing any error , the input file remains same .Pls advice – Sen Feb 18 '14 at 16:59

Shell Scripting -awk option help required

1 Answers1