sed replace group of files string

Question

I am trying to replace a string in multiple lines. It is late and I'm getting restless, maybe someone would prefer to give it a shot for some SO points. The string I'm replacing is 'STORED AS TEXTFILE' from the SQL below...

PARTITIONED BY(load string, date string, source_file string)
STORED AS TEXTFILE
LOCATION '${staging_directory}/${tablename}';

And to make it look like...

PARTITIONED BY(load string, date string, source_file string)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
STORED AS TEXTFILE
LOCATION '${staging_directory}/${tablename}';

So my expression is

:%s/)\nSTORED AS TEXTFILE\nLOCATION '/)\rROW FORMAT DELIMITED \rFIELDS TERMINATED BY ',' \rSTORED AS TEXTFILE \rLOCATION '/g

Which works inside the file (with vim) but I can't get it to work with one command on all of the files in the directory. I've tried so far...

sed -i "s/)\nSTORED AS TEXTFILE\nLOCATION '/)\rROW FORMAT DELIMITED \rFIELDS TERMINATED BY ',' \rSTORED AS TEXTFILE \rLOCATION '/g"

... and I also tried the above statement with all the spaces escaped. Please help!

RomanPerekhrest · Accepted Answer · 2017-06-13T14:45:58.873

2

gawk in-place editing approach - available since GNU awk 4.1.0:

gawk -i inplace '$0~/STORED AS TEXTFILE/{ $0="ROW FORMAT DELIMITED" ORS "FIELDS TERMINATED BY \047,\047" ORS $0 }1' file*

-i inplace - in-place editing of each input file

edited Jun 13 '17 at 14:45

answered Jun 13 '17 at 07:43

RomanPerekhrest

88,541
4
65
105

what *Usage message*? – RomanPerekhrest Jun 13 '17 at 14:29
Usage: gawk [POSIX or GNU style options] -f progfile [--] file ... Usage: gawk [POSIX or GNU style options] [--] 'program' file ... POSIX options: GNU long options: -f progfile --file=progfile -F fs --field-separator=fs -v var=val --assign=var=val – uh_big_mike_boi Jun 13 '17 at 14:34
what's your OS? – RomanPerekhrest Jun 13 '17 at 14:35
It is CentOS 6.8 – uh_big_mike_boi Jun 13 '17 at 14:41
@Mike, got it. I should have mentioned that in-place extension available since GNU awk 4.1.0, added a note to the answer. https://rpmfind.net/linux/rpm2html/search.php?query=gawk – RomanPerekhrest Jun 13 '17 at 14:47

Nahuel Fouilleul · Answer 2 · 2017-06-13T07:20:57.627

sed processes file line by line following link gives a solution for multiline processing https://unix.stackexchange.com/questions/26284/how-can-i-use-sed-to-replace-a-multi-line-string

otherwise in perl the default input line separator $/ can be changed or undefined (read whole file):

perl -i.BAK -pe 'BEGIN{undef$/}s/.../.../g' file

After reading comments on accepted answer of link, GNU sed has the -z option which does quite the same uses NUL character as line delimiter ($\="\0"), whereas undef $/ uses no delimiter.

sed replace group of files string

2 Answers2