Using sed to handle a complete section of text

Question

I am trying bash scripting for the first time and I'm stuck on using sed to alter a config file. The keys that I need to edit are pretty generic and used throughout the file, so I have to rely on a section to know where to do my "touching".

So, as said, there are multiple sections around the file, declared by [section name] and then under it, you have the configurations for that section, indented by one tab. There are keys like enabled, type etc. which are used all over the file.

So a complete section would look like this:

[backend]
        # enabled = no
        # data source = average
        # type = graphite
        # destination = localhost
        # prefix = netdata
        # hostname = localhost
        # update every = 10
        # buffer on failures = 10
        # timeout ms = 20000

So what I need to do is:

Uncomment the whole section
Change enabled = no to enabled = yes
Change destination's value to an IP address with a port (192.168.99.38:2003)
Change hostname's value to another hostname, maybe the machine's hostname ($HOSTNAME?).

Thing is, I'm not sure how to tackle this at all. I have looked around for sed multiline handling, but I definitely don't know how to match the [backend] section only. This is very new to me.

So you only want to edit the configurations below the `[backend]` if that exists or not you want to edit all section below `[section-name`]? — Shakiba Moshiri, Apr 19 '17 at 11:13
@k-five Only what's under `[backend]`, nothing else. Sorry for not being clear enough, — aborted, Apr 19 '17 at 12:15
@Inian I cannot see your answer. I saw one earlier and I was going to apply it, but it's gone now. — aborted, Apr 19 '17 at 12:22
@Aborted: removed it because it didn't satisfy your `Only what's under [backend]` condition, fixing it now — Inian, Apr 19 '17 at 12:23

score 1 · Answer 1 · answered Apr 19 '17 at 14:05

1

sed is not the ideal tool for this kind of stuff, but it can be done:

/^\[.*]$/h
{G;/\n\[backend\]$/{
  s/#//
  /enabled/s/no/yes/
  /destination/s/localhost/whatever/
  /hostname/s/localhost/something else/
  }
  s/\n\[.*\]$//
}

answered Apr 19 '17 at 14:05

Michael Vehrs

3,293
11
10

Shakiba Moshiri · Accepted Answer · 2017-04-21T09:05:09.173

As you comment me: Only what's under [backend], nothing else

It may interest you doing that with Perl and if not just comment me; I will delete and the answer.

Say you have this file:

[no-backend]
        # enabled = no
        # data source = average
        # type = graphite
        # destination = localhost
        # prefix = netdata
        # hostname = localhost
        # update every = 10
        # buffer on failures = 10
        # timeout ms = 20000

[backend]
        # enabled = no
        # data source = average
        # type = graphite
        # destination = localhost
        # prefix = netdata
        # hostname = localhost
        # update every = 10
        # buffer on failures = 10
        # timeout ms = 20000

[no-backend]
        # enabled = no
        # data source = average
        # type = graphite
        # destination = localhost
        # prefix = netdata
        # hostname = localhost
        # update every = 10
        # buffer on failures = 10
        # timeout ms = 20000

This one-liner finds that section for you:

perl  -lne '$b=$.; $e=($b+10) if /\[backend\]/;print if $b++<$e' file

or readable version
perl -lne 'if( /\[backend\]/ ){ $b=$.; $e=( $b+10 ); }; if( $b++ < $e ){ print }' file

and the output:

[backend]
        # enabled = no
        # data source = average
        # type = graphite
        # destination = localhost
        # prefix = netdata
        # hostname = localhost
        # update every = 10
        # buffer on failures = 10
        # timeout ms = 20000

and now instead of print you can modify that section with:

s/#//;s/no/yes/;s/(?<=destination = ).+$/192.168.99.38:2003/;s/(?<=hostname = ).+$/$HOSTNAME/

the full one-liner

perl -lpe 'if(/\[backend\]/){$b=$.;$e=($b+10);};if($b++<$e){ s/#//;s/no/yes/;s/(?<=destination = ).+$/192.168.99.38:2003/;s/(?<=hostname = ).+$/\$HOSTNAME/ }' file

and the output:

[no-backend]
        # enabled = no
        # data source = average
        # type = graphite
        # destination = localhost
        # prefix = netdata
        # hostname = localhost
        # update every = 10
        # buffer on failures = 10
        # timeout ms = 20000
[backend]
         enabled = yes
         data source = average
         type = graphite
         destination = 192.168.99.38:2003
         prefix = netdata
         hostname = $HOSTNAME
         update every = 10
         buffer on failures = 10
         timeout ms = 20000

[no-backend]
        # enabled = no
        # data source = average
        # type = graphite
        # destination = localhost
        # prefix = netdata
        # hostname = localhost
        # update every = 10
        # buffer on failures = 10
        # timeout ms = 20000

and finally after checking the output if everything was good, then you can use -i option to use edit-in-place feature, like:

perl -i.bak -lne '...the rest of the script...' file

.bak is just for getting backup of your old file. ( like: file.txt.bak )

UPDATE for your comment

perl -lpe '$hn=qx(cat /etc/hostname);chomp $hn;if(/\[backend\]/){$b=$.;$e=($b+10);};if($b++<$e){s/#//;s/no/yes/;s/(?<=destination = ).+$/192.168.99.38:2003/;s/(?<=hostname = ).+$/$hn/ }' file

and the output:

...
...
[backend]
         enabled = yes
         data source = average
         type = graphite
         destination = 192.168.99.38:2003
         prefix = netdata
         hostname = k-five
         update every = 10
         buffer on failures = 10
         timeout ms = 20000
...
...

I don't mind Perl. I'm fairly certain it's installed by default on all Ubuntu versions. But one question: What are the `[no-backend]` sections? — aborted, Apr 20 '17 at 07:42
@Aborted Most Linux Distribution have **Perl** by default and Ubuntu as well. `[no-backend]` is just another section that should not be match, Just for sure — Shakiba Moshiri, Apr 20 '17 at 09:54
Another question: I don't want the literal `$HOSTNAME` string. `$HOSTNAME` would be a system variable which includes the system's hostname. Is your Perl script concatenating this variable or putting the literal expression in there? — aborted, Apr 21 '17 at 08:48
@Aborted [First of all see here](http://stackoverflow.com/questions/16663115/how-to-get-hostname-in-solaris-machines-through-perl-script) and for the result you can add `$hn= qx( cat /etc/hostname );chomp $hn;` before first `if` and in place of $HOSTNAME at the end use `$hn` like: `/$hn/ }' file` — Shakiba Moshiri, Apr 21 '17 at 08:58
Sorry for the excessive requests. You have been a huge help to me. Last question: Lets say I use the `hostname` command to get the hostname (ie: `$ hostname`, the output is the hostname) and I read the user's input for the IP address like: `read -p "Enter the current system's IP address: " IPADDRESS`, then the variable would be `$IPADDRESS`. How would I incorporate these to the perl command? I tried something like: `perl -lpe 'if(/\[backend\]/){$b=$.;$e=($b+10);};if($b++<$e){ s/#//;s/no/yes/;s/(?<=destination = ).+$/'${IPADDRESS}'/;s/(?<=hostname = ).+$/'${HOSTNAME}'/ }' file` - not working. — aborted, Apr 21 '17 at 09:25
@Aborted Your code is getting more complex and may it is better to write a script by anyway to can pass that variable to the **Perl** as argument. [SEE HERE](http://www.perlmonks.org/?node_id=844923) and [SEE HERE](http://stackoverflow.com/questions/13093709/how-to-use-shell-variables-in-perl-command-call-in-a-bash-shell-script). **and for example** try on terminal: `i=one` and then `perl -le 'print $ARGV[0]' "$i"` the `$ARGV[0]` is the first argument that is passed to the program. YOU can store them inside, like previous `$hn` and use it — Shakiba Moshiri, Apr 21 '17 at 09:39
I'm now using the method to place the arguments after the code. I don't exactly understand what's happening here though: `if(/\[backend\]/){$b=$.;$e=($b+10);}`. Thanks for the huge help you've given to me. If I could accept your answer again, I would. — aborted, Apr 24 '17 at 12:25
@Aborted **Notice** that section that should be considered has 10 lines. from line `[backend]` up to `timeout...`. So that if checks when it can be match against `/beckend/` after that `$b` = begin and `$e` = end and `$.` is a line counter in **Perl**. So begin = `$.` and since that section has 10 line, and end => `$n = $. + 10` — Shakiba Moshiri, Apr 24 '17 at 13:08
@Aborted Play with this part: `perl -lne '$b=$.; $e=($b+10) if /\[backend\]/;print if $b++<$e' file` I you will understand how it works — Shakiba Moshiri, Apr 24 '17 at 13:11

score 1 · Answer 3 · answered Apr 19 '17 at 14:23

Assuming the text is in configfile and there is another section underneath starting with "[", the solution below should work

sed -i '/\[backend\]/,/\[/{s/#//;s/enabled = no/enabled = yes/;s/\(destination = \)\(.*\)\($\)/\1192\.168\.99\.38\:2003\3/;s/\(hostname = \)\(.*\)\($\)/\1'$HOSTNAME'\3/}' configfile

We search for the text between [backend] and [ and then execute the sed commands enclosed in {} We remove # and then edit the destination as well as finally the hostname.

If there are no other sections below backend, change the command to:

sed -i '/\[backend\]/,/^$/{s/#//;s/enabled = no/enabled = yes/;s/\(destination = \)\(.*\)\($\)/\1192\.168\.99\.38\:2003\3/;s/\(hostname = \)\(.*\)\($\)/\1'$HOSTNAME'\3/}' configfile

Using sed to handle a complete section of text

3 Answers3