Move contents of one section above another section in a configuration file

Question

I have a requirement to move [mysqld] section in a mysql configuration file above another section named #SAFE# in a mysql configuration file and after the section is moved there should be space between the sections,Please find the below code and requirement:

code used:

#!/bin/bash
for server in `cat /home/servers.txt'`
username=$1
password=$2
do
sshpass-p $PWD ssh -o stricthostchecking=no $USERNAME@$server "ed -s /home/my.cnf <<'EOF'
/^\[mysqld\]$/;/^\[.*/ -1m$
?\[mysqld\]?i

.
w
EOF"
done

Actual my.cnf:

   [client]
   a=2
   b=7
   
   [mysql]
    d=6
    e=7
 
   [mysqld]
    
    disable-log-bin = 1
    
    skip-name-resolve = 1
    performance-schema = 0
    local-infile = 0
    mysqlx = 0
    open_files_limit = 200000
    max_allowed_packet = 256M
    sql_mode="NO_ENGINE_SUBSTITUTION"

    #the below are password related of [mysqld]
     validate password=1
    
    innodb_dedicated_server = 1
    innodb_buffer_pool_instances = 48
    
    [myisamck]
    a=3
    b=4
    
    [sst]
    d=3
    c=4
    
    # SAFE #
    d=0
    f=0

Requirement:

The [mysqld] section and its sections should be pasted above # SAFE #( section in very few cases section is given as ##SAFE## or #SAFE# or # SAFE # in few of the my.cnf files),Please support.

Expected output:

  [client]
   a=2
   b=7
   
   [mysql]
    d=6
    e=7
    
    [myisamck]
    a=3
    b=4
    
    [sst]
    d=3
    c=4
    
   [mysqld]
    
    disable-log-bin = 1
    
    skip-name-resolve = 1
    performance-schema = 0
    local-infile = 0
    mysqlx = 0
    open_files_limit = 200000
    max_allowed_packet = 256M
    sql_mode="NO_ENGINE_SUBSTITUTION"
    
    #the below are password related of [mysqld]
     validate password=1

    innodb_dedicated_server = 1
    innodb_buffer_pool_instances = 48
    
    # SAFE #
    d=0
    f=0

Does this answer your question? [Move certain section information above another section in a configuration file](https://stackoverflow.com/questions/76748132/move-certain-section-information-above-another-section-in-a-configuration-file) — Jens, Jul 28 '23 at 16:56
@Jens As per my requirement i need to move it above a specific section. — Raj, Jul 28 '23 at 16:58
will the `[mysqld]` block *always* come before the `# SAFE #` line? — markp-fuso, Jul 28 '23 at 18:06
Look for INI file parsers, but you'll need to do some coding. — glenn jackman, Jul 28 '23 at 18:12
I'm not asking about where to *move* the `[mysqld]` block, I'm asking about where the `[mysqld]` block appears in the source file, *before* the move must take place — markp-fuso, Jul 29 '23 at 13:52
@markp-fuso it usually appears after [client] and [mysql] sections. — Raj, Jul 30 '23 at 01:04
my question was: *before the move* .... does the `[mysqld]` block ***always*** come before the `# SAFE #` line? — markp-fuso, Jul 30 '23 at 13:40

Jetchisel · Accepted Answer · 2023-07-29T03:19:43.480

1

I don't use sshpass but plain ol ssh with key, something like:

#!/usr/bin/env bash

servers=(
  server1
  server2
  server3
  ...
)

for server in "${servers[@]}"; do
  ssh -t "jetchisel@$server" 'ed -s ~/my.cnf <<EOF
  /#*[[:space:]]*SAFE[[:space:]]*#*/kx
  /^\[mysqld\]$/;/^\[.*/-1m'\''x-1
  ,p
  Q
EOF'
done

To add the content of /home/servers.txt to an array, one way is to use mapfile aka readarray which is a bash4+ feature.

mapfile -t servers < /home/servers.txt

Add w after the line where ,p is at to actually write/edit the file in-place
Remove the ,p to silence the output.

edited Jul 29 '23 at 03:19

answered Jul 29 '23 at 02:44

Jetchisel

7,493
2
19
18

#!/bin/bash username=$1 PWD=$2 mapfile -t servers < /home/servers.txt for server in "${servers[@]}"; do sshpass-p $PWD ssh -o stricthostchecking=no $USERNAME@$server 'ed -s ~/my.cnf < – Raj Jul 29 '23 at 08:51
I did iinstall sshpass, and it works as expected, so I don't know what is wrong with your setup. – Jetchisel Jul 29 '23 at 10:28
Iam getting ? as the output – Raj Jul 29 '23 at 12:50
Well, to start with the `stricthostchecking=no` is an error on my side, it should be `StrictHostKeyChecking=no ` ... – Jetchisel Jul 29 '23 at 20:32
Also, a heredoc requires to have a literal new lines, not in one-line like what you have. – Jetchisel Jul 29 '23 at 21:43
There was a problem with my setup,It is working now. – Raj Jul 31 '23 at 08:37

markp-fuso · Answer 2 · 2023-07-30T18:51:12.963

Assumptions/Understandings:

input lines may have leading white space (based on OP's sample my.cnf file)
the [mysqld] block may come before or after the # SAFE # line; options include storing the entire file in memory structures or making two passes through the file
OP has mentioned both #SAFE# and # SAFE # so we'll look for a line with the 3 strings # / SAFE / # where they may be white space between the strings
the line # SAFE # (with or without white space) only occurs once in the file

One awk idea requiring two passes through the input file:

awk '
BEGIN { regex_hdr  = "^[[:space:]]*" "[[]" "[^]]*"        "[]]"                     "[[:space:]]*$"
        regex_safe = "^[[:space:]]*" "#"   "[[:space:]]*" "SAFE" "[[:space:]]*" "#" "[[:space:]]*$"
      }

      { if ($0 ~ regex_hdr) {               # if line looks like "[....]" then ...
           if ($1 == "[mysqld]") {          # if the header is "[mysqld]" then ...
              if   (FNR==NR) save = 1       # if 1st file set the "save" flag else ...
              else           skip = 1       # (2nd file) set the "skip" flag
           }
           else
              save = skip = 0               # (not "[mysqld]") then clear flags
        }

        if ($0 ~ regex_safe) {              # if line looks like "# SAFE #" and ...
           if (NR>FNR)                      # if processing 2nd file then ...
              for (i=1; i<=blkcnt; i++)     # loop through the block[] array and ...
                  print block[i]            # print the "[mysqld]" block
           save = skip = 0                  # reset variables
        }

        if (save)                           # if 1st file and in the "[mysqld]" block then ...
           block[++blkcnt] = $0             # save the line in our block[] array
        if ( NR>FNR && !skip )              # if 2nd file and not skipping then ...
           print                            # print current line
      }
' my.cnf my.cnf                             # two copies of file are read/processed

Where:

the regex's are broken into smaller strings for easier reading/understanding (awk will concatenate them into a single string during processing)
FNR==NR - is true when processing the 1st file
NR>FNR - is true when processing the 2nd file
during processing of the 1st file we find and save (in the block[] array) the [mysqld] block
during processing of the 2nd file we a) ignore all lines in the [mysqld] block, b) print all other lines to stdout except c) if the line consists of the string # SAFE # then we'll first print the contents of the block[] array (ie, the [mysqld] block) to stdout and then print the # SAFE # line to stdout

This generates:

[client]
a=2
b=7

[mysql]
 d=6
 e=7

 [myisamck]
 a=3
 b=4

 [sst]
 d=3
 c=4

[mysqld]

 disable-log-bin = 1

 skip-name-resolve = 1
 performance-schema = 0
 local-infile = 0
 mysqlx = 0
 open_files_limit = 200000
 max_allowed_packet = 256M
 sql_mode="NO_ENGINE_SUBSTITUTION"

 #the below are password related of [mysqld]
  validate password=1

 innodb_dedicated_server = 1
 innodb_buffer_pool_instances = 48

 # SAFE #
 d=0
 f=0

NOTES:

once OP is satisifed with the output, and assuming the intention is to overwrite the original file, then a) redirect the output to a temporary file (eg, awk '...' my.cnf my.cnf > tmpfile) and then b) mv tmpfile my.cnf
if using a newer version GNU awk it is possible to have awk overwrite the original file but that will require playing with the inplace library and toggling some related variables on the command line ... doable but a bit more complicated
if getting this to work in a ssh/EOF construct is too wieldly, perhaps consider copying the file to the remove host (eg, scp) before executing (via ssh) ... ?

Your answer is also working,Thanks for for your efforts for writing such a complicated code.Iam not able to mark two correct answers. — Raj, Jul 31 '23 at 09:05
@Raj in our earlier exchange of questions/comments you confirmed that, *before* performing the edit, the `[mysqld]` block could come before *or* after the `# SAFE #`; the answer you've accepted does not appear to work if, *before* performing the edit, the `[mysqld]` comes after the `# SAFE #` — markp-fuso, Jul 31 '23 at 16:43

score 0 · Answer 3 · answered Jul 30 '23 at 23:41

0

$ awk '
   /\[mysqld\]/,/\[myisamck\]/{ 
       if($0 !~ / *\[myisamck\] */) {
          section=section"\n"$0
          next
       } 
   }
   /#?# ?SAFE ?##?/{
       print section
   }1
' my.cnf

answered Jul 30 '23 at 23:41

ufopilot

3,269
2
10
12

Move contents of one section above another section in a configuration file

3 Answers3