How to parse fields from different lines to build a new record whith all this fileds

Question

I have a file with this structure:

http://paste.ubuntu.com/21136265/

And I have to capture all the data from the line 'ADSTART ACTION(ADD)' to the next line with this same text, to create a single record, or line.

Sorry but I Can't post an example of the output because is all the data between the 'ADSTART' lines in a single line or record, I'm working under z/OS and we have the concept of record length.

I'm trying this in REXX for z/OS and in AWK in UNIX SYSTEM SERVICES for z/OS, but I'm stuck putting all fields in a line, and I can't figure out how to do it.

I'm capturing the data trough nested loops, but I don't know haw to put it in a single line.

your question is not clear.. and instead of this big difficult to understand file, give a smaller, simpler example input file and expected output.. also add the commands you tried to question.. see http://stackoverflow.com/help/mcve for more details — Sundeep, Jul 27 '16 at 12:44
you could create a manual example with 5-10 lines and give sample output for that.. again, you'll have to show your attempt at solving as well.. — Sundeep, Jul 27 '16 at 13:54

David Crayford · Answer 1 · 2016-07-29T07:33:06.767

If you're using REXX then why don't you just use the parse instruction to scrape the report file? The parse instruction uses a template pattern which is very simple but powerful.

Here's an example:

/* REXX */

queue "ADSTART  ACTION(ADD)"
queue "  ADID(ABCD0B          ) ADVALFROM(111230) CALENDAR(CALSEM7J        )"
queue "  DESCR('DESCRIPTION  ')"
queue "  ADTYPE(A)"
queue "  GROUP(PBQOPC  )"
queue "  OWNER('OWNER1')"
queue "  PRIORITY( 5) ADSTAT(A)"
queue "  ODESCR('ALADIN                  ')"
queue "ADRUN ACTION(ADD)"
queue "  PERIOD(HEB     )  RULE(3) VALFROM(091230)  VALTO(711231)"
queue "  SHIFT(   0)             SHSIGN(F)"
queue "  DESCR('DESCRIPTION')"
queue "  TYPE(N)"
queue "    IADAYS(  1,  2,  3,  4,  5,  6,  7)"
queue "  IATIME(1700) DLDAY(   1)   DLTIME(0600)"

do while queued() > 0
  parse pull rec
  select
    when startswith(rec,"ADSTART") then do
      p. = '' /* the output record */
      parse var rec with . 'ACTION('p.action')'
      do queued()
        parse pull rec
        if left(rec,1) /= ' ' then do
          /* End of parameter group. Re-queue the record and break */
          push rec
          leave
        end
        select
          when startswith(rec, "  ADID") then do
            parse var rec with . "ADID("p.adid") ADVALFROM("p.advalfrom")" ,
              "CALENDAR("p.calendar")"
          end
          when startswith(rec, "  DESCR") then do
            parse var rec with "DESCR('"p.desc"')"
          end
          when startswith(rec, "  PRI") then do
            parse var rec with "PRIORITY("p.priority") ASTAT("p.adstat")"
          end
          otherwise nop
        end
      end
      /* write out the record in 1 line */
      say strip(p.action) strip(p.adid) strip(p.advalfrom) strip(p.calendar),
          strip(p.desc) strip(p.priority) strip(p.adstat)
    end
    when startswith(rec,"ADRUN") then do
      /* do some stuff to parse this */
    end
    otherwise nop
  end
end

exit 0

startswith:
  parse arg input, prefix
  input_len = length(input)
  if input_len = 0 then return 0
  prefix_len = length(prefix)
  if prefix_len = 0 then return 0
  return input_len >= prefix_len & left(input,prefix_len) = prefix

Seeing as you're comfortable in z/OS UNIX environment, if you want something a little bit more powerful than REXX and/or AWK you should checkout my z/OS port of Lua. It comes with an LPeg package which makes it trivially easy to write lexers and parsers with very few lines of code.

If all you want to do is text flow the TWS control statements onto one line without capturing the fields then that's very simple to do.

/* REXX */                                                                   

queue "ADSTART  ACTION(ADD)"                                                 
queue "  ADID(ABCD0B          ) ADVALFROM(111230) CALENDAR(CALSEM7J        )"
queue "  DESCR('DESCRIPTION  ')"                                             
queue "  ADTYPE(A)"                                                          
queue "  GROUP(PBQOPC  )"                                                    
queue "  OWNER('OWNER1')"                                                    
queue "  PRIORITY( 5) ADSTAT(A)"                                             
queue "  ODESCR('ALADIN                  ')"                                 
queue "ADRUN ACTION(ADD)"                                                    
queue "  PERIOD(HEB     )  RULE(3) VALFROM(091230)  VALTO(711231)"           
queue "  SHIFT(   0)             SHSIGN(F)"                                  
queue "  DESCR('DESCRIPTION')"                                               
queue "  TYPE(N)"                                                            
queue "    IADAYS(  1,  2,  3,  4,  5,  6,  7)"                              
queue "  IATIME(1700) DLDAY(   1)   DLTIME(0600)"                            

do while queued() > 0                                                        
  parse pull rec                                                             
  if left(rec,1) /= ' ' then do                                              
    line = rec                                                               
    do queued()                                                              
      parse pull rec                                                         
      if left(rec,1) /= ' ' then do                                          
        push rec;leave                                                                
      end                                                                    
      line = line rec                                                        
    end                                                                      
    say space(line,1) 
  end                                                                        
end                                                                          

exit 0

Thank you David, http://lua4z.com/ seems pretty interesting! I will check out this tool. — Patricio Eduardo Rodrguez Oliv, Aug 02 '16 at 12:04

score 0 · Answer 2 · answered Jul 28 '16 at 07:59

0

Try this;

sed -n '/ADSTART  ACTION(ADD)/,/ADRUN/p'  <filename> | sed 's/ADRUN ACTION(ADD)//g'

answered Jul 28 '16 at 07:59

Mustafa DOGRU

3,994
1
16
24

Thank you very much – Patricio Eduardo Rodrguez Oliv Aug 02 '16 at 12:07

score 0 · Answer 3 · answered Jul 28 '16 at 08:00

Maybe this would do it:

awk '/ADSTART  ACTION\(ADD\)/{print buf; buf=""} {buf=buf""$0" "} END{print buf}' test.in

Commented version:

/ADSTART  ACTION\(ADD\)/ { # for records where ADSTART occurs
  print buf                # output the buffer variable
  buf=""                   # then empty the buffer
} 
{                          # for all records
  # gsub(/^ +| +$/,"")     # here you could trim leading and trailing space
  buf=buf""$0" "           # build the buffer 
} 
END {                      # in the end
  print buf                # output the remaining buffer
}

score 0 · Answer 4 · answered Aug 01 '16 at 19:12

Although the solution above could work for not many lines per block, a solution that only prints the text between ADSTART ACTION (ADD) and assumes only one block will be printed

Bash:

gawk 'BEGIN{s=0} /ADSTART.*ACTION(ADD)/ {s=(s+1)%2} (s==1){ print }' | sed ':a;N;$!ba;s/\n//g'

(ADSTART... lines are omitted)

score 0 · Answer 5 · answered Aug 02 '16 at 12:03

Thank you very much for all the answers.

At last it was pretty easy, because when I do an FTP from z/OS to USS (Unix System Services for z/OS) in binary, all the data is in one line.

At first I was working with a file transfered with FTP (ASCII xlate) to my PC, and then transmitted to USS in binary FTP with WinSCP.

This is the code I used to replace a text pattern with carriage return:

    sed 's/ADSTART  ACTION(ADD)/\
    /g' <input file> ><output file>

with carriage return inserted by pressing enter key, because /r /'$''' /n /x0D didn't worked in USS, I don't know why.

Thank you all again for your time.

Patricio.

How to parse fields from different lines to build a new record whith all this fileds

5 Answers5