AWK split file needs to a close

Question

I have a text file that looks like this:

valcred: requestValCred [up]
 certificate pki_001  [up]
 certificate pki_002  [up]
 certificate pki_003  [up]
 certificate pki_004  [up]
 certificate pki_005  [up]
valcred: internalValCred [up]
 certificate pki_021  [up]
 certificate pki_012  [up]
 certificate pki_103  [up]
 certificate pki_404  [up]
 certificate pki_555  [up]

I want to split this into a new text file everytime "valcred" appears. Then, I want to place all the certificates into the file with said valcred. Below are the contents of one file. There are around 100 of these files each with a different number of entries. I get about 20 files when the error occurs.

valcred: internalValCred [up]
 certificate pki_021  [up]
 certificate pki_012  [up]
 certificate pki_103  [up]
 certificate pki_404  [up]
 certificate pki_555  [up]

Im using this command:

nawk '/valcred/{x="F"++i;}{print > x;}' input_file.txt

but this is where it goes wrong. I run into this error:

bash-2.03$ nawk '/valcred/{x="F"++i;}{print > x;}' input_files.txt
nawk: F21 makes too many open files
 input record number 1743, file input_files.txt
 source line number 1

I thought I just need to close the print statement. Nothing has worked for me. Can someone help me add a close statement to this that would make it work? Or an alternative solution?

Maybe its a unix limit, not awk. Type 'ulimit -a' to see what the maximum number of open files you can have is. 20 is the old max from the way back. — Totoro, Jun 12 '14 at 19:13

score 1 · Accepted Answer · answered Jun 12 '14 at 19:12

1

You need to close the output file before opening a new one. Also I expect you want to append >> to the file x rather then overwriting > it again and again:

nawk '/valcred/{close(x);x="F"++i;printf "" > x}{print >> x;}' input_file.txt

Also note, that I'm truncating the file on it's first usage using printf "" > x.

answered Jun 12 '14 at 19:12

hek2mgl

152,036
28
249
266

Is there any documentation on the placement of the "close" ? I kept trying to place it later nawk '/valcred/{x="F"++i;}{print > (x); close(print > (x));}' <--Of course this didn't work but I had other documentation that appeared to lead me to this attempt. – user3735276 Jun 12 '14 at 19:37

score 0 · Answer 2 · answered Jun 12 '14 at 19:21

0

Put this in a script file and run it on the text file you want to break up.

#!/usr/bin/awk -f

BEGIN { valnum=0 }

/^valcred/ {valnum++
            close(output)
            print $0 > valnum}
/^ certificate/ { print $0 > valnum }

answered Jun 12 '14 at 19:21

William Everett

751
1
8
18

AWK split file needs to a close

2 Answers2