How to concat rotated logs back together

Question

How to concatenate the rotated logs back together to make the original file?

huali-access.log     huali-access.log.15  huali-access.log.21  huali-access.log.28  huali-access.log.34  huali-access.log.40  huali-access.log.47  huali-access.log.6
huali-access.log.1   huali-access.log.16  huali-access.log.22  huali-access.log.29  huali-access.log.35  huali-access.log.41  huali-access.log.48  huali-access.log.7
huali-access.log.10  huali-access.log.17  huali-access.log.23  huali-access.log.3   huali-access.log.36  huali-access.log.42  huali-access.log.49  huali-access.log.8
huali-access.log.11  huali-access.log.18  huali-access.log.24  huali-access.log.30  huali-access.log.37  huali-access.log.43  huali-access.log.5   huali-access.log.9
huali-access.log.12  huali-access.log.19  huali-access.log.25  huali-access.log.31  huali-access.log.38  huali-access.log.44  huali-access.log.50
huali-access.log.13  huali-access.log.2   huali-access.log.26  huali-access.log.32  huali-access.log.39  huali-access.log.45  huali-access.log.51
huali-access.log.14  huali-access.log.20  huali-access.log.27  huali-access.log.33  huali-access.log.4   huali-access.log.46  huali-access.log.52

score 9 · Answer 1 · edited Feb 27 '15 at 10:24

9

If the files have the correct modification times set (e.g. you did not copy them around without taking care of preserving the modification times), you can use

 cat $(ls -t huali-access.log*) > output.log

The -t option in ls will sort it by modification time.

edited Feb 27 '15 at 10:24

Felix

119
5

answered Feb 20 '13 at 03:52

Daniel t.

9,291
1
33
36

cat $(ls -t -r huali-access.log*) > output.log to reverse the file order. Oldest file first so that the chronological order is maintained on the output.log. – teroi Aug 09 '19 at 09:10

score 6 · Accepted Answer · answered Feb 20 '13 at 03:41

6

like this:

cat huali-access.log* > merged-huali-access.log

or to be sure its chronologically in order:

echo -n "" > merged-huali-access.log # creating new file and making sure its empty
for i in {1..52}
do
    cat huali-access.log.${i} >> merged-huali-access.log
done
cat huali-access.log >> merged-huali-access.log

answered Feb 20 '13 at 03:41

replay

3,240
14
17

Why do you have the 'echo -n ""' bit there? That is superfluous. The concatenation ">>" operation will create an empty file just as done by ">". – Daniel Mar 05 '15 at 21:53
1

@Daniel If the file is already there, it won't empty the file. – ianweller Mar 05 '15 at 22:45
@ianweller ... ack ... too true. – Daniel Mar 06 '15 at 00:54

PMG · Answer 3 · 2015-03-06T10:05:34.500

From @mauro.stettler, Fixed issue with order of files, also made it generic:

for LOG in *.log; do 
    ( for i in {100..1}; do 
       F=${LOG}.${i}; 
       [ -e $F ] && cat $F; 
    done ; cat ${LOG} ) > aggregated_${LOG};
done

Or you can use on accesslogs, not so generic as the other that you can apply to any logs, and will not work with among different months:

cat accesslog.log* | sort -nk 4 > aggregated_accesslog.log

score 0 · Answer 4 · answered Jul 02 '15 at 19:51

0

This will concatenate all log* files (including gzipped) into log.all Just replace "log" to use it

l='log'; test -f ${l}.all && rm ${l}.all; ls -1tr ${l}* | xargs zcat -f >> $l.all

answered Jul 02 '15 at 19:51

Whitehat

331
2
2

user3721740 · Answer 5 · 2015-03-07T15:11:25.613

-1

ls -1t *.access.log* | xargs zcat >  access.all.log

edited Mar 07 '15 at 15:11

answered Mar 05 '15 at 22:29

user3721740

107
1
4

How to concat rotated logs back together

5 Answers5