0

I have several servers that log some data into .csv files and send the files to a NAS. I need to add each .csv file to a corresponding aggregate logfile. Googling has turned up nothing relevant (perhaps I'm not using the right searchstring?)

The way I'm doing this now is by having a script like so:

#! /usr/bin/bash

cat a.log a.csv > a.log
cat b.log b.csv > b.log
cat c.log c.csv > c.log
:

If new logging data is created then I need to edit the script and add a new line.

Surely, it must be possible to do this more easily and flexible. Any hints are appreciated.

Mausy5043
  • 906
  • 2
  • 17
  • 39
  • 1
    you're wiping out your existing X.log file with the `> a.log`. Why not just `cat a.csv >> a.log`? Read about `>>` (i.e. append redirection). Better be working with some replaceable test files until you get this working ;-) Good luck. – shellter May 30 '14 at 15:16
  • Yes the `>>` does seem smarter. Thanks for the hint. – Mausy5043 May 30 '14 at 17:30

2 Answers2

2

Why are you dumping the contents of a.log into a.log? This seems to make no sense.

I think you actually want to do this:

for logfile in *.log; do
    csvfile=${logfile/.log/.csv}
    cat $csvfile >> $logfile
done

which, I believe, is the equivalent of this:

cat a.csv >> a.log
cat b.csv >> b.log
cat c.csv >> c.log

Caveat: I don't have a

DwB
  • 37,124
  • 11
  • 56
  • 82
  • I think you're right adding the new data by using `>>` does seem smarter. Could you explain the second line in your proposed solution? I don't understand what it does. Is it some kind of `sed` shorthand? – Mausy5043 May 30 '14 at 17:29
  • there are 3 "regions". 1: the value in the logfile variable. 2. change one occurrence of this string. 3. into this string. Basically: in the value (in the logfile variable) change .log into .csv – DwB May 30 '14 at 17:40
  • For more details, search for "${parameter/pattern/string}" on this page: http://www.gnu.org/software/bash/manual/bashref.html#Word-Splitting – DwB May 30 '14 at 17:42
  • Brilliant! I didn't know bash could do that :-D You learn something new every day. – Mausy5043 May 30 '14 at 17:43
0

assuming that all log files have an equivalent csv file

for i in $(ls *.log); do
    logfile=$i
    csvfile=${i/.log/.csv}
    cat $logfile $csvfile > $logfile
done

it basically asks for a list of log file in the directory.

then, it 'computes' for the csv file name

then it runs your command, but this time automatically for each log file in your directory.

Mustard_Jedi
  • 25
  • 5