9

I have a cron job which runs every 30 minutes to generate log files with time-stamp like this: 

test20130215100531.log, 
test20130215102031.log

I would like to create one folder daily with date time-stamp and push log files in to respective date folder when generated.

I need to achieve this on AIX server with bash.

user1746666
  • 161
  • 1
  • 2
  • 9
  • What about a cronjob that creates the directory at 00.00 every night? And then modify actual cronjob to push files to this directory. – fedorqui Feb 15 '13 at 12:34
  • I would recommend `mkdir -p` for creating missing directory each time without complaining if it already exists. What if 00:00 job fails to run for some reason (system being down), do we want all daily jobs to fail because of it? – Anton Kovalenko Feb 15 '13 at 12:36
  • Yes, you are right. I prefer your option. – fedorqui Feb 15 '13 at 12:42
  • Thanks for the suggestions. I am not going to configure the above requirement in the cronjob. I have a shell script which creates log files. i need to write some script to have the directory and push generated logs in to that – user1746666 Feb 15 '13 at 12:54
  • OK. What did you code so far? – fedorqui Feb 15 '13 at 13:17
  • Why don't you write the log files in the right directory in the first place instead of "pushing" them in afterwards with another script? – dogbane Feb 15 '13 at 13:50
  • yeah.. i need to the same.. creating a directory just once when the day starts and write logs for the same day in to the directory.. Right now, i am forming the logs through a script and placing in one location with timestamp. – user1746666 Feb 15 '13 at 14:17

2 Answers2

20

Maybe you are looking for a script like this:

#!/bin/bash

shopt -s nullglob  # This line is so that it does not complain when no logfiles are found
for filename in test*.log; do # Files considered are the ones starting with test and ending in .log
    foldername=$(echo "$filename" | awk '{print (substr($0, 5, 8));}'); # The foldername is characters 5 to 13 from the filename (if they exist)
    mkdir -p "$foldername"  # -p so that we don't get "folder exists" warning
    mv "$filename" "$foldername"
    echo "$filename $foldername" ;
done

I only tested with your sample, so do a proper testing before using in a directory that contains important stuff.

Edit in response to comments:

Change your original script to this:

foldername=$(date +%Y%m%d)
mkdir -p  /home/app/logs/"$foldername"
sh sample.sh > /home/app/logs/"$foldername"/test$(date +%Y%m%d%H%M%S).log

Or if the directory is created somewhere else, just do this:

sh sample.sh > /home/app/logs/$(date +%Y%m%d)/test$(date +%Y%m%d%H%M%S).log
user3360767
  • 868
  • 7
  • 18
user000001
  • 32,226
  • 12
  • 81
  • 108
  • Thanks so much !! How would that be if i can create a directory just once when the day starts and write logs for the same day in to the directory. Right now.. i am forming the logs through a script placing in one location with timestamp. – user1746666 Feb 15 '13 at 14:19
  • Could you post the part of your script that creates the log file with the timestamp? It is easier to work from that. _(If you edit your question, also leave a comment here, so I get notified)_ – user000001 Feb 15 '13 at 14:26
  • // am doing some configuration stuffs here then executing another script to create the file as below. sh sample.sh > /home/app/logs/test$(date +%Y%m%d%H%M%S).log The above runs every 30 minutes, which would generate 48 logs in a day – user1746666 Feb 15 '13 at 14:38
  • Read my edit. This creates a folder name YYYY/MM/DD (if it does not exist), and stores the data there. – user000001 Feb 15 '13 at 14:48
  • thanks so much ! Hope the files for one date will not get misplaced with the above solution.. – user1746666 Feb 15 '13 at 14:50
  • I guess there is a small chance (in the last script) that the date changes between the two `date` calls, but I would think it is a bit slim. But do some test first, anyway – user000001 Feb 15 '13 at 14:53
  • Hi @user000001. This one worked for me: mkdir -p /home/app/logs/$(date +%Y%m%d) But when i assign date to a variable 'foldername', its not creating any folder: mkdir -p /home/app/logs/"$foldername" Even i tried by removing double quotes, but no success. However in my another bash script, i am using variable the same way (witout double quotes) and it works fine. Any idea whats wrong ?? – user1746666 Feb 18 '13 at 10:46
  • 1. As for the quotes, it is better to use them, otherwise the script will fail if the variable has spaces in it. See also _[word splitting](http://mywiki.wooledge.org/WordSplitting)_. 2. One reason for the above command not to work would be if you have spaces before or after the `=`. Do an `echo "$foldername"` to see if it gets assigned. You can also do a quick check of the script at http://www.whatswrongwithmyscript.com/. – user000001 Feb 18 '13 at 11:01
  • folder_name="$(date +%Y%m%d)" echo "$folder_name" This itself doesnt print any output and there is nothing wrong shown in whatwrongwithmyscript URL. – user1746666 Feb 18 '13 at 12:54
  • :-) This too didn't work. Anyways Thanks for all your help. Let me investigate more on this. – user1746666 Feb 18 '13 at 13:06
  • Well if `echo "THIS SHOULD BE PRINTED"; folder_name="$(date +%Y%m%d)"; echo "$folder_name"` does not print either then you know the problem is somewhere else... – user000001 Feb 18 '13 at 13:24
  • let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/24716/discussion-between-user1746666-and-user000001) – user1746666 Feb 18 '13 at 16:22
  • I was executing all this before EOF.. (ssh node < – user1746666 Feb 19 '13 at 16:33
  • Ok that explains it... No problem – user000001 Feb 19 '13 at 16:36
2

You should use logrotate! It can do this for you already, and you can just write to the same log file.

Check their man pages for info: http://linuxcommand.org/man_pages/logrotate8.html

Billy Lazzaro
  • 476
  • 2
  • 13