0

I have implemented a log file that will be storing the cpu and memory state of a process after every minute.I have limited the maximum size of the file to 3MB (thats enough for my purpose).

The script will be called by a cron job after every minute and the script will log the details for that minute and will rename the file as "Log_.log".

When the size reaches "3MB - 100 bytes" I reset the file pointer to point to the begining and will overwrite the first entry in the log file and will now rename the file as "Log_<0+some offset>.log".

As I am renaming the file after every minute to update the file pointer position, is it a good/efficient way ?

I do not want to maintain more than one log file for this purpose.

Another option for me is to maintain the file pointer position in a file ,but ....another file !! not interested in maintaining one if this option is good :)

Thanks in Advance.

Arunmu
  • 6,837
  • 1
  • 24
  • 46
  • Everyone is pointing a different issue with the question, may be you should rephrase it. I removed my answer after I found out your interest is not to change the way you solved the problem but to just find out renaming a file -- a lot -- is a bad thing or no. – Nylon Smile Jan 13 '11 at 07:40
  • yes I need to know if renaming a file after every one minute interval will impact performance on File System/ Code runtime or not – Arunmu Jan 13 '11 at 07:50

2 Answers2

4

Are you an engineer? This is a nice example of some simple task, solved by a perfectly working but overly complex solution.

Unless the content you put in takes exactly as many bytes as the content you take out, writing "in" a file will actually cause the whole following part after your writing position to be rewritten to disk. Append is much cheaper.

Renaming the file to store the pointer works - but it's not very elegant, and makes stuff more complex (for one, your process needs write rights to the directory in which the file resides - else just write access to two files is sufficient)

Unless disk space is an issue (and really, it rarely is), your approach is less efficient than say, append everything to a file, and rotate the file when it reaches its maximum size. This way you always have the last 3MB of logs available, and maximum 3MB more in your current file. It will make parsing the file a lot easier too, instead of recalculating the entire pointer position thing.

Update to answer your comment:

Renaming a file every minute (or even every second) shouldn't slow down your system significantly, don't worry about that.

Our concerns are mainly with "why you think you need to rename the file". It's not better technically, it's not better from a logical point of view, it makes a lot of other (future) tasks harder. You could store the file pointer in a seperate file, or at the end of your file, and there are better^H^H^H^H^H^H simpler solutions that don't require the file pointer at all.

Community
  • 1
  • 1
Konerak
  • 39,272
  • 12
  • 98
  • 118
  • Yes I am an engineer :) .dont know I somehow like to do complicated thing :) to make my job more interesting .Right now the thing is working perfectly fine for me and precision of calculation here doesnt matter much. As the records are almost of same size I can get to know at which line number(approx.) the last update was made. Maintaining two 3MB files would be lot easier and wont be much interesting :). But would renaming a file after every minute efficient considering the filesysytem changes an OS would have to do ? This question is still unanswered. Thaks. – Arunmu Jan 13 '11 at 06:39
  • 2
    If it's a pet project or you're just learning, `interesting` is fine. But if you're working with a team (and your future collegues/self count as team-members, too!) you should not focus on complexity for the sake of interesting. Everyone can invent a complex solution - the art is finding the simplest, most elegant, maintainable, understandable solution :) – Konerak Jan 13 '11 at 08:23
0

I'm confused why you would rename your file. What does this accomplish?

Are the log entries fixed size? Or variable size?

If the entries are fixed size, then there is no trouble in re-writing the existing file from the start: you won't ever have incomplete entries in your file, and if you are writing a counter or timestamps to the file, it should be clear where the 'cursor' is located.

If the entries are variable size, then you should probably not begin re-writing the file from the beginning without somehow making it clear where the 'cursor' is located in the file, and write code that is resilient to reading truncated log entries.

Can you re-use existing tools such as RRDtool?

sarnold
  • 102,305
  • 22
  • 181
  • 238
  • 1) I am renaming the file to contain the file pointer , so that when the log file pointer is reset at some time i can know where to insert the data next time. 2) The entries will be of somewhat constant size , at max I will be at loss of max 1 old entry..which I can live with 3) No, I do not want to use that tool for this purpose What I want to know is the current method efficient ? – Arunmu Jan 13 '11 at 05:04