-4

This is the code:

#!/usr/bin/env python

#Import the datetime 
from datetime import datetime
import re

#Create two datetime object for limit 1 and limit 2 as dt1 and dt2 respectively
dt1 = datetime.strptime("01:00:00","%H:%M:%S").time()

dt2 = datetime.strptime("04:59:59","%H:%M:%S").time()

#Create a compiler for regular expression
init_re = re.compile(r'(INIT)')

time_re = re.compile(r'(\d+:\d+:\d+)')

# read line from test.log file
for line in open("test.log", "r"):

        match = time_re.search(line) #Search time format for each line
        if match:
             matchtime = match.group(1)
             dt_match = datetime.strptime( matchtime,    '%H:%M:%S').time()       
              #Time formmat match
              if dt_match >= dt1 and dt_match <= dt2:
                  match1 = init_re.search(line) #search INIT format
                  if match1:
                      matchinit = match1.group(0)
                      print match.string.strip()

Below is a partial part of the log file:

2015-12-15 00:51:01,904 INFO restser.py 113 [INIT] [netkv_restser: peek] [req_id: f0aa7ab5-6192-4231-93cd-82a53936a072] [request: {u'key_space': u'martech_user_index', u'table_name': u'nettopic', u'key': 9569}]

I want output like this:

[Date: ] [Time: ] [INIT] [netkv_restser: ] [req_id: ]

Example:

[Date:2015-12-15 ] [Time:00:51:01,904 ] [INIT] [netkv_restser:peek ] [req_id:f0aa7ab5-6192-4231-93cd-82a53936a072 ]

Note: If you are nice enough to edit the code, be kind enough to provide solutions. I don't wanna sound rude but it irks me.

NOTE: I am using Python 2.7.6.

Ganesh Kashyap
  • 1
  • 7
  • 5
    Why you ask [a single question](http://stackoverflow.com/questions/36975197/how-to-extract-specific-data-from-a-string) multiple times? – EbraHim May 02 '16 at 09:41
  • I wanted help and no one provided it. So I rebuilt my code and asked again. Buddy, Help out no. – Ganesh Kashyap May 02 '16 at 09:42
  • Asking it twice won't help you and is not legal based on SO rules. Anyway, I don't wanna sound rude, but I edited your last question and left an answer too. – EbraHim May 02 '16 at 09:44
  • I will, but I want answer for this; That question was wrong. – Ganesh Kashyap May 02 '16 at 09:44
  • @EH, Brother, You provided for raw input. I want it to be in script. – Ganesh Kashyap May 02 '16 at 09:46
  • 1
    You shouldn't delete a question that received an answer! You must leave comments under the answer if it not what you wanted and make updates/edits for your question instead of asking it again! – EbraHim May 02 '16 at 09:48
  • 1
    I'm not riding my back dude, I'm just trying to help you receive your answer and trying to make a cleaner SO for other user. – EbraHim May 02 '16 at 09:53
  • comments are not a chat. if you want a cleaner SO for other users then please you both delete your comments. meanwhile i'm trying to write an answer ;) – Francesco May 02 '16 at 09:59

2 Answers2

0

You could use a regex to parse each log line. It depends on how fixed the structure of your log file is but this works on the input line you provided.

template = '[Date:%s] [Time:%s] [%s] [netkv_restser:%s] [req_id:%s]'
details = re.search('(\d{4}-\d{2}-\d{2}) (\d{2}:\d{2}:\d{2},\d{3}).*\[(INIT)\].*\[netkv_restser: *(.*?)\].*\[req_id: *(.*?)\]',s).groups()
output = template % details

the output is 

'[Date:2015-12-15] [Time:00:51:01,904] [INIT] [netkv_restser:peek] [req_id:f0aa7ab5-6192-4231-93cd-82a53936a072]'

of course you can compile the regex and include in the loop over the file lines

pattern = re.compile('(\d{4}-\d{2}-\d{2}) (\d{2}:\d{2}:\d{2},\d{3}).*\[(INIT)\].*\[netkv_restser: *(.*?)\].*\[req_id: *(.*?)\]')
template = '[Date:%s] [Time:%s] [%s] [netkv_restser:%s] [req_id:%s]'
for line in open("test.log", "r"):
    details = re.search(pattern, line).groups()
    print template % details
Francesco
  • 4,052
  • 2
  • 21
  • 29
  • I added an example based on your code. the output of the `search` is a tuple. `%s` is a placeholder for a string. You use the format operator `%` to substitute placeholders with a tuple of values – Francesco May 02 '16 at 10:18
  • If you look at the example there's everything you need – Francesco May 02 '16 at 10:26
  • Traceback (most recent call last): File "testcase1.py", line 32, in print template % details TypeError: not enough arguments for format string – Ganesh Kashyap May 02 '16 at 12:11
  • File "testcase1.py", line 31, in details = re.search(pattern, s).groups() NameError: name 's' is not defined – Ganesh Kashyap May 02 '16 at 12:13
  • Plus it's giving re.search shouldn't have groups()!! what does it mean? – Ganesh Kashyap May 02 '16 at 12:14
  • sorry it was another typo. `s` was my string but then i changed it to `line` to match your code. – Francesco May 02 '16 at 12:26
  • Traceback (most recent call last): File "log_format_re.py", line 26, in details = re.search(pattern, line).groups(1) AttributeError: 'NoneType' object has no attribute 'groups' – Ganesh Kashyap May 03 '16 at 06:28
0

I am posting answer, because it is nice!! Special thanks to Mr.Francesco and Mr.EbraHim.

!/usr/bin/env python

Import the datetime

from datetime import datetime import re

Create two datetime object for limit 1 and limit 2 as dt1 and dt2 respectively

dt1 = datetime.strptime("00:00:00","%H:%M:%S").time()

dt2 = datetime.strptime("03:59:59","%H:%M:%S").time()

Create compiler for regular expression

time_re = re.compile(r'(\d+:\d+:\d+)')

pattern = re.compile('(\d{4}-\d{2}-\d{2}) (\d{2}:\d{2}:\d{2},\d{3}).[(INIT)].[netkv_restser: (.?)].*[req_id: (.?)]')

set the template for output

template = '[Date: %s] [Time: %s] [%s] [netkv_restser: %s] [req_id: %s]'

for line in open("sample.log", "r"):

    match = time_re.search(line) #Search time format for each line
    matchtime = match.group(1)
    #Time format match
    dt_match = datetime.strptime(matchtime, '%H:%M:%S').time() 
    if dt_match >= dt1 and dt_match <= dt2: #time limit is set
        #print value
        detail = re.match(pattern, line)
        if detail:
            print template % detail.groups()
Community
  • 1
  • 1
Ganesh Kashyap
  • 1
  • 7