Search and group from logfile

Question

I have a logfile with many entries. Some of them begin with a date, some others not. I want to search for all entries from this/last month with "UpgradeResource] part: 3-V12345678-12-" in the line and count the results grouped by box. Actually there are 9 boxes counting from 1 to 9 but if we buy another box there will be a 10 or 11… the box-counter is always followed by -1 at the end of the line.

The lines I search for look like this:

2016-04-27 11:49:43,895 INFO  [ajp-apr-8009-exec-9] [com.xxx.shared.yyy.UpgradeResource] part: 3-V12345678-12-5-245, box: 3-V12345678-38-3-1
...
2016-04-27 11:49:43,895 INFO  [ajp-apr-8009-exec-9][com.xxx.shared.yyy.UpgradeResource] part: 3-V12345678-12-4-112, box: 3-V12345678-38-1-1

My result-output should be:

Month 03/2016:
Box 1: 10 times
Box 2: 123 times
Box 3: 65 times

Month 04/2016:
Box 1: 75 times
Box 2: 13 times
Box 3: 147 times

I am not very firm in using powershell and tried this but get errors and think I am not on the right way:

$inputfile = "C:\temp\result.txt"
$matchstring = "(?\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}).*UpgradeResource] part: 3-V12345678-12-(?.*?), box: 3-V12345678-38-(\d{1})-1"
Get-Content $inputfile | foreach { 
    if ($_ -match $matchstring) {
        "" | select @{n='Date';e={$matches.date}},@{n='Key';e={$matches.Key}},@{n='PD';e={$matches.PD}}
    }
}

The Error I get:

"(?\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}).*UpgradeResource] part:
3-V12345678-12-(?.*?), box: 3-V1001686-38-(\d{1})-1" wird analysiert -
Unbekanntes Gruppierungskonstrukt.
In C:\temp\count.ps1:16 Zeichen:6
+ if ($_ -match $matchstring)
+     ~~~~~~~~~~~~~~~~~~~~~~
     + CategoryInfo : OperationStopped: (:) [], ArgumentException
     + FullyQualifiedErrorId : System.ArgumentException

You can test your regex in a tool like https://regex101.com/ . There are some `?`in your matchstring, which are not accepted. — Martin, Apr 28 '16 at 09:16

score 0 · Answer 1 · answered Apr 28 '16 at 09:21

0

Does that fit ?

$inputfile = "C:\temp\result.txt"
$matchstring = "(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}).*UpgradeResource] part: 3-V12345678-12-(.*), box: 3-V12345678-38-(\d{1})-1"

Get-Content $inputfile | foreach { 
if ($_ -match $matchstring) {

    "" | select @{n='Date';e={$matches.1}},@{n='Key';e={$matches.2}},@{n='PD';e={$matches.3}}
}

}

Gives me that Output:

Date                                                    Key                                                     PD                                                    
----                                                    ---                                                     --                                                    
2016-04-27 11:49:43                                     5-245                                                   3                                                     
2016-04-27 11:49:43                                     4-112                                                   1

answered Apr 28 '16 at 09:21

Martin

1,853
3
15
22

Yes, this works. And now how do I count and group by PD? And how can I get results for this/last month grouped? – marchusar Apr 28 '16 at 09:38
You can add an `| group pd` to the `"" | select ..`line for grouping by PD. Grouping by month requires a little more string operation, as a `group date` will group by exact date. – Martin Apr 28 '16 at 09:42
OK, I will try more things. I now have seen that there are many entries with the same timestamp. I need to find out, how I can count only one entry with the same timestamp. – marchusar Apr 28 '16 at 10:21
You can break down the date, or just take the information `year-month` of it. – Martin Apr 28 '16 at 10:27

score 0 · Accepted Answer · answered Apr 28 '16 at 10:46

The error you're getting is because (?...) is not a valid grouping construct. If you want to use named groups (which the rest of your code suggests) the question mark must be followed by the group name in angular brackets ((?<name>...)). For a non-capturing group it must be followed by a colon ((?:...)).

See here for further information.

Your code should probably look somewhat like this:

$inputfile   = 'C:\temp\result.txt'
$matchstring = '(?<date>\d{4}-\d{2}-\d{2}) (?<time>\d{2}:\d{2}:\d{2})' +
               '.*UpgradeResource] ' +
               'part: 3-V12345678-12-(?<Key>.*?), ' +
               'box: 3-V12345678-38-(?<PD>\d{1})-1'
Get-Content $inputfile | Where-Object {
  $_ -match $matchstring
} | ForEach-Object {
  New-Object -Type PSObject -Property  @{
    'Date' = $matches.date
    'Time' = $matches.time
    'Key'  = $matches.Key
    'Box'  = 'Box ' + $matches.PD
  }
} | Group-Object Date, Box

Search and group from logfile

2 Answers2