Extracting strings from String in Powershell

Question

I have a string long string: its a teamcity buildLog This is a buildLog from teamcity.

[11:27:30] :     [Step 5/5] 15:27:30 INFO: Average times: total 0.455, latency 0.455, connect 0.004
[11:27:30] :     [Step 5/5] 15:27:30 INFO: Percentiles:
[11:27:30] :     [Step 5/5] +---------------+---------------+
[11:27:30] :     [Step 5/5] | Percentile, % | Resp. Time, s |
[11:27:30] :     [Step 5/5] +---------------+---------------+
[11:27:30] :     [Step 5/5] |           0.0 |         0.021 |
[11:27:30] :     [Step 5/5] |          50.0 |         0.103 |
[11:27:30] :     [Step 5/5] |          90.0 |         1.166 |
[11:27:30] :     [Step 5/5] |          95.0 |          2.27 |
[11:27:30] :     [Step 5/5] |          99.0 |          2.77 |
[11:27:30] :     [Step 5/5] |          99.9 |         6.996 |
[11:27:30] :     [Step 5/5] |         100.0 |        10.312 |
[11:27:30] :     [Step 5/5] +---------------+---------------+
[11:27:30] :     [Step 5/5] 15:27:30 INFO: Request label stats:
[11:27:30] :     [Step 5/5] +------------------------------------------+--------+---------+--------+-------------+
[11:27:30] :     [Step 5/5] | label                                    | status |    succ | avg_rt | error       |
[11:27:30] :     [Step 5/5] +------------------------------------------+--------+---------+--------+-------------+
[11:27:30] :     [Step 5/5] | Activity History                         |   OK   | 100.00% |  1.608 |             |
[11:27:30] :     [Step 5/5] | Asset Allocation                         |   OK   | 100.00% |  0.100 |             |
[11:27:30] :     [Step 5/5] | Dashboard Cards and Employee Information |   OK   | 100.00% |  0.255 |             |
[11:27:30] :     [Step 5/5] | Fund Details                             |   OK   | 100.00% |  0.825 |             |
[11:27:30] :     [Step 5/5] | Investments                              |   OK   | 100.00% |  0.132 |             |
[11:27:30] :     [Step 5/5] | Minimum Version                          |   OK   | 100.00% |  0.032 |             |
[11:27:30] :     [Step 5/5] | Rate of Return                           |   OK   | 100.00% |  0.047 |             |
[11:27:30] :     [Step 5/5] | Retirement Outlook Card                  |   OK   | 100.00% |  1.166 |             |
[11:27:30] :     [Step 5/5] | Retirement Outlook Full                  |   OK   | 100.00% |  1.160 |             |
[11:27:30] :     [Step 5/5] | Savings Rate                             |   OK   | 100.00% |  0.112 |             |
[11:27:30] :     [Step 5/5] | Secure Auth Login                        |  FAIL  |  98.58% |  0.207 | Bad Request |
[11:27:30] :     [Step 5/5] | Validate Savings Rate Change             |   OK   | 100.00% |  0.127 |             |
[11:27:30] :     [Step 5/5] | Vested Balance                           |   OK   | 100.00% |  0.157 |             |
[11:27:30] :     [Step 5/5] +------------------------------------------+--------+---------+--------+-------------+
[11:27:35] :     [Step 5/5] 15:27:35 INFO: Ending data feeding...
[11:27:36] :     [Step 5/5] 15:27:36 INFO: Online report link: https://a.blazemeter.com/app/#/masters/36669958

From above build Log, I have to fetch Percentiles Table , Request Label Stats table and Online Report Link

I tried below code but it is returning none:

$firststring = "Percentiles:"
$secondstring = "Request label stats:"
$pattern =  "$firststring(.*?)$secondstring"
$result = [regex]::Match($file,$pattern).Groups[1].Value
$result >> returns none

and below code to get the strings.

$Regex = [Regex]::new("(?<=Percentiles:)(.*)(?=Request label stats:)")
$Match = $Regex.Match($status)
if($Match.Success)
{
  $Match.Value
}

This also returns none. any help would be really appreciated.

Did you really mean to tag this with [tag:powershell-3.0]? – codewario Oct 15 '21 at 16:07 — codewario, Oct 15 '21 at 16:07
But .. that's basically all there is in the string anyway. – marsze Oct 15 '21 at 16:56 — marsze, Oct 15 '21 at 16:56

codewario · Accepted Answer · 2021-10-15T17:05:56.333

I can't check PowerShell 3.0 for you. But the following works in Windows PowerShell 5.1. I've got two solutions, one which includes the first info line as part of the match and one that does not.

# Set up your strings to look for
$firststring = [regex]::Escape("Percentiles:")
$secondstring = [regex]::Escape("Request label stats:")

# Pattern which includes the line with $firststring
$withInfoPattern = ".*$firststring(.*\n)*(?=.*$secondstring)"

# Pattern which omits the line with $firststring
$withoutInfoPattern = "(?<=$firststring\s+)(.*\n)*(?=.*$secondstring)"

# We will use $withInfoPattern in this example but you could also use $withoutInfoPattern
# This assumes your content string is in a variable called $content
$matchedContent = if( $content -match $withInfoPattern ) {
  $matches[0]
}

This shows how to match on your first case, the Percentiles table data. To grab the second table and report link you should be able to use the code above and explanations below to extract this data as well as a learning exercise.

At this point $matchedContent will contain the output you are looking for. You can also match using $withoutInfoPattern instead if you don't want the first line of the match to be returned. I'll explain this below:

While not required in this case, it's good practice to put strings that will be inserted into your regex pattern through [regex]::Escape(string). This is a good practice that will prevent you from inadvertently forgetting to escape strings with special regex characters in the future.
$withInfoPattern matches on any line with or without characters preceeding $firststring. $withoutInfoPattern instead uses a positive-lookbehind to ensure what is matched on proceeds an occurrence of $firstString.
(.*\n)* uses a capturing group to match on any or no characters (this is what .* means), followed by a newline. Newlines are not matched on by default with . and this behavior can't be altered with the -match operator. The trailing * looks for any or no instances of the preceeding capture group.
- -match normally won't match across newlines. Using (.*\n)* in this way gets around that limitation.
(?=.*$secondString) is a positive lookahead to make sure that the pattern main pattern preceeds the pattern specified in the lookahead. .*$secondString will match starting with the start of the line, so we are looking for any characters followed by $secondString on that line.
Invoke the -match operator to look for $withInfoPattern (or $withoutInfoPattern) within $content. This assumes the string you are searching for is stored in a variable called $content as a string (not an array of strings as Get-Content will do by default.
- You can use $content = Get-Content -Raw to read the file as a single string with newlines or join the $content array with the newline character like so before matching on it: $content -join "`n"
-match will return $true or $false. If $true, you can get the matched content from the automatic variable $matches in the first index. As arrays use zero-based indices, this translates to $matches[0].

Additional Regex Tips

I highly recommend using the site https://regexr.com/ to test your expressions in. While this only supports JavaScript and PCRE engines, both are close enough to the .NET regex engine to where it's often not an issue. Even if you find some expressions don't work, it explains the different regex tokens very well and as you develop an expression it will explain in the bottom pane what each character sequence means.

Daniel · Answer 2 · 2021-10-15T19:53:10.367

Once you have the regex patterns down, you can use switch to run through the log lines and build objects from your tables. Below I've broken this down into separate functions.

$log = @'
[11:27:30] :     [Step 5/5] 15:27:30 INFO: Average times: total 0.455, latency 0.455, connect 0.004
[11:27:30] :     [Step 5/5] 15:27:30 INFO: Percentiles:
[11:27:30] :     [Step 5/5] +---------------+---------------+
[11:27:30] :     [Step 5/5] | Percentile, % | Resp. Time, s |
[11:27:30] :     [Step 5/5] +---------------+---------------+
[11:27:30] :     [Step 5/5] |           0.0 |         0.021 |
[11:27:30] :     [Step 5/5] |          50.0 |         0.103 |
[11:27:30] :     [Step 5/5] |          90.0 |         1.166 |
[11:27:30] :     [Step 5/5] |          95.0 |          2.27 |
[11:27:30] :     [Step 5/5] |          99.0 |          2.77 |
[11:27:30] :     [Step 5/5] |          99.9 |         6.996 |
[11:27:30] :     [Step 5/5] |         100.0 |        10.312 |
[11:27:30] :     [Step 5/5] +---------------+---------------+
[11:27:30] :     [Step 5/5] 15:27:30 INFO: Request label stats:
[11:27:30] :     [Step 5/5] +------------------------------------------+--------+---------+--------+-------------+
[11:27:30] :     [Step 5/5] | label                                    | status |    succ | avg_rt | error       |
[11:27:30] :     [Step 5/5] +------------------------------------------+--------+---------+--------+-------------+
[11:27:30] :     [Step 5/5] | Activity History                         |   OK   | 100.00% |  1.608 |             |
[11:27:30] :     [Step 5/5] | Asset Allocation                         |   OK   | 100.00% |  0.100 |             |
[11:27:30] :     [Step 5/5] | Dashboard Cards and Employee Information |   OK   | 100.00% |  0.255 |             |
[11:27:30] :     [Step 5/5] | Fund Details                             |   OK   | 100.00% |  0.825 |             |
[11:27:30] :     [Step 5/5] | Investments                              |   OK   | 100.00% |  0.132 |             |
[11:27:30] :     [Step 5/5] | Minimum Version                          |   OK   | 100.00% |  0.032 |             |
[11:27:30] :     [Step 5/5] | Rate of Return                           |   OK   | 100.00% |  0.047 |             |
[11:27:30] :     [Step 5/5] | Retirement Outlook Card                  |   OK   | 100.00% |  1.166 |             |
[11:27:30] :     [Step 5/5] | Retirement Outlook Full                  |   OK   | 100.00% |  1.160 |             |
[11:27:30] :     [Step 5/5] | Savings Rate                             |   OK   | 100.00% |  0.112 |             |
[11:27:30] :     [Step 5/5] | Secure Auth Login                        |  FAIL  |  98.58% |  0.207 | Bad Request |
[11:27:30] :     [Step 5/5] | Validate Savings Rate Change             |   OK   | 100.00% |  0.127 |             |
[11:27:30] :     [Step 5/5] | Vested Balance                           |   OK   | 100.00% |  0.157 |             |
[11:27:30] :     [Step 5/5] +------------------------------------------+--------+---------+--------+-------------+
[11:27:35] :     [Step 5/5] 15:27:35 INFO: Ending data feeding...
[11:27:36] :     [Step 5/5] 15:27:36 INFO: Online report link: https://a.blazemeter.com/app/#/masters/36669958
'@ -split '\r?\n'

function get-percentiles {
    param()
    $headerPattern = '\| Percentile'
    $endPattern = '^[^|+]*$'
    $inTable = $false
    switch -regex ($log) {
        $headerPattern {
            $inTable = $true
            continue
        }
        $endPattern {
            if ($inTable) { break } else { continue }
        }
        '\|([^|]+)\|([^|]+).*$' {
            if ($inTable) {
                [PSCustomObject]@{
                    Percentile = $Matches[1].Trim()
                    Time       = $Matches[2].Trim()
                }
            }
        }
    }
}

function get-labeltable {
    param()
    $headerPattern = '\| label'
    $endPattern = '^[^|+]*$'
    $inTable = $false
    switch -regex ($log) {
        $headerPattern {
            $inTable = $true
            continue
        }
        $endPattern {
            if ($inTable) { break } else { continue }
        }
        '\|([^|]+)\|([^|]+)\|([^|]+)\|([^|]+)\|([^|]+).*$' {
            if ($inTable) {
                [PSCustomObject]@{
                    Label  = $Matches[1].Trim()
                    Status = $Matches[2].Trim()
                    Succ   = $Matches[3].Trim()
                    AvgRT  = $Matches[4].Trim()
                    Error  = $Matches[5].Trim()
                }
            }
        }
    }
}

get-percentiles | Format-Table
get-labeltable | Format-Table

Output

Percentile Time
---------- ----
0.0        0.021
50.0       0.103
90.0       1.166
95.0       2.27
99.0       2.77
99.9       6.996
100.0      10.312


Label                                    Status Succ    AvgRT Error
-----                                    ------ ----    ----- -----
Activity History                         OK     100.00% 1.608
Asset Allocation                         OK     100.00% 0.100
Dashboard Cards and Employee Information OK     100.00% 0.255
Fund Details                             OK     100.00% 0.825
Investments                              OK     100.00% 0.132
Minimum Version                          OK     100.00% 0.032
Rate of Return                           OK     100.00% 0.047
Retirement Outlook Card                  OK     100.00% 1.166
Retirement Outlook Full                  OK     100.00% 1.160
Savings Rate                             OK     100.00% 0.112 
Secure Auth Login                        FAIL   98.58%  0.207 Bad Request
Validate Savings Rate Change             OK     100.00% 0.127
Vested Balance                           OK     100.00% 0.157

Thank you @Daniel. It worked. Really appreciate your time and efforts. — born2Learn, Oct 15 '21 at 17:50

Theo · Answer 3 · 2021-10-17T12:37:49.723

I know this question has already been answered, but here's my two cents:

$log = @"
[11:27:30] :     [Step 5/5] 15:27:30 INFO: Average times: total 0.455, latency 0.455, connect 0.004
[11:27:30] :     [Step 5/5] 15:27:30 INFO: Percentiles:
[11:27:30] :     [Step 5/5] +---------------+---------------+
[11:27:30] :     [Step 5/5] | Percentile, % | Resp. Time, s |
[11:27:30] :     [Step 5/5] +---------------+---------------+
[11:27:30] :     [Step 5/5] |           0.0 |         0.021 |
[11:27:30] :     [Step 5/5] |          50.0 |         0.103 |
[11:27:30] :     [Step 5/5] |          90.0 |         1.166 |
[11:27:30] :     [Step 5/5] |          95.0 |          2.27 |
[11:27:30] :     [Step 5/5] |          99.0 |          2.77 |
[11:27:30] :     [Step 5/5] |          99.9 |         6.996 |
[11:27:30] :     [Step 5/5] |         100.0 |        10.312 |
[11:27:30] :     [Step 5/5] +---------------+---------------+
[11:27:30] :     [Step 5/5] 15:27:30 INFO: Request label stats:
[11:27:30] :     [Step 5/5] +------------------------------------------+--------+---------+--------+-------------+
[11:27:30] :     [Step 5/5] | label                                    | status |    succ | avg_rt | error       |
[11:27:30] :     [Step 5/5] +------------------------------------------+--------+---------+--------+-------------+
[11:27:30] :     [Step 5/5] | Activity History                         |   OK   | 100.00% |  1.608 |             |
[11:27:30] :     [Step 5/5] | Asset Allocation                         |   OK   | 100.00% |  0.100 |             |
[11:27:30] :     [Step 5/5] | Dashboard Cards and Employee Information |   OK   | 100.00% |  0.255 |             |
[11:27:30] :     [Step 5/5] | Fund Details                             |   OK   | 100.00% |  0.825 |             |
[11:27:30] :     [Step 5/5] | Investments                              |   OK   | 100.00% |  0.132 |             |
[11:27:30] :     [Step 5/5] | Minimum Version                          |   OK   | 100.00% |  0.032 |             |
[11:27:30] :     [Step 5/5] | Rate of Return                           |   OK   | 100.00% |  0.047 |             |
[11:27:30] :     [Step 5/5] | Retirement Outlook Card                  |   OK   | 100.00% |  1.166 |             |
[11:27:30] :     [Step 5/5] | Retirement Outlook Full                  |   OK   | 100.00% |  1.160 |             |
[11:27:30] :     [Step 5/5] | Savings Rate                             |   OK   | 100.00% |  0.112 |             |
[11:27:30] :     [Step 5/5] | Secure Auth Login                        |  FAIL  |  98.58% |  0.207 | Bad Request |
[11:27:30] :     [Step 5/5] | Validate Savings Rate Change             |   OK   | 100.00% |  0.127 |             |
[11:27:30] :     [Step 5/5] | Vested Balance                           |   OK   | 100.00% |  0.157 |             |
[11:27:30] :     [Step 5/5] +------------------------------------------+--------+---------+--------+-------------+
[11:27:35] :     [Step 5/5] 15:27:35 INFO: Ending data feeding...
[11:27:36] :     [Step 5/5] 15:27:36 INFO: Online report link: https://a.blazemeter.com/app/#/masters/36669958

"@

# first parse out the link
$link = ([regex]'(?m)Online report link:\s(.*)$').Match($log).Groups[1].Value

# next remove all except the tables themselves and split into separate lines
$log = $log -replace '(?m)^[^|+]+' -split '\r?\n'

# create two List objects to capture the tables as PsObjects
$percentList = [System.Collections.Generic.List[object]]::new()
$requestList = [System.Collections.Generic.List[object]]::new()

# use switch to loop over the lines
$percentTable = $requestTable = $false
switch -Regex ($log) {
    '^\|\s(Percentile|label)'  { 
        $percentTable = ($matches[1] -eq 'Percentile')
        $requestTable = !$percentTable
        $headers = ($_.Trim("|") -split '\|').Trim()
        continue
    }
    '^\|.*\|$' {
        $tempHash = [ordered]@{}
        $values = ($_.Trim("|") -split '\|').Trim()
        for ($i = 0; $i -lt $headers.Count; $i++) {
            # if you want the numeric values as [double] instead of [string] in the resulting objects, use:
            # $tempHash[$headers[$i]] = if ($values[$i] -as [double]) {[double]$values[$i]} else {$values[$i]}
            $tempHash[$headers[$i]] = $values[$i]
        }
        if ($percentTable) { 
            $percentList.Add([PscustomObject]$tempHash) 
        } 
        else { 
            $requestList.Add([PscustomObject]$tempHash)
        }
    }
}

# show what we have on screen
$percentList | Format-Table -AutoSize
$requestList | Format-Table -AutoSize
$link

Output:

Percentile, % Resp. Time, s
------------- -------------
0.0           0.021        
50.0          0.103        
90.0          1.166        
95.0          2.27         
99.0          2.77         
99.9          6.996        
100.0         10.312       



label                                    status succ    avg_rt error      
-----                                    ------ ----    ------ -----      
Activity History                         OK     100.00% 1.608             
Asset Allocation                         OK     100.00% 0.100             
Dashboard Cards and Employee Information OK     100.00% 0.255             
Fund Details                             OK     100.00% 0.825             
Investments                              OK     100.00% 0.132             
Minimum Version                          OK     100.00% 0.032             
Rate of Return                           OK     100.00% 0.047             
Retirement Outlook Card                  OK     100.00% 1.166             
Retirement Outlook Full                  OK     100.00% 1.160             
Savings Rate                             OK     100.00% 0.112             
Secure Auth Login                        FAIL   98.58%  0.207  Bad Request
Validate Savings Rate Change             OK     100.00% 0.127             
Vested Balance                           OK     100.00% 0.157             


https://a.blazemeter.com/app/#/masters/36669958

Thank you for all your efforts @Theo! – born2Learn Oct 18 '21 at 18:31 — born2Learn, Oct 18 '21 at 18:31

Extracting strings from String in Powershell

3 Answers3

Additional Regex Tips