Convert a robocopy log into a csv file

Question

Is there a way to get the output logs of robocopy into a csv file? A column for size copied, date, source and destination? and if I run multiple robocopys have each copy have its own line. Thanks

$Logfile = "C:\Powershell\robocopy.txt"
Clear-Content "C:\Powershell\robocopy.txt" -Force
$EmailFrom = "testing@test.com"
$EmailTo = "test@testing.com"
$EmailBody = "completed successfully. See attached log file"
$EmailSubject = "Summary"

$files = @("SCRIPT")

for($i = 0; $i -lt $files.Count; $i++){
    robocopy "C:\$($files[$i])" "C:\NEW TEST\folder\folder\$($files[$i])" /Z /e /xx /W:5 /NFL /NDL /NJH /nc /np /unilog+:$Logfile
}

Send-MailMessage -To $EmailTo -from $EmailFrom -Subject $EmailSubject -Body $EmailBody -attachment $Logfile -smtpserver 192.168.243.22 -Port 25

log of output:

------------------------------------------------------------------------------
               Total    Copied   Skipped  Mismatch    FAILED    Extras
    Dirs :         4         0         4         0         0         0
   Files :        17         0        17         0         0         0
   Bytes :    27.0 k         0    27.0 k         0         0         0
   Times :   0:00:00   0:00:00                       0:00:00   0:00:00
   Ended : Thursday, April 29, 2021 11:55:52 AM

Answer 1

There's a lot more going on here that what's in the question. See this question, and my answer.

In summary the OP wanted to add a custom summary line from each RoboCopy job to an email message.

Example:

Bytes Copied: 27.0 k on Thursday, April 29, 2021 6:15:20 PM

A few things became evident:

1. Based on the fact that he had multiple RoboCopy jobs running in a loop, and that he was using RoboCopy's /unilog+:$LogFile, it was clear that a line needed to be crafted and added to an email body per RoboCopy Job.
2. From the comments we're to look at Bytes copied. Note: that differs from the impression noted in Doug's clever approach/answer.
3. Also via the comments the OP expanded the question to export the CSV data as well as the email body.

Needless to say, but the answer got messier as the scope expanded. It did cross my mind to have him ask a new question for the CSV stuff. Ironically I didn't know about this other question. However, I wanted to make sure the 2 tasks were optimally combined. That said, please consider the below distilled examples combining suggestions and realizations from the other case:

Clear-Content "C:\Powershell\robocopy.txt" -Force
$Logfile      = "C:\PowerShell\robocopy.txt"
$CsvFile      = "C:\PowerShell\RoboCsv.txt"
$EmailFrom    = "testing@test.com"
$EmailTo      = "test@test.com"
$EmailBody    = [Collections.ArrayList]@("completed successfully. See attached log file & below summary","")
$EmailSubject = "Summary"
$CsvData      = [Collections.ArrayList]@()

$Files = @("SCRIPT")

ForEach( $File in $Files )
{
    $Source = "C:\$File"
    $Dest   =  "C:\NEW TEST\folder\folder\$File"

    $Log = robocopy $Source $Dest /Z /e /xx /W:5 /MAXAGE:2 /NFL /NDL /NJH /nc /np /unilog+:$Logfile /tee

    $Date = ( ( $log -match "^\s+Ended.+" ) -split ":", 2 )[1].Trim()
    $Line = ( $Log -match "^\s+Bytes.+" ) -split "\s+"
        
    $Copy = $Line[5]
    $Mult = $Line[4]

    $Line = "Bytes Copied: $Copy $Mult on $Date"

    [Void]$EmailBody.Add($Line)

    # For Csv output:
    [Void]$CsvData.Add(
        [PSCustomObject]@{
            SizeCopied  = $Copy
            Date        = $Date
            Source      = $Source
            Destination = $Dest
        } )    
}    

# Flip the $EmailBody array back to being a regular string.
$EmailBody = $EmailBody -join "`r`n"

Send-MailMessage -To $EmailTo -From $EmailFrom -Subject $EmailSubject -Body $EmailBody -Attachment $Logfile -SMTPserver 192.168.243.22

# Output to CSVFile
$CsvData | Export-Csv -Path $CsvFile -NoTypeInformation

Explanation:

1. Replaces traditional For loop Construct with a ForEach(...). There was no discernable reason to use the more cryptic traditional loop.
2. The key piece here is to add the /TEE parameter to the RoboCopy command. That will, send output to the console's success stream where we'll capture it in the $Log variable.
3. Because of the logging options chosen $Log will only ever have the small summary data from a given RoboCopy job. Therefore, there's no memory concern...
4. $Log should be a typical [Object[]] array, so -match will return the matching elements. So, Assuming a full fidelity log segment (more on that later), both $Line & $Date should populate no problem. $Date is split such that it will only hold the date string. And, $Line will be an array from which we can easily relate the data points to the indices.
5. Convert $Line to the desired string for the email body then add it to the $EmailBody array list.
6. Create a [PSCustomObject] with the desired properties, adding it to the $CSVData array list for later export.
7. Finally when we're out of the loop, we can convert $EmailBody to a monolithic string for use in Send-EmailMessage's -Body argument. And we'll also Export $CSVData to a CSV file.

As mentioned in the other discussion I don't particularly like accumulating arrays in this way. The alternate approach might have been to assign the loop output to $CSVData not bothering to collect the email body lines. Then when that loop is complete I could run a second post-process loop on $CSVData to compile the email body. However, the multiplier plays a role here. It was not part of the specification for the CSV data. Arguably it should be considering you wouldn't know if you were looking at KB or MB. That said, short of executing a lot more logic to common-denominate on bytes or adding another column for the multiplier, I just thought this was good enough.

Also, I didn't bother casting numeric and dates. As it currently stands all calculated data would be converted to string for both the email body line and the CSV data. Again if we were tasked with calculating a common multiplier or perhaps wanted a different date string the story might be different.

Caution

If the RoboCopy produces unexpected output some of this logic will fail. Particularly matches may not be returned! In particular this can happen if the RoboCopy job doesn't run because of an issue with the initial command, for example:

Robocopy C:\DoesNotExist C:\temp\SomeOtherFolder

Will return:

    -------------------------------------------------------------------------------
       ROBOCOPY     ::     Robust File Copy for Windows
    -------------------------------------------------------------------------------
    
      Started : Friday, April 30, 2021 8:06:46 PM
       Source : c:\DoesNotExist\
         Dest : C:\temp\SomeOtherFolder\
    
        Files : *.*
    
      Options : *.* /DCOPY:DA /COPY:DAT /R:1000000 /W:30
    
    ------------------------------------------------------------------------------

Obviously, there are no lines starting with "Bytes" or "Ended".

I have a feeling RoboCopy is being used incorrectly:

It is a common error to specify source & destination file paths as the first 2 positional arguments. However RoboCopy is designed to copy folder structures, so the first 2 arguments should be directories!

RoboCopy Help:

Usage :: ROBOCOPY source destination [file [file]...] [options]

The fact that we're drawing from an array named $Files indicates a problem. Moreover, $File is being concatenated onto an existing path. Also, the discussion attached to the other answer shows exactly the kinds of no match and null value issues you'd expect.

Assuming I'm correct the RoboCopy Command should look more like the below example:

Clear-Content "C:\Powershell\robocopy.txt" -Force
$Logfile      = "C:\PowerShell\robocopy.txt"
$CsvFile      = "C:\PowerShell\RoboCsv.txt"
$EmailFrom    = "testing@test.com"
$EmailTo      = "test@test.com"
$EmailBody    = [Collections.ArrayList]@("completed successfully. See attached log file & below summary","")
$EmailSubject = "Summary"
$CsvData      = [Collections.ArrayList]@()

$Files = @("SCRIPT")

ForEach( $File in $Files )
{
    $Source  = "C:\"
    $Dest   =  "C:\NEW TEST\folder\folder\"

    $Log = robocopy $Source $Dest $File /Z /e /xx /W:5 /MAXAGE:2 /NFL /NDL /NJH /nc /np /unilog+:$Logfile /tee

    $Date = ( ( $log -match "^\s+Ended.+" ) -split ":", 2 )[1].Trim()
    $Line = ( $Log -match "^\s+Bytes.+" ) -split "\s+"
        
    $Copy = $Line[5]
    $Mult = $Line[4]

    $Line = "Bytes Copied: $Copy $Mult on $Date"

    [Void]$EmailBody.Add($Line)

    # For Csv output:
    [Void]$CsvData.Add(
        [PSCustomObject]@{
            SizeCopied  = $Copy
            Date        = $Date
            Source      = (Join-Path $Source $File)
            Destination = (Join-Path $Dest $File)
        } )
}

# Flip the $EmailBody array back to being a regular string.
$EmailBody = $EmailBody -join "`r`n"

Send-MailMessage -To $EmailTo -From $EmailFrom -Subject $EmailSubject -Body $EmailBody -Attachment $Logfile -SMTPserver 192.168.243.22

# Output to CSVFile
$CsvData | Export-Csv -Path $CsvFile -NoTypeInformation

Note: This required a complete example because $Source & $Dest changed therefore the assignments in the [PSCustomObject] declaration also changed.

If I'm correct that the RoboCopy syntax is incorrect it begs another totally different question; Why are we using RoboCopy at all. Don't get me wrong I love RoboCopy, but it's overkill to simply copy 1 file at a time. Furthermore, I can think of more eloquent code patterns using typical Copy-Item complete with similar email & reporting.

Answer 2

If I understood your comment correct, your "Size Copied" should be the number of files copied. You could use a switch statement with some regex to read the file and grab the two pieces of information you desire. Combine that with your source and destination.

for($i = 0; $i -lt $files.Count; $i++){

$source = "C:\$($files[$i])"
$destination = "C:\NEW TEST\folder\folder\$($files[$i])"

    robocopy $source $destination /Z /e /xx /W:5 /NFL /NDL /NJH /nc /np /unilog+:$Logfile

    $copied,$date = switch -Regex -File $Logfile {
        'Files.+?\d+\s+(\d+)' {[int]$Matches.1}
        'Ended.*:\s+(.+$)' {[DateTime]$matches.1}
    }

    [PSCustomObject]@{
        'Files Copied' = $copied
        Date           = $date
        Source         = $source
        Destination    = $destination
    }
}