How handle errors with copy-item

Question

I have a backup script that copies several directories of files to a backup location. Unfortunately, not all of the files in the folder are accessible. What the history is, is that they were archived, and what's left is a filename with a grey x on it. When I try to copy it, I get the following message:

Copy-Item : Access to the path 'E:\Backup\Loc\DO\zOLD_Under Review for NOT USED_keep for now\2006-06\N.doc' is denied.
At C:\Users\me\Documents\powershellFiles\Backup.ps1:13 char:4
+    Copy-Item -Path $SourcePath -Destination $DestinationPath -Force - ...
+    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : PermissionDenied: (N.doc:FileInfo) [Copy-Item], UnauthorizedAccessException
    + FullyQualifiedErrorId : CopyDirectoryInfoItemUnauthorizedAccessError,Microsoft.PowerShell.Commands.CopyItemCommand

Yes, it's complaining at the To Location, not the From Location. I have opened up the directory/files so it's not read only, but still get this.

Also, getting through these copy errors takes a really long time. If I could avoid trying to copy these files in the first place, it would be much quicker. There's probably 200 of these files that are archived.

However, I'm copying the folder, not the filenames individually. There are ones that aren't archived in that folder. There isn't a plan to clean them up. I'm trying to identify when an error occurs, but it's only hitting my breakpoint if $error.Exception -ne $null statement after it writes the errors to screen and takes forever failing (see comment).

Any idea how I can either filter out the ones that are archived, or grab them and check them against an array or list so I don't get an error message? I haven't figured out how to find them as they happen since it's copying the entire folder.

I was looking at error checking during copy-item but I don't see how to apply that to my issue.

This is the copy method:

function CopyFileToFolderUNC($SourcePath, $DestinationPath){
   if(-Not (Test-Path $SourcePath)) 
   {
      $global:ErrorStrings.Add("Exception: No such path, $SourcePath;;  ")
      write-output  "++ Error: An error occured during copy operation. No such path, $SourcePath ++"
   }
   Copy-Item -Path $SourcePath -Destination $DestinationPath -Force -Recurse -errorVariable errors 
   foreach($error in $errors)
   {
        if ($error.Exception -ne $null)
        {
            $global:ErrorStrings.Add("Exception: $($error.Exception);;  ")
            write-output  "++ Error: An error occured during copy operation. Exception: $($error.Exception) ++" #this breakpoint is hit after giving errors on screen and taking a long time/failing to copy files it can't reach
        }
        write-output  "Error: An error occured during copy operation. Exception: $($error.Exception)"
    }
}

This is my latest try based on what was suggested by @theo, but it's trying to copy files I hadn't tested attributes for a file I can copy, just the dir above it:

function CopyFileToFolderUNC($SourcePath, $DestinationPath, $exclude){
   if(-Not (Test-Path $SourcePath )) #-PathType Container
   {
      $global:ErrorStrings.Add("Exception: No such path, $SourcePath;;  ")
      write-output  "++ Error: An error occured during copy operation. No such path, $SourcePath ++"
   }
   #$tempFileName = [System.IO.Path]::GetFileName($SourcePath)
   #$tempDestFileNamePath = "$DestinationPath\$tempFileName"
   Get-ChildItem -Path $SourcePath -Recurse -Force | ForEach {$_} {
      #test if maybe we are dealing with an off-line file here
      #or use the enum name 'Offline'
      # or use the numeric value: 4096
      #$oldAttribs = $null
      $attr = $_.Attributes.value__
      write-output  "++ $_ $attr ++"
      if (($_.Attributes -eq [System.IO.FileAttributes]::Offline) -or ($_.Attributes.value__ -eq "4096")) {
         $_.Attributes=[System.IO.FileAttributes]::Normal
         #$oldAttribs = $_.Attributes
         #make it a 'normal' file with only the Archive bit set
         #$_.Attributes = [System.IO.FileAttributes]::Archive
         #log that the file was an issue and copy the other ones
         $global:ErrorStrings.Add("Found offline file in backup dir, $_. Logging info and not copying this file. Offline. Please investigate.;;  ")
         write-output  "++ Error: An error occured during copy operation. No such path or file, Offline $_ ++"
      } #if
      elseif(($_.Attributes -eq [System.IO.Fileattributes]::Archive) -or ($_.Attributes.value__ -eq "32")) {
         $global:ErrorStrings.Add("Found offline file in backup dir, $_. Logging info and not copying this file. Archive. Please investigate.;;  ")
         write-output  "++ Error: An error occured during copy operation. No such path or file, Archive $_ ++"
      } #elseif
      elseif(($_.Attributes -eq [System.IO.Fileattributes]::SparseFile) -or ($_.Attributes.value__ -eq "512")) {
         $global:ErrorStrings.Add("Found offline file in backup dir, $_. Logging info and not copying this file. SparseFile. Please investigate.;;  ")
         write-output  "++ Error: An error occured during copy operation. No such path or file, SparseFile $_ ++"
      } #elseif
      elseif(($_.Attributes -eq [System.IO.Fileattributes]::ReparsePoint) -or ($_.Attributes.value__ -eq "1024")) {
         $global:ErrorStrings.Add("Found offline file in backup dir, $_. Logging info and not copying this file. ReparsePoint. Please investigate.;;  ")
         write-output  "++ Error: An error occured during copy operation. No such path or file, ReparsePoint $_ ++"
      } #elseif
      else {

         #the file is not or no longer off-line, so proceed with the copy
         $_ | Copy-Item -Destination $DestinationPath -Force -Recurse -ErrorVariable errors
         foreach($error in $errors)
         {
           if ($error.Exception -ne $null)
           {
               $global:ErrorStrings.Add("Exception: $($error.Exception);;  ")
               write-output  "++ Error: An error occured during copy operation. Exception: $($error.Exception) ++"
           }
           write-output  "Error: An error occured during copy operation. Exception: $($error.Exception)"
         }
      } #else
      #if ($oldAttribs) {
      #   $_.Attributes = $oldAttribs
      #}
   } #Get-ChildItem

For example, I'm testing a dir at \\drive\folder\Forms\C Forms\ and it says it's a good attribute "16", but there's a file in that dir that the copy is trying to copy over to my dest dir and I'm seeing it has this for attributes: file.pdf Archive, SparseFile, ReparsePoint, Offline. But I'm not testing that file, the last thing I tested attributes for was the dir it's in.

that makes me think that perhaps the archived file in the source is not a file ... instead, it may be a symlink. i don't know how to tell `Copy-Item` to ignore them directly. however, you can run the command, collect the error file names, save them to a file for later use. if you use `Get-ChildItem` to build a list to send to `Copy-Item`, you can use the saved list of files to filter them out of the items sent to the copy command. — Lee_Dailey, Dec 27 '18 at 16:10
I'm thinking this might be offline files. Explorer displays these as dimmed items with a gray X overlay — Theo, Dec 28 '18 at 09:21
Please take a better look at my functions. I use `-File` with Get-Childitem. You should not decline files based upon the Archive bit, they are OK. There is no reason to check the attribute on BOTH `[System.IO.FileAttributes]` AND also the numeric value thereof. Just choose the syntax you feel comfortable with. As an aside: Using `Switch` will make cleaner code then using all those `elseif` constructions. — Theo, Jan 04 '19 at 21:30
Also, the first `if` checks the Offline bit. All tests following that test for another attribute value, so if you want logging change the error messages so they actually reflect the test. — Theo, Jan 04 '19 at 21:38
@Theo - When I do "Get-ChildItem -Path $SourcePath -File | ForEach-Object", it misses most of the files and all subdirs. When I use "Get-ChildItem -Path $SourcePath -Recurse -Depth 2 | ForEach {$_}" it hits all of the files and subdirs. It's just trying to copy all files in the subdir before I check if they are copy-able first. — Michele, Jan 07 '19 at 13:27

Theo · Accepted Answer · 2019-01-02T15:56:54.560

To me it looks like the files you describe are Off-line files.
In your function you can test if that is the case using something like this:

function CopyFileToFolderUNC($SourcePath, $DestinationPath){
    if(-Not (Test-Path $SourcePath)) {
        $global:ErrorStrings.Add("Exception: No such path, $SourcePath;;  ")
        Write-Output  "++ Error: An error occured during copy operation. No such path, $SourcePath ++"
    }

    # test if maybe we are dealing with an off-line file here
    if ((Get-Item -Path $SourcePath).Attributes -band 4096) {  # or use the enum name 'Offline'
        # or use .Net:
        # [System.IO.File]::GetAttributes($SourcePath) -band 4096
        Write-Output  "Did not copy: File '$SourcePath' is currently off-line. "
    }
    else {
        # the file is not off-line, so proceed with the copy
        Copy-Item -Path $SourcePath -Destination $DestinationPath -Force -Recurse -errorVariable errors 
        foreach($error in $errors)  {
            if ($error.Exception) {
                $global:ErrorStrings.Add("Exception: $($error.Exception);;  ")
                Write-Output  "++ Error: An error occured during copy operation. Exception: $($error.Exception) ++" #this breakpoint is hit after giving errors on screen and taking a long time/failing to copy files it can't reach
            }
            Write-Output  "Error: An error occured during copy operation. Exception: $($error.Exception)"
        }
    }
}

EDIT

From your comment I understand the function was not meant for a single file, but for all files found in a directory named $SourcePath.

In that case, here's an updated function that should do the trick:

function CopyFileToFolderUNC($SourcePath, $DestinationPath){
    if(-Not (Test-Path $SourcePath -PathType Container)) {
        $global:ErrorStrings.Add("Exception: No such path '$SourcePath'")
        Write-Output  "++ Error: An error occured during copy operation. No such path '$SourcePath' ++"
    }

    Get-ChildItem -Path $SourcePath -File | ForEach-Object {
        # test if maybe we are dealing with an off-line file here
        # or use the enum name 'Offline'
        # or use the numeric value: 4096
        if ($_.Attributes -band [System.IO.FileAttributes]::Offline) {  
            Write-Output  "Did not copy: File '$($_.FullName)' is currently off-line."
        }
        else {
            # the file is not off-line, so proceed with the copy
            $_ | Copy-Item -Destination $DestinationPath -Force -Recurse -ErrorVariable errors 
            foreach($error in $errors)  {
                if ($error.Exception) {
                    $global:ErrorStrings.Add("Exception: $($error.Exception);;  ")
                    Write-Output  "++ Error: An error occured during copy operation. Exception: $($error.Exception) ++"
                }
                Write-Output  "Error: An error occured during copy operation. Exception: $($error.Exception)"
            }
        }
    }
}

If by dealing with the files you mean you still want to copy them, use this instead:

function CopyFileToFolderUNC($SourcePath, $DestinationPath){
    if(-Not (Test-Path $SourcePath -PathType Container)) {
        $global:ErrorStrings.Add("Exception: No such path '$SourcePath'")
        Write-Output  "++ Error: An error occured during copy operation. No such path '$SourcePath' ++"
    }

    Get-ChildItem -Path $SourcePath -File | ForEach-Object {
        # test if maybe we are dealing with an off-line file here
        # or use the enum name 'Offline'
        # or use the numeric value: 4096
        $oldAttribs = $null
        if ($_.Attributes -band [System.IO.FileAttributes]::Offline) {  
            $oldAttribs = $_.Attributes
            # make it a 'normal' file with only the Archive bit set
            $_.Attributes = [System.IO.FileAttributes]::Archive
        }

        # the file is not or no longer off-line, so proceed with the copy
        $_ | Copy-Item -Destination $DestinationPath -Force -Recurse -ErrorVariable errors 
        foreach($error in $errors)  {
            if ($error.Exception) {
                $global:ErrorStrings.Add("Exception: $($error.Exception);;  ")
                Write-Output  "++ Error: An error occured during copy operation. Exception: $($error.Exception) ++"
            }
            Write-Output  "Error: An error occured during copy operation. Exception: $($error.Exception)"
        }

        # if you want the attributes in the original file to be restored, use:
        if ($oldAttribs) {
            $_.Attributes = $oldAttribs
        }
    }
}

See FileAttributes Enum for all possible attribute values.

Hope that helps

Thanks for the great idea! Since the $SourcePath is a directory and it's files in the dir that may be offline, I don't think if/else will work for that situation, including the write-output for the error message. Any ideas how to deal with the files in the $SourcePath directory with the issues? There's hundreds of files so I can't give them individually to the copy, I don't think. — Michele, Jan 02 '19 at 15:27
@Michele I have edited my answer to deal with all files in the `$SourcePath` folder. Cheers! — Theo, Jan 02 '19 at 15:57
Unfortunately, the dir's have subdirs, but this is a good start. I'll put what I have so far using what you suggest. Maybe you know how to test individual files in deeper subdirs. For example, I'm testing a dir at \\\drive\folder\Forms\C Forms\ and it says it's a good attribute "16", but there's a file in that dir that the copy is trying to copy over to my dest dir and I'm seeing it has this for attributes: file.pdf Archive, SparseFile, ReparsePoint, Offline. But I'm not testing that file, the last thing I tested attributes for was the dir it's in. — Michele, Jan 04 '19 at 17:56
@Michele ok, add `-Recurse` to the `Get-Childitem` command. As for **ReparsePoint**: As I understand it, these are links to other files or drives. I suggest you simply skip those. — Theo, Jan 04 '19 at 19:45
I tried adding -recurse to Get-Childitem already, as you can see in my update to the question, but it's not letting me look at the files in the subdirs before I copy them. — Michele, Jan 04 '19 at 19:58

How handle errors with copy-item

1 Answers1

Linked