Copy items from Source to Destination if they don't already exist

Question

I have a pretty basic powershell copy script that copies items from a source folder to a destination folder. However this is moving way too much data, and I'd like to check if the filename already exists so that file can be ignored. I don't need this as complex as verifying created date/checksum/etc.

Currently it's along the lines of:

Copy-Item source destination -recurse
Copy-Item source2 destination2 -recurse

I'd imagine I need to add the Test-Path cmdlet, but I'm uncertain how to implement it.

Answer 1

You could always call ROBOCOPY from PowerShell for this.

Use the /xc (exclude changed) /xn (exclude newer) and /xo (exclude older) flags:

robocopy /xc /xn /xo source destination 

This will ONLY copy those files that are not in the destination folder.

For more option type robocopy /?

Answer 2

$exclude = Get-ChildItem -recurse $dest
Copy-Item -Recurse $file $dest -Verbose -Exclude $exclude

Answer 3

While I agree that Robocopy is the best tool for something like this, I'm all for giving the customer what they asked for and it was an interesting PowerShell exercise.

This script should do just what you asked for: copy a file from Source to Destination only if it does not already exist in the Destination with a minimum of frills. Since you had the -recurse option in your example, that made for a bit more coding than just simply testing for the filename in the Destination folder.

$Source = "C:\SourceFolder"
$Destination = "C:\DestinationFolder"

Get-ChildItem $Source -Recurse | ForEach {
    $ModifiedDestination = $($_.FullName).Replace("$Source","$Destination")
    If ((Test-Path $ModifiedDestination) -eq $False) {
        Copy-Item $_.FullName $ModifiedDestination
        }
    }

Answer 4

Building off of Wai Ha Lee's post, here's an example that worked for me:

$Source = "<your path here>"
$Dest = "<your path here>"
$Exclude = Get-ChildItem -recurse $Dest

Get-ChildItem $Source -Recurse -Filter "*.pdf" | Copy-Item -Destination $Dest -Verbose -Exclude $Exclude

This builds a list to exclude, then copies any pdf in the source directory and sub-directories to the destination in a single folder...excluding the existing files. Again, this is an example from my needs, but similar to yours. Should be easy enough to tweak to your hearts content.

Answer 5

Function Copy-IfNotPresent will accept one file at a time but it's easy to loop for all files you want to copy. Here's an example:

gci c:\temp\1\*.* -Recurse -File | % { Copy-IfNotPresent -FilePath $_ -Destination "C:\temp\2\$(Resolve-Path $_ -relative)" -Verbose }

Here's the function. It will generate the folder tree if necessary. Here's the gists link: https://gist.github.com/pollusb/cd47b4afeda8edbf8943a8808c880eb8

Function Copy-IfNotPresent {
<#
    Copy file only if not present at destination.
    This is a one file at a time call. It's not meant to replace complex call like ROBOCOPY.
    Destination can be a file or folder. If it's a folder, you can use -Container to force Folder creation when not exists
#>
[CmdletBinding()]
Param (
    [Parameter(Mandatory)]
    $FilePath,

    [Parameter(Mandatory)]
    [string]$Destination,

    [switch]$Container,

    [switch]$WhatIf
)
#region validations
if ($FilePath -isnot [System.IO.FileInfo]){
    $File = Get-ChildItem $FilePath -File
} else {
    $File = $FilePath
}

if (!$File.Count){
    Write-Warning "$FilePath no file found."
    return
} elseif ($File.Count -gt 1) {
    Write-Warning "$FilePath must resolve to one file only."
    return
}
#endregion

# Destination is a folder
if ($Container -or (Test-Path -Path $Destination -PathType Container)) {
    if (!(Test-Path $Destination)) {
        New-Item -Path $Destination -ItemType Container | Out-Null
    }
    $Destination += "\$($File.Name)"
}

# Destination is a file
if (!(Test-Path $Destination)) {
    if ($WhatIf) {
        Write-Host "WhatIf:Copy-IfNotPresent $FilePath -> $Destination"
    } else {
        # Force creation of parent folder
        $Parent = Split-Path $Destination -Parent
        if (!(Test-Path $Parent)) { 
            New-Item $Parent -ItemType Container | Out-Null
        }
        Copy-Item -Path $FilePath -Destination $Destination  
        Write-Verbose "Copy-IfNotPresent $FilePath -> $Destination (is absent) copying"
    }
} else {
    Write-Verbose "Copy-IfNotPresent $Destination (is present) not copying"
}
}

Answer 6

Here is a recursive script that syncronizes 2 folders ignoring existing files:

function Copy-FilesAndFolders([string]$folderFrom, [string]$folderTo) {
    $itensFrom = Get-ChildItem $folderFrom
    foreach ($i in $itensFrom) 
    {
        if ($i.PSIsContainer)
        {
            $subFolderFrom = $folderFrom + "\" + $i.BaseName
            $subFolderTo = $folderTo + "\" + $i.BaseName
            Copy-FilesAndFolders $subFolderFrom $subFolderTo | Out-Null
        }
        else
        {
            $from = $folderFrom + "\" + $i.Name
            $to = $folderTo + "\" + $i.Name
            if (!(Test-Path $from)) # only copies non-existing files
            {
                if (!(Test-Path $folderTo)) # if folder doesn't exist, creates it
                {
                    New-Item -ItemType "directory" -Path $folderTo
                }
                Copy-Item $from $folderTo
            }           
        }
    }   
}

To call it:

Copy-FilesAndFolders "C:\FromFolder" "C:\ToFolder"

Answer 7

$source = "c:\source"

$destination = "c:\destination"

1. Create a list of files to exclude, i.e. files already existing in the destination.

$exclude = Get-Childitem -Recurse $destination | ForEach-Object { $_.FullName -replace [Regex]::Escape($destination ), "" }

2. Recursively copy all contents from the source to the destination excluding the previously collected files.

Copy-Item -Recurse -Path (Join-Path $source "*") -Destination $destination -Exclude $exclude -Force -Verbose

• (Join-Path $source "*") add a wildcard at end ensuring that you get the children of the source folder instead of the source folder itself.
• Force is used because I don't mind that there are already existing folders (results in error messages). Use with caution.
• ForEach-Object { $_.FullName -replace [Regex]::Escape($destination ), "" } transforms the existing file full names into values which can be used as Exclude parameter

Answer 8

Lots of great answers in here, here's my contribution as it relates to keeping an mp3 player in sync with a music library.

#Tom Hubbard, 10-19-2021
#Copy only new music to mp3 player, saves time by only copying items that don't exist on the destination.
#Leaving the hardcoded directories and paths in here, sometimes too much variable substitution is confusing for newer PS users.

#Gets all of the albums in the source directory such as your music library
$albumsInLibrary = gci -Directory -path "C:\users\tom\OneDrive\Music" | select -ExpandProperty Name

#Gets all of the albums of your destination folder, such as your mp3 player
$albumsOnPlayer = gci -Directory -Path "e:\" | select -ExpandProperty name

#For illustration, it will list the differences between the music library and the music player.
Compare-Object -DifferenceObject $albumsInLibrary -ReferenceObject $albumsOnPlayer

#Loop through each album in the library
foreach ($album in $albumsInLibrary)
{
    #Check to see if the music player contains this directory from the music library
    if ($albumsOnPlayer -notcontains $album)
{
    #If the album doesn't exist on the music player, copy it and it's child items from the library to the player
    write-host "$album is not on music player, copying to music player" -ForegroundColor Cyan
    Copy-Item -path "C:\users\Tom\OneDrive\music\$album" -Recurse -Destination e:\$album
}
}