0

Oh I have a server backing up data to a NAS, then I do replication from the source NAS to the destination NAS.

I'm looking for a way to verify that the number of sizes that have been sent to the destination NAS match those on the source NAS.

because the amount of moving data is very large, it is not possible to see one by one.

Does anyone have any suggestions and solutions regarding my problem?

Thank You

Tommy Armando
  • 27
  • 3
  • 2
    what have you tried so far? sounds like a job for [`Compare-Object`](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/compare-object?view=powershell-7.2) – Otter Dec 23 '21 at 12:22
  • using `Get-ChildItem` would gather a metadata for the files including their size `lenght` , the cmdlet would have no problem with big files but rather with a lot of files. – Vasil Nikolov Dec 23 '21 at 14:04
  • if there are LOTS of files, then `Get-ChildItem` would likely be rather slow. you could run `robocopy.exe` with the `/L` option to show what it WOULD do, and then capture the output from each location ... and finally compare the two. – Lee_Dailey Dec 23 '21 at 15:03
  • I smell an [XY](https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem). Instead of doing some brute-force scanning of source and destination (which are "very large") and subject to race condition problems, you should address _how you are replicating_ between your NASs that will guarantee consistency and warn you if there was a problem during replication. (I assume you're asking at all because you are getting inconsistent replication the way you are currently doing it.) – msanford Dec 24 '21 at 22:10
  • This really seems like a case to use Robocopy, there's a nice guide https://adamtheautomator.com/robocopy/ – ninMonkey Dec 26 '21 at 04:16

1 Answers1

1

I think your best option if you need to verify data replicated between 2 directories is to compare file hashes not file sizes.

you can use the below script to identify inconsistent files

$Dir1 = "D:\SRC"
$Dir2 = "D:\DST"
$Dir1Hash = Get-ChildItem $Dir1 -recurse -file | Get-FileHash | select hash,Path,@{n='RemotePath';e={$_.path.replace($Dir1,$Dir2)}}
$Dir2Hash = Get-ChildItem $Dir2 -recurse -File | Get-FileHash | select hash,Path,@{n='RemotePath';e={$_.path.replace($Dir2,$Dir1)}}
foreach ($item in $Dir1Hash){
$ReplicaFile = $Dir2Hash | where {$_.RemotePath -eq $item.path}
if ($ReplicaFile) {
    if ($item.Hash -ne $ReplicaFile.Hash){
        Write-host "Incorrect hash of file: $($item.Path) on Replica folder" -ForegroundColor Cyan
    }
}
else {
    Write-Output "File: $($item.Path) doesn't exist on Replica Folder"
}
}
Mahmoud Moawad
  • 697
  • 7
  • 14
  • Hii , The solution you've provided so far is working fine and as I wanted it to, however I encountered a problem like this: Get-ChildItem : The specified wildcard character pattern is not valid: [W-4X] PG-OLVM BI FAST At line:1 char:13 + $Dir1Hash = Get-ChildItem $Dir1 -recurse -file | Get-FileHash | selec ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : NotSpecified: (:) [Get-ChildItem], WildcardPatternException + FullyQualifiedErrorId : RuntimeException,Microsoft.PowerShell.Commands.GetChildItemCommand – Tommy Armando Dec 27 '21 at 10:32
  • I think this is because I used the "[ ]" sign for my folder name. I can't change my folder naming system is there a way to get around this? – Tommy Armando Dec 27 '21 at 10:37
  • hello Tommy, try to put the directories in single quote instead of double quotes, hope it helps .... (please mark the answer as answer if its really whats you wanted) – Mahmoud Moawad Dec 27 '21 at 10:46
  • Hi Mahmoud What do you mean like this? : $Dir1 = 'D:\SRC' $Dir2 = 'D:\DST' I've tried it but still get the same error, Thank You – Tommy Armando Dec 27 '21 at 11:19
  • Hi Tommy, I mean you just need to replace the variable Dir1 with you source folder I.e. $Dir1='source path' and replace $Dir1 with the destination path I.e. $Dir2='destination path' – Mahmoud Moawad Dec 28 '21 at 09:32
  • $Dir1 = 'D:\My Documents\Downloads\[D-30X]-SOURCE' $Dir2 = 'D:\[D-30X]-DESTINATION' – Tommy Armando Dec 29 '21 at 04:44