get-itemPropertyValue doesn't bind Path when using \\?\UNC\ (long unc path) in start-job scriptblock

Question

I'm trying to multithread this

$var3 = Get-ItemPropertyValue $var2 -Name fullName,CreationTime,lastAccessTime,LastWriteTime,length;

by replacing it with:

$var3 = forEach($file in $var2) {
        start-job -name "bla" -scriptblock {Get-ItemPropertyValue $file.FullName -Name fullName,LastWriteTime,CreationTime,lastAccessTime,length} | out-null
    }

where $var2 holds a list of files retrieved via gci.

This works for normal paths, but not for long UNC Paths prefixed by \\?\UNC\.

The long path itself works fine with the get-itemPropertyValue '\\?\UNC\some long path in Webdav' but doesn't as soon as I put it into the scriptBlock from start-job.

The errormessage says (in german):

"Das Argument für den Parameter "Path" kann nicht überprüft werden. Das Argument ist NULL oder leer. Geben Sie ein Argument an, das nicht NULL oder leer ist, und führen Sie den Befehl erneut aus. + CategoryInfo : InvalidData: (:) [Get-ItemPropertyValue], ParameterBindingValidationException + FullyQualifiedErrorId : ParameterArgumentValidationError,Microsoft.PowerShell.Commands.GetItemPropertyValueCommand + PSComputerName : localhost

When I try to bind it with get-itemPropertyValue -LocalPath '\\?\UNC\some long path in Webdav' it also fails but with this Errormessage:

Das Argument kann nicht an den Parameter "LiteralPath" gebunden werden, da es NULL ist. + CategoryInfo : InvalidData: (:) [Get-ItemPropertyValue], ParameterBindingValidationException + FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.GetItemPropertyValueCommand + PSComputerName : localhost

Can anyone please check, if it is true, that this doesn't work, or show a running example otherwise? I'm running PS Version 5.1 and have to use this version.

Thanks in advance.

Answer 1

As in my comment, the scope of a job is not the same as your current scope, meaning that, the jobs you start cannot see variables defined outside it's scope (i.e.: $file). You need to pass the variables to it's scope, either with $using:foo or with -ArgumentList:

$props = 'FullName','LastWriteTime','CreationTime','LastAccessTime','Length'

$var3 = forEach($file in (Get-ChildItem -File))
{
    Start-Job -Name "bla" -ScriptBlock {
        Get-ItemPropertyValue $using:file.FullName -Name $using:props
    }
}

$var3 | Receive-Job -Wait -AutoRemoveJob

# OR

$var3 = forEach($file in (Get-ChildItem -File))
{
    Start-Job -Name "bla" -ScriptBlock {
        param($file, $props)

        Get-ItemPropertyValue $file.FullName -Name $props
    } -ArgumentList $file, $props
}

$var3 | Receive-Job -Wait -AutoRemoveJob

As for @TheMadTechnician's comment, he is right, there is no need to use Get-ItemProperty if you're already calling Get-ChildItem but for the sake of explaining what I meant in my next comment: ...divide the array you're looping through into chunks and passing that chunk to a job instead of file by file as it..., starting a job for each file would be not only many times slower than a normal foreach loop but also would consume a lot of memory. Start-Job in general is slower than a linear loop, if you're looking for a multithread alternative, you should be looking at either RunSpace or Start-ThreadJob from the ThreadJob Module. I have done some testing comparing the performance Linear Loops vs ThreadJob vs Runspace when looping through local directories and files, you can download the script from my GitHub if you're interested (bear in mind, it requires having the ThreadJob module installed).

• To show you a simple example of what I meant by passing chunk to a job, first store all directories in a variable and define the number of threads you want to use (number of jobs running at the same time):

$directories = Get-ChildItem . -Directory -Recurse
$numberOfThreads = 10

• Then we divide this array of directories into the the number of threads, 10 in this case:

$groupSize = [math]::Ceiling($directories.Count / $numberOfThreads)
$counter = [pscustomobject]@{ Value = 0 }
$groups = $directories | Group-Object -Property {
    [math]::Floor($counter.Value++ / $groupSize)
}

• Lastly, we pass these 10 chunks of folders to each job. Each job will be counting the number of files on each folder:

$var3 = foreach($chunk in $groups)
{
    Start-Job -ScriptBlock {
        $folders = $using:chunk.Group.FullName
        foreach($folder in $folders)
        {
            [pscustomobject]@{
                DirectoryFullName = $folder
                NumberOfFiles = (Get-ChildItem $folder -File).count
            }
        }
    }
}

$result = $var3 | Receive-Job -Wait -AutoRemoveJob |
Sort-Object NumberOfFiles -Descending |
Select-Object * -ExcludeProperty RunspaceID,PSSourceJobInstanceId

If you try this on your computer, you would be getting something like:

DirectoryFullName      NumberOfFiles
-----------------      -------------
X:\ExampleUser\Folder0          1954
X:\ExampleUser\Folder1           649
X:\ExampleUser\Folder2            64
X:\ExampleUser\Folder3            36
X:\ExampleUser\Folder4            23
X:\ExampleUser\Folder5            16
X:\ExampleUser\Folder6            15
X:\ExampleUser\Folder7            15
X:\ExampleUser\Folder8            12
X:\ExampleUser\Folder9            10