Utilize Results from Synchronized Hashtable (Runspacepool 6000+ clients)

Question

Adapting a script to do multiple functions, starting with test-connection to gather data, will be hitting 6000+ machines so I am using RunspacePools adapted from the below site;

http://learn-powershell.net/2013/04/19/sharing-variables-and-live-objects-between-powershell-runspaces/

The data comes out as below, I would like to get it sorted into an array (I think that's the terminology), so I can sort the data via results. This will be adapted to multiple other functions pulling anything from Serial Numbers to IAVM data.

Is there any way I can use the comma delimited data and have it spit the Values below into columns? IE

Name    IPAddress    ResponseTime    Subnet
x        qwe           qweeqwe        qweqwe

The added values aren't so important at the moment, just the ability to add the values and pull them.

Name                           Value                                                                                                                        
&#8212;-                           &#8212;&#8211;                                                                                                                        
x-410ZWG                \\x-DHMVV1\root\cimv2:Win32_PingStatus.Address="x-410ZWG",BufferSize=32,NoFragmentation=false,RecordRoute=0,&#8230;
x-47045Q                \\x-DHMVV1\root\cimv2:Win32_PingStatus.Address="x-47045Q",BufferSize=32,NoFragmentation=false,RecordRoute=0,&#8230;
x-440J26                \\x-DHMVV1\root\cimv2:Win32_PingStatus.Address="x-440J26",BufferSize=32,NoFragmentation=false,RecordRoute=0,&#8230;
x-410Y45                \\x-DHMVV1\root\cimv2:Win32_PingStatus.Address="x-410Y45",BufferSize=32,NoFragmentation=false,RecordRoute=0,&#8230;
x-DJKVV1                \\x-DHMVV1\root\cimv2:Win32_PingStatus.Address="x-DJKVV1",BufferSize=32,NoFragmentation=false,RecordRoute=0,&#8230;
nonexistant                                                                                                                                                 
x-DDMVV1                \\x-DHMVV1\root\cimv2:Win32_PingStatus.Address="x-DDMVV1",BufferSize=32,NoFragmentation=false,RecordRoute=0,&#8230;
x-470481                \\x-DHMVV1\root\cimv2:Win32_PingStatus.Address="x-470481",BufferSize=32,NoFragmentation=false,RecordRoute=0,&#8230;
x-DHKVV1                \\x-DHMVV1\root\cimv2:Win32_PingStatus.Address="x-DHKVV1",BufferSize=32,NoFragmentation=false,RecordRoute=0,&#8230;
x-430XXF                \\x-DHMVV1\root\cimv2:Win32_PingStatus.Address="x-430XXF",BufferSize=32,NoFragmentation=false,RecordRoute=0,&#8230;
x-DLKVV1                \\x-DHMVV1\root\cimv2:Win32_PingStatus.Address="x-DLKVV1",BufferSize=32,NoFragmentation=false,RecordRoute=0,&#8230;
x-410S86                \\x-DHMVV1\root\cimv2:Win32_PingStatus.Address="x-410S86",BufferSize=32,NoFragmentation=false,RecordRoute=0,&#8230;
x-SCH004                \\x-DHMVV1\root\cimv2:Win32_PingStatus.Address="x-SCH004",BufferSize=32,NoFragmentation=false,RecordRoute=0,&#8230;
x-431KMS                                                                                                                                             
x-440J22                \\x-DHMVV1\root\cimv2:Win32_PingStatus.Address="x-440J22",BufferSize=32,NoFragmentation=false,RecordRoute=0,&#8230;

Thank for any help!

Code currently

Function Get-RunspaceData {
    [cmdletbinding()]
    param(
        [switch]$Wait
    )
    Do {
        $more = $false         
        Foreach($runspace in $runspaces) {
            If ($runspace.Runspace.isCompleted) {
                $runspace.powershell.EndInvoke($runspace.Runspace)
                $runspace.powershell.dispose()
                $runspace.Runspace = $null
                $runspace.powershell = $null                 
            } ElseIf ($runspace.Runspace -ne $null) {
                $more = $true
            }
        }
        If ($more -AND $PSBoundParameters['Wait']) {
            Start-Sleep -Milliseconds 100
        }   
        #Clean out unused runspace jobs
        $temphash = $runspaces.clone()
        $temphash | Where {
            $_.runspace -eq $Null
        } | ForEach {
            Write-Verbose ("Removing {0}" -f $_.computer)
            $Runspaces.remove($_)
        }  
        Write-Host ("Remaining Runspace Jobs: {0}" -f ((@($runspaces | Where {$_.Runspace -ne $Null}).Count)))             
    } while ($more -AND $PSBoundParameters['Wait'])
}


#Begin
#What each runspace will do
$ScriptBlock = {
    Param ($computer,$hash)
    $Ping = test-connection $computer -count 1 -ea 0
    $hash[$Computer]= $Ping
        }

#Setup the runspace
$Script:runspaces = New-Object System.Collections.ArrayList   
# Data table for all of the runspaces
$hash = [hashtable]::Synchronized(@{})
$sessionstate = [system.management.automation.runspaces.initialsessionstate]::CreateDefault()
$runspacepool = [runspacefactory]::CreateRunspacePool(1, 100, $sessionstate, $Host)
$runspacepool.Open() 

#Process
ForEach ($Computer in $Computername) {
    #Create the powershell instance and supply the scriptblock with the other parameters 
    $powershell = [powershell]::Create().AddScript($scriptBlock).AddArgument($computer).AddArgument($hash)

    #Add the runspace into the powershell instance
    $powershell.RunspacePool = $runspacepool

    #Create a temporary collection for each runspace
    $temp = "" | Select-Object PowerShell,Runspace,Computer
    $Temp.Computer = $Computer
    $temp.PowerShell = $powershell

    #Save the handle output when calling BeginInvoke() that will be used later to end the runspace
    $temp.Runspace = $powershell.BeginInvoke()
    Write-Verbose ("Adding {0} collection" -f $temp.Computer)
    $runspaces.Add($temp) | Out-Null               
}

# Wait for all runspaces to finish
#End
Get-RunspaceData -Wait 
$stoptimer = Get-Date 
#Display info, and display in GridView
Write-Host
Write-Host "Availability check complete!" -ForegroundColor Cyan
"Execution Time: {0} Minutes" -f [math]::round(($stoptimer &#8211; $starttimer).TotalMinutes , 2)
$hash | ogv

Trying to wrap my head around why you need that synchronized hash table. You aren't sharing any data between runspaces, so there doesn't appear to be any need for it. IMHO, it would be much simpler to have the runspace scripts create PS Custom Objects from the ping results and output those, and then just pick the up from the pipeline when you do the .EndInvoke(). — mjolinor, Oct 01 '14 at 01:40
I've been searching for some way to pull the results from my commands within the runspace (new to runspaces), that was the only thing I found close to it. I'll look into having each runspace create a PS Custom Object and output to them. Any guidance as to where I can read up on it? Every time I've create PSObject it was done within the scriptblock which yielded no results, I'm guessing I placed them in the incorrect spot after reading to place it at .EndInvoke() Thanks for the help! — Chris Crill, Oct 01 '14 at 03:42
Added an answer with a sample script to demonstrate returning pipeline data to the pipeline from runspaces. — mjolinor, Oct 01 '14 at 11:17

score 2 · Accepted Answer · answered Oct 01 '14 at 11:13

When you use runspaces, you write the scriptblock for the runspace pretty much the same way you would for a function. You write whatever you want the return to be to the pipeline, and then either assign it to a variable, pipe it to another cmdlet or function, or just let it output to the console. The difference is that while the function returns it's results automatically, with the runspace they collect in the runspace output buffer and aren't returned until you do the .EndInvoke() on the runspace handle.

As a general rule, the objective of a Powershell script is (or should be) to create objects, and the objective of using the runspaces is to speed up the process by multi-threading. You could return string data from the runspaces back to the main script and then use that to create objects there, but that's going to be a single threaded process. Do your object creation in the runspace, so that it's also multi-threaded.

Here's a sample script that uses a runspace pool to do a pingsweep of a class C subnet:

Param (
 [int]$timeout = 200
 )

 $scriptPath = (Split-Path -Path $MyInvocation.MyCommand.Definition -Parent)


While (
        ($network -notmatch "\d{1,3}\.\d{1,3}\.\d{1,3}\.0") -and -not
        ($network -as [ipaddress])
       )

   { $network = read-host 'Enter network to scan (ex. 10.106.31.0)' }

$scriptblock = 
{
  Param (
   [string]$network,
   [int]$LastOctet,
   [int]$timeout
   )

  $options = new-object system.net.networkinformation.pingoptions
  $options.TTL = 128
  $options.DontFragment = $false
  $buffer=([system.text.encoding]::ASCII).getbytes('a'*32)
  $Address = $($network.trim("0")) + $LastOctet
  $ping = new-object system.net.networkinformation.ping
  $reply = $ping.Send($Address,$timeout,$buffer,$options)

  Try { $hostname = ([System.Net.Dns]::GetHostEntry($Address)).hostname }
  Catch { $hostname = 'No RDNS' }

  if ( $reply.status -eq 'Success' )
    { $ping_result = 'Yes' }

   else { $ping_result = 'No' }

  [PSCustomObject]@{
   Address = $Address
   Ping    = $ping_result
   DNS     = $hostname
   }
}

$RunspacePool = [RunspaceFactory]::CreateRunspacePool(100,100)
$RunspacePool.Open()
$Jobs = 
   foreach ( $LastOctet in 1..254 )
    {
     $Job = [powershell]::Create().
            AddScript($ScriptBlock).
            AddArgument($Network).
            AddArgument($LastOctet).
            AddArgument($Timeout)
     $Job.RunspacePool = $RunspacePool

     [PSCustomObject]@{
      Pipe = $Job
      Result = $Job.BeginInvoke()
     }
}

Write-Host 'Working..' -NoNewline

Do {
   Write-Host '.' -NoNewline
   Start-Sleep -Seconds 1
} While ( $Jobs.Result.IsCompleted -contains $false)

Write-Host ' Done! Writing output file.'
Write-host "Output file is $scriptPath\$network.Ping.csv"

$(ForEach ($Job in $Jobs)
{ $Job.Pipe.EndInvoke($Job.Result) }) |
 Export-Csv $scriptPath\$network.ping.csv -NoTypeInformation

$RunspacePool.Close()
$RunspacePool.Dispose()

The runspace script does a ping on each address, and if it gets successful ping attempts to resolve the host name from DNS. Then it builds a custom object from that data, which is output to the pipeline. At the end, those objects are returned when the .EndInvoke() is done on the runspace jobs and piped directly into Export-CSV, but it could just as easily be output to the console, or saved into a variable.

Thank you! I've adapted this and it is working wonderfully for my ping sweep, I've even adapted it for calculating MS patches. Now if only I fully understood how to establish these (study time) and how to set a time-out. But that will be another thread. Thanks for the help, you're a genius! Won't let me Up-vote, not enough reputation, but you deserve an up-vote! — Chris Crill, Oct 03 '14 at 22:14
Here's one take on enforcing a timeout: http://mjolinor.wordpress.com/2014/05/26/accurate-timeout-for-threads-in-runspace-pool/ — mjolinor, Oct 03 '14 at 22:25

Utilize Results from Synchronized Hashtable (Runspacepool 6000+ clients)

1 Answers1

Linked