2

What I'm trying to do

The below script loops through every item in an Array of data streams and requests a summary value for output to a text file. This external request is by far the most expensive part of the process, and so I am now using a Runspacepool to run multiple (5) requests in parallel, and whichever finishes first outputs its results.

These requests all write to a synchronised hashtable, $hash, which holds a running total ($hash.counter) and tracks which thread ($hash.thread) is updating the total and a .txt output file, to avoid potential write collisions.


What isn't working

Each thread is able to update the counter easily enough $hash.counter+=$r, but when I try and Read the value into an Add-Content statement: Add-Content C:\Temp\test.txt "$hash.counter|$r|$p|$ThreadID" it adds an object reference rather than a number:

System.Collections.Hashtable+SyncHashtable.counter|123|MyStreamName|21252

And so I've ended up passing the counter through a temporary variable that can be used in the string:

[int]$t = $hash.counter+0

Add-Content C:\Temp\test.txt "$t|$r|$p|$ThreadID"

Which does output the true total:

14565423|123|MyStreamName|21252


What I'm asking

  1. Is it possible to remove this temporary variable and output directly from the hashtable? Why does the object reference have a '+' in the middle?
  2. I've had to add logic to 'lock' the hashtable to prevent data collisions. Should this be necessary? I'd been told that synchronised hashtables were supposed to be thread-safe for R/W operations, but without this logic my counter doesn't reach the correct total.

Full code for the loop itself below - I've left out setup of the Runspacepool etc

ForEach($i in $Array){
# Save down the data stream name and create parameter list for passing to the new job
$p = $i.Point.Name
$parameters = @{
    hash = $hash
    conn = $Conn
    p = $p
}

# Instantiate new powershell runspace and send a script to it
$PowerShell = [powershell]::Create()
$PowerShell.RunspacePool = $RunspacePool
[void]$Powershell.AddScript({

    # Receive parameter list, retrieve threadid
    Param (
        $hash,
        $conn,
        $p
    )
    $ThreadID = [appdomain]::GetCurrentThreadId()

    # Send data request to the PI Data Archive using the existing connection
    $q = Get-something (actual code removed)
    [int]$r = $q.Values.Values[0].Value

    # Lock out other threads from writing to the hashtable and output file to prevent collisions
    # If the thread isn't locked, attempt to lock it. If it is locked, sleep for 1ms and try again. Tracked by synchronised Hashtable.
    Do{
        if($hash.thread -eq 0){
            $hash.thread = $ThreadID
        }
        # Sleep for 1ms and check that the lock wasn't overwritten by another parallel thread.
        Start-Sleep -Milliseconds 1
    }Until($hash.thread -eq $ThreadID)

    # Increment the synchronised hash counter. Save the new result to a temporary variable (can't figure out how to get the hash counter itself to output to the file)
    $hash.counter+=$r
    [int]$t = $hash.counter+0

    # Write to output file new counter total, result, pointName and threadID
    Add-Content C:\Temp\test.txt "$t|$r|$p|$ThreadID"

    # release lock on the hashtable and output file
    $hash.thread = 0
})

# Add parameter list to instance (matching param() list from the script. Invoke the new instance and save a handle for closing it
[void]$Powershell.AddParameters($parameters)
$Handle = $PowerShell.BeginInvoke()

# Save down the handle into the $jobs list for closing the instances afterwards
$temp = [PSCustomObject]@{
    PowerShell=$Powershell
    Handle=$Handle
} 
[void]$jobs.Add($Temp)
}
Steve can help
  • 470
  • 7
  • 19
  • 3
    `Add-Content C:\Temp\test.txt "$hash.counter|$r|$p|$ThreadID"` --> `Add-Content C:\Temp\test.txt "$($hash.counter)|$r|$p|$ThreadID"`. `"$hash.counter"` won't interpolate as expected because the `.` has a special function in variable value retrieval. It is stringifying `$hash` and then adding the string `.counter`. Since `$hash` cannot be converted to a string as expected, you see the type listed instead, which is the result of the `.ToString()` method. – AdminOfThings Dec 23 '19 at 12:34
  • @AdminOfThings - appreciate the guidance, that's really helped. If you post that again as an Answer I'll happily enough accept, as I don't feel I'm going to get an answer on the second part. – Steve can help Jan 03 '20 at 09:11

0 Answers0