What I'm trying to do
The below script loops through every item in an Array of data streams and requests a summary value for output to a text file. This external request is by far the most expensive part of the process, and so I am now using a Runspacepool to run multiple (5) requests in parallel, and whichever finishes first outputs its results.
These requests all write to a synchronised hashtable, $hash, which holds a running total ($hash.counter) and tracks which thread ($hash.thread) is updating the total and a .txt output file, to avoid potential write collisions.
What isn't working
Each thread is able to update the counter easily enough $hash.counter+=$r
, but when I try and Read the value into an Add-Content statement:
Add-Content C:\Temp\test.txt "$hash.counter|$r|$p|$ThreadID"
it adds an object reference rather than a number:
System.Collections.Hashtable+SyncHashtable.counter|123|MyStreamName|21252
And so I've ended up passing the counter through a temporary variable that can be used in the string:
[int]$t = $hash.counter+0
Add-Content C:\Temp\test.txt "$t|$r|$p|$ThreadID"
Which does output the true total:
14565423|123|MyStreamName|21252
What I'm asking
- Is it possible to remove this temporary variable and output directly from the hashtable? Why does the object reference have a '+' in the middle?
- I've had to add logic to 'lock' the hashtable to prevent data collisions. Should this be necessary? I'd been told that synchronised hashtables were supposed to be thread-safe for R/W operations, but without this logic my counter doesn't reach the correct total.
Full code for the loop itself below - I've left out setup of the Runspacepool etc
ForEach($i in $Array){
# Save down the data stream name and create parameter list for passing to the new job
$p = $i.Point.Name
$parameters = @{
hash = $hash
conn = $Conn
p = $p
}
# Instantiate new powershell runspace and send a script to it
$PowerShell = [powershell]::Create()
$PowerShell.RunspacePool = $RunspacePool
[void]$Powershell.AddScript({
# Receive parameter list, retrieve threadid
Param (
$hash,
$conn,
$p
)
$ThreadID = [appdomain]::GetCurrentThreadId()
# Send data request to the PI Data Archive using the existing connection
$q = Get-something (actual code removed)
[int]$r = $q.Values.Values[0].Value
# Lock out other threads from writing to the hashtable and output file to prevent collisions
# If the thread isn't locked, attempt to lock it. If it is locked, sleep for 1ms and try again. Tracked by synchronised Hashtable.
Do{
if($hash.thread -eq 0){
$hash.thread = $ThreadID
}
# Sleep for 1ms and check that the lock wasn't overwritten by another parallel thread.
Start-Sleep -Milliseconds 1
}Until($hash.thread -eq $ThreadID)
# Increment the synchronised hash counter. Save the new result to a temporary variable (can't figure out how to get the hash counter itself to output to the file)
$hash.counter+=$r
[int]$t = $hash.counter+0
# Write to output file new counter total, result, pointName and threadID
Add-Content C:\Temp\test.txt "$t|$r|$p|$ThreadID"
# release lock on the hashtable and output file
$hash.thread = 0
})
# Add parameter list to instance (matching param() list from the script. Invoke the new instance and save a handle for closing it
[void]$Powershell.AddParameters($parameters)
$Handle = $PowerShell.BeginInvoke()
# Save down the handle into the $jobs list for closing the instances afterwards
$temp = [PSCustomObject]@{
PowerShell=$Powershell
Handle=$Handle
}
[void]$jobs.Add($Temp)
}