Read and write to same file simultaneously from Invoke-RestMethod in PowerShell

Question

We have a program that controls door access on our premises. Whenever a person opens the door with their tag, an event is registered in the programs database. These events can be read by enabling a HTTP integration, which makes it possible to view them in the localhost web browser.

We want to export the events viewed from the HTTP URL to Splunk. To do that, I have been writing a PowerShell script that uses Invoke-RestMethod to fetch the data from the URL to a file on C:\Scripts, which Splunk then monitors.

Here's the PowerShell script I have so far:

$getRestMethodParams = @{
    Uri = 'http://localhost:5004/eventexport?end_date=keep'
    Method = 'Get'
    Credential = $Creds
    OutFile = 'C:\Scripts\SplunkOutput.xml'
}
Invoke-RestMethod @getRestMethodParams

The URI used will keep the connection open with a heartbeat with end_date=keep so we are monitoring the events in real-time. The script will also output the results into the file 'C:\Scripts\SplunkOutput.xml'. So far, so good.

However, the code will also always keep the file in an open/used state (because of the heartbeat parameter), which prevents Splunk from reading from the file until I terminate the script, which we don't want to do (well, we will have to at some point to prevent the file from growing too big, but that will be done later on).

A colleague suggested I tried to use [System.IO.File] to manipulate the file streams, but I only got so far. This is the code I used:

$file = [System.IO.File]::Open('C:\Scripts\SplunkOutput.xml')

$getRestMethodParams = @{
    Uri = 'http://localhost:5004/eventexport?end_date=keep'
    Method = 'Get'
    Credential = $Creds
    OutFile = $file
}
Invoke-RestMethod @getRestMethodParams

Unfortunately, that gave me the output as:

Cannot find an overload for "Open" and the argument count: "1".
At C:\Scripts\SplunkPoller1.ps1:12 char:1
+ $file = [System.IO.File]::Open('C:\Scripts\SplunkOutput.xml')
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodException
    + FullyQualifiedErrorId : MethodCountCouldNotFindBest

I also tried with (from PowerShell Closing FileStream):

$inFile = 'C:\Scripts\SplunkOutput.xml'
$inFS = New-Object FileStream($inFile, [FileMode]::Open)

$getRestMethodParams = @{
    Uri = 'http://localhost:5004/eventexport?end_date=keep'
    Method = 'Get'
    Credential = $Creds
    OutFile = $inFS
}
Invoke-RestMethod @getRestMethodParams

Which gave me:

Unable to find type [FileMode].
At C:\Scripts\SplunkPoller1.ps1:11 char:40
+ $inFS = New-Object FileStream($inFile, [FileMode]::Open)
+                                        ~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (FileMode:TypeName) [], RuntimeException
    + FullyQualifiedErrorId : TypeNotFound

Any and all tips would be greatly appreciated on how to approach this issue! Thanks.

You maybe doing it wrong. the open method takes 2 params. `Open(String, FileMode)`. FileMode decides the action you are going to take on the file and is defined here https://learn.microsoft.com/en-us/dotnet/api/system.io.filemode?view=netframework-4.8. In your case, the number should be `6` for `Append` — Sid, Feb 25 '20 at 10:11
This might not solve your underlying problem, but you need to fully qualify the type, so “[System.IO.FileMode]::Open”. You can omit System if you like though, so “[IO.FileMode]::Open” works as well but I personally prefer the full name... — mclayton, Feb 25 '20 at 22:05

score 1 · Accepted Answer · answered Feb 25 '20 at 22:54

1

Rather than use Monitor in your inputs.conf, on Windows, you can also try using MonitorNoHandle, documented at https://docs.splunk.com/Documentation/Splunk/8.0.2/Data/Monitorfilesanddirectorieswithinputs.conf#MonitorNoHandle_syntax

MonitorNoHandle doesn't use Windows File Handles to read the file, so can be used for files that are kept open.

answered Feb 25 '20 at 22:54

Simon Duff

2,631
2
7
15

1

Thank you! I never knew that Splunk had that type of monitor tag. It now works wonders, thank you! :) – tsvenbla Feb 27 '20 at 12:05

Read and write to same file simultaneously from Invoke-RestMethod in PowerShell

1 Answers1