I have a wrapper function around Process to easy call some external procedures (similar to pythonic check_output):
struct Output {
public var code: Int32
public var stdout: String
public var stderr: String
}
func env(workingDir: String, command: [String]) -> Output {
let stdout = Pipe()
let stderr = Pipe()
let process = Process()
process.launchPath = "/usr/bin/env"
process.arguments = command
process.standardError = stderr
process.standardOutput = stdout
process.currentDirectoryPath = workingDir
var out = Data()
var err = Data()
stdout.fileHandleForReading.readabilityHandler = { fh in
out.append(fh.availableData)
}
stderr.fileHandleForReading.readabilityHandler = { fh in
err.append(fh.availableData)
}
process.launch()
process.waitUntilExit()
let code = process.terminationStatus
let outstr = String(data: out, encoding: .utf8) ?? ""
let errstr = String(data: err, encoding: .utf8) ?? ""
return .init(code: code, stdout: outstr, stderr: errstr)
}
Unfortunately, sometimes it fails. I'm building a small program, that runs thousands of those, for example:
env(workingDir: ".", command: ["file", "-b", "--mime-type", file.path])
And sometimes, very very rarely it outputs nothing with exit code 0.
I tried to reproduce it in tests:
func testEnv() {
let checkEcho: (String) -> () -> () = { mode in {
let speech = "Hello, \(mode) world!"
let output = autoreleasepool {
Process.env(workingDir: ".", command: ["echo", speech])
}
XCTAssertEqual(output.code, 0)
XCTAssertEqual(output.stderr, "")
XCTAssertEqual(output.stdout, speech + "\n")
} }
let performNTimesLoop: (Int, () -> Void) -> Void = {
for _ in 0..<$0 { $1() }
}
let performNTimesConc: (Int, () -> Void) -> Void = { count, code in
DispatchQueue.concurrentPerform(
iterations: count, execute: { _ in code() })
}
performNTimesLoop(1000, checkEcho("serial"))
performNTimesConc(1000, checkEcho("concurrent"))
}
It goes very well for serial loop, but for concurrent one it is failing systematically. Although, I don't have concurrency in my program (but I want to add some in near future), I think the reasons of failing are might be similar. I tried to add some Locks, Semaphores and DispatchGroups here and there, but got no luck.
This is very annoying, so any help would be much appreciated. Thanks!
UPD. As I understand this happens because it is possible that final outstr creation will execute before last callback (readabilityHandler) will finish, as it runs on background thread. In confirmation of this today I got ThreadSanitizer triggered on
...
out.append(fh.availableData) // modifying access
...
let outstr = String(data: out, encoding: .utf8) ?? "" //read acces
...
But I can't figure out a way to say "wait until every readability callbalck will finish to execute". Is it even possible? May be I need some other api to do that? At first glance it seems like this is a trivial task to be solved with high level api.
