22

I want to fork a go process and get back the id of the new process(es), but all I can see in the exec or os libraries is to start a new process.

Shashwat
  • 2,538
  • 7
  • 37
  • 56

2 Answers2

55

You supposedly want syscall.ForkExec() from the syscall package.

Note that fork() has been invented at the time when no threads were used at all, and a process had always had just a single thread of execution in it, and hence forking it was safe. With Go, the situation is radically different as it heavily uses OS-level threads to power its goroutine scheduling.

Now, unadorned fork(2) on Linux will make the child process have just the single thread—the one which called fork(2) in the parent process—among all those which were active, including some crucial threads used by the Go runtime. Basically this means that you simply cannot expect the child process to be able to continue executing Go code, and the only thing you can sensibly do is to somehow immediately perform exec(2). Notice that that's what syscall.ForkExec() is supposed to be used for.

And now think about the problem further. I'd say these days the only thing a direct call to fork(2) is useful for is "best-effort asynchronous process state snapshotting"—the kind, say, Redis uses. This technique relies on the fact the child process inherits all the memory data pages from its parent, but the OS uses copy-on-write technique to not really copy all that data, so the child can just sit there and save all the data structures to disk while its parent is chugging away modifying them in its own address space. Every other conceivable use for fork() implies immediate exec(), and that's what exec.Command() et al is for, so why just not use it?

Tiago Lopo
  • 7,619
  • 1
  • 30
  • 51
kostix
  • 51,517
  • 14
  • 93
  • 176
  • 13
    "Every other conceivable use for fork() implies immediate exec(), and that's what exec.Command() et al is for, so why just not use it?" There's setrlimit(), setting up SELinux, chroot()ing, closing or setting up FDs before handing over control -- and those need to happen between fork() and exec(), no? – Ivan Vučica Apr 23 '16 at 22:47
  • What if "best-effort asynchronous process state snapshotting" is *precisely* what I am trying to do? – cpcallen May 05 '17 at 17:40
  • 1
    @cpcallen, I'm afraid, that would be "not in Go" *this way.* The essense of the problem is that the Go runtime does two things to the OS-level threads: a) uses them to power your goroutines; b) uses them for its own needs. Since only a single threads survives a `fork()`, as soon as it's resumed in a new process, it will be in a half-assed state, impossible to work properly. – kostix May 12 '17 at 07:17
  • 1
    @cpcallen, there are two things possibly worthy of doing. 1) You may `fork()` in a call to some `C` code arranged via `cgo`; and that code must not "return" (that is, it must eventually call `exit(2)`). The upside is that the Go runtime, while its data pages will be "inherited" by the forked process, won't ever be given control. The downside is that your `C` code will have hard time accessing regular data managed by the Go runtime because of [this](https://github.com/golang/go/issues/12416). – kostix May 12 '17 at 07:22
  • 1
    @cpcallen, 2) You sidestep a Go's GC (and the strict rules attached to it w.r.t. `cgo` calls) by allocating data for Go values from a custom memory region not owned by the Go runtime (typically obtained by `mmap(2)`-ing an anonymous memory region). This is cumbersome to maintain, but such data may be freely shared between the C and the Go worlds. – kostix May 12 '17 at 07:24
  • 1
    @cpcallen, all in all, if we squint at it harder, we'll see that process-state snapshotting via forking is actually employing a hack, so a proper solution would be having a similar hack somewhere in the Go's runtime -- working cooperatively with its GC. So may be start a thread on the [Go mailing list](https://groups.google.com/d/forum/golang-nuts) soliciting the insight of those who know the intimate details of the runtime? – kostix May 12 '17 at 07:28
  • 5
    @IvanVučica, Go has direct support for some of this stuff in its `syscall.ForkExec()`: this function accepts an instance of a platform-speficic `syscall.ProcAttr` type which gives access to an assorted set of features including inheriting of FDs, setting the `cwd`, the environment block, controlling of session creation, chrooting, credentials, `clone(2)` flags etc. I'm afraid SELinux is not supported (and neither is AppArmor and friends). See `go doc syscall.ProcAttr` and `go doc syscall.SysProcAttr` for more info. – kostix May 12 '17 at 07:34
  • Thanks, I will take a look, but I do need cgroups, selinux, rlimits, etc. All I was saying is there are reasons for doing it multistep. – Ivan Vučica May 12 '17 at 08:19
  • @IvanVučica, sure! That's all about tradeoffs: when the original `fork()` was envisioned and implemented, there existed no software with a kind of layer of abstraction over the native OS threads the Go runtime implements, so this model simply does not work for Go. I'm sure its designers were fully aware about this problem but they had to make this decision ;-) Again, SO is not fit for such broad discussions -- I invite you to post to a mailing list instead. – kostix May 12 '17 at 08:25
  • 2
    @kostix: very useful insights & suggestions. Re: `fork()`: when it was envisioned there was no "software with a kind of layer of abstraction over native OS threads" because [UNIX didn't get native OS threads until nearly a decade later](http://www.serpentine.com/blog/threads-faq/the-history-of-threads/). :-) – cpcallen May 12 '17 at 14:00
2

syscall.Syscall(syscall.SYS_FORK, 0, 0, 0) might work, and first return value is id you want.

Here's an example:

func main() {
    foo := 4
    bar := 10
    id, _, _ := syscall.Syscall(syscall.SYS_FORK, 0, 0, 0)
    if id == 0 {
        foo++
        fmt.Println("In child:", id, foo, bar)
    } else {
        bar++
        fmt.Println("In parent:", id, foo, bar)
    }
}

then get output similar to this:

In parent: 16397 4 11
In child: 0 5 10
KujouRinka
  • 21
  • 3