4

In a project I'm working on I need to use a bunch of C functions from Go. The current implementation is using cgo to achieve that, but doing so has a massive performance impact, which I am trying to remove.

  • cgo marks all C code as a syscall, causing the scheduler to allocate a new thread if needed. Since ~80% of the CPU time in my project is spent doing the heavy C work, this quickly results in having a lot of threads (500+ instead of GOMAXPROCS (= 8)), which my Linux kernel does not like much.
  • Doing the usual 'limiter' approach (have a buffered channel lock the cgo calls) results in a lot of mutexes, slowing the program down.

I want to keep using Go for this project as it's a great way of managing concurrency.

Things I have tried :

  • Comment out some stuff like entersyscall() in src/runtime/cgocall.go (didn't seem to help much, and I'd prefer to avoid modifying the Go code).
  • Put my code in a .c file and call it from Go, without the import "C". No luck here - I couldn't call the function and when I wrote the function names as package·function the compiler complained about invalid characters in the file.
  • Use a .s file and just write it as assembly. My asm isn't that great and I couldn't figure it out.

So, my question is: how does one use C functions from Go while avoiding the overhead of cgo?

Side note: I'm aware of why cgo marks all calls as a syscall, but in this specific case the functions I call do not block on locks or IO.

Tom van der Woerdt
  • 29,532
  • 7
  • 72
  • 105
  • I think gccgo may have less overhead when calling C code. You could try doing some further research on that. – Thomas Jun 08 '18 at 02:52

1 Answers1

0

What you want to do is not possible. There is a reason that C code has a high overhead, which is that C code uses a different ABI (the platform's native ABI) which doesn't support the short stacks Go uses. Thus whenever Go code calls C code, execution has to continue on the threads native stack. This causes the overhead you see. There is no way to remove this overhead. Think about it: If the overhead wasn't needed, it wouldn't exist.

It's hard to say what you should do instead without knowing what your program looks like.

fuz
  • 88,405
  • 25
  • 200
  • 352
  • Your logic is completely flawed. People don't remove unneeded overhead all the time, simply because they don't need to expend energy on removing it. – Puppy Jan 24 '15 at 17:16
  • Right, so what you're saying is that the extra stack in cgo is unavoidable - and I agree. But that doesn't mean that all the other overhead can't be removed – Tom van der Woerdt Jan 24 '15 at 18:07
  • @TomvanderWoerdt duh... all the extra overhead is required exactly for that to work. – fuz Jan 24 '15 at 19:12
  • @Puppy I'm pretty sure that the Go team would take any possible mean to reduce the latency of cgo calls as it's a very important feature. – fuz Jan 24 '15 at 19:13
  • 1
    @FUZxxl: "You could compile your C code with the C compiler from the Go project". Go 1.5 has deleted the Go C compilers. – peterSO Jan 24 '15 at 23:04
  • 1
    @peterSO That's why I said “your code will definitely break in a future version of Go” – fuz Jan 24 '15 at 23:06