8

In the Golang wiki, under "Turning C arrays into Go slices", there is a block of code that demonstrates how to create a Go slice backed by a C array.

import "C"
import "unsafe"
...
        var theCArray *C.YourType = C.getTheArray()
        length := C.getTheArrayLength()
        slice := (*[1 << 30]C.YourType)(unsafe.Pointer(theCArray))[:length:length]

Can anyone explain exactly what (*[1 << 30]C.YourType) does? How does it turn an unsafe.Pointer into a Go slice?

Some Noob Student
  • 14,186
  • 13
  • 65
  • 103

1 Answers1

16

*[1 << 30]C.YourType doesn't do anything itself, it's a type. Specifically, it's a pointer to an array of size 1 << 30, of C.YourType values. The size is arbitrary, and only represents an upper bound that needs to be valid on the host system.

What you're doing in the third expression is a type conversion. This converts the unsafe.Pointer to a *[1 << 30]C.YourType.

Then, you're taking that converted array value, and turning it into a slice with a full slice expression (Array values don't need to be dereferenced for a slice expression, so there is no need to prefix the value with a *, even though it is a pointer).

You could expand this out a bit like so:

// unsafe.Pointer to the C array
unsafePtr := unsafe.Pointer(theCArray)

// convert unsafePtr to a pointer of the type *[1 << 30]C.YourType
arrayPtr := (*[1 << 30]C.YourType)(unsafePtr)

// slice the array into a Go slice, with the same backing array
// as theCArray, making sure to specify the capacity as well as
// the length.
slice := arrayPtr[:length:length]

This construct has been replaced by a generalized unsafe.Slice function in go1.17:

slice := unsafe.Slice(theCArray, length)
JimB
  • 104,193
  • 13
  • 262
  • 255
  • Thanks for the insights! `[1<<30]` is used because the compiler rejects variable array bounds! So devs just pick a ridiculously large number! – Some Noob Student Feb 13 '18 at 05:33
  • @SomeNoobStudent: it's really just setting an upper bound for size. `1<<30` is often used in examples, because it's valid for both 32 and 64 bit systems, and it looks nicer than `1<<31-1`. – JimB Feb 13 '18 at 14:47
  • can the `length` bigger than `1<<30`? or should I use `1<<63` on 64bit platform to make sure the index space is enough for access all the underlying data? – HM.Yang May 09 '21 at 01:10
  • @HM.Yang, `length` can be anything up to what your system supports, so if you need more than `1<<30` bytes, then specify more than `1<<30` bytes. You will just have to use different source files for 32bit architectures, or limit building to 64bit only. – JimB May 11 '21 at 16:33
  • Note that starting with Go 1.17 one could use `unsafe.Slice` instead – Eli Bendersky Jan 19 '22 at 19:02
  • 1
    thanks @EliBendersky, I've been updating these with the new function as I see them too ;) – JimB Jan 19 '22 at 19:54