Is it safe to convert *C.uint8_t to []uint8?

Question

I'm trying to pass uint8_t array from C code to Go, then read some bytes from file and store them into this array in Go code.

Sample code is here.

main.c :

#define BUFFER_SIZE 16384

int read_go(uint8_t* buffer, int bufferSize);

void read_buf() {
     uint8_t* buffer = (uint8_t*)malloc(BUFFER_SIZE);
     read_go(buffer, BUFFER_SIZE)

     // do something with buffer

     free(buffer)
}

read.go :

 /*
 #include "main.c"

 extern int read_go(uint8_t* buffer, int bufferSize);
 */
 import "C"

 //export read_go
 func read_go(buffer *C.uint8_t, bufferSize C.int) C.int {
      f, _ := os.Open("filename")
      defer f.close()

      buff := *(*[]uint8)(unsafe.Pointer(&buffer))
      n, _ := f.Read(buff)

      return C.int(n)
 }

It works well, but I'm worrying about segmentation fault. Because I cannot specify the buffer size at Read function unlike fread function of C.

I know that Read function reads bytes from file as much as len(buff). But I cannot guarantee that len(buff) is the same as bufferSize.

Is it safe to convert *C.uint8_t to []uint8?

It's always unsafe, as indicated by the use of the `unsafe` package, but you still need to use it to convert types from C. You have the `bufferSize` value right there though, so what do you mean that you can't specify it? — JimB, Nov 28 '18 at 15:09
I don't know if you built this on Linux or other Unix-like, but naming your function `read` is going to conflict with the system C library. Yours will take priority but other code that wants to `read` things will likely call your function. With the wrong arguments. Chaos and destruction will ensue. — Zan Lynx, Nov 28 '18 at 16:18
@ZanLynx I'm working on MAC, so it is Unix. But the code above is just a sample, I didn't consider about that. The name of the function in real code is not 'read'. Thank you for your comment. — Hoon, Nov 29 '18 at 02:41
@JimB I mean I cannot specify the size of buffer when I do `n, _ := f.Read(buff)`. Usually, in C code, we can put the size of buffer as a parameter in the read functions. — Hoon, Nov 29 '18 at 02:45

peterSO · Accepted Answer · 2018-11-28T16:44:27.583

Your code is wrong.

A Go slice is implemented as a Go struct:

type slice struct {
    array unsafe.Pointer
    len   int
    cap   int
}

Your values for slice.len and slice.cap are undefined.

Questions seeking debugging help ("why isn't this code working?") must include the desired behavior, a specific problem or error and the shortest code necessary to reproduce it in the question itself.

Your code fragments don't compile and are incomplete.

buf is a Go byte slice to satisfy io.Reader.

type Reader interface {
    Read(p []byte) (n int, err error)
}

buffer *C.uint8_t
bufferSize C.int

buf := (*[1 << 30]byte)(unsafe.Pointer(buffer))[:bufferSize:bufferSize]
n, err := f.Read(buf)

Here is my solution in reproducible form.

main.c:

#include <stdint.h>
#include <stdlib.h>
#include <stdio.h>

#define BUFFER_SIZE 16384

int read_go(uint8_t* buffer, int bufferSize);

void read_buf() {
    uint8_t* buffer = (uint8_t*)malloc(BUFFER_SIZE+1);
    buffer[BUFFER_SIZE]='\0';
    int n = read_go(buffer, BUFFER_SIZE);
    if (n < 0) {
        printf("read_go: error: %d\n", n);
        n = 0;
    }
    if (n > BUFFER_SIZE) {
        n = BUFFER_SIZE;
    }
    buffer[n] = '\0';

    // do something with buffer
    int width = n;
    printf("%d\n", width);
    if (width > 16) {
        width = 16;
    }
    for (int i = 0; i < width; i++)     {
        printf("%02X", buffer[i]);
    }
    printf("\n");
    for (int i = 0; i < width; i++)     {
        printf("%-2c", buffer[i]);
    }
    printf("\n");

    free(buffer);
}

int main(int argc, char *argv[]) {
    read_buf();
    return EXIT_SUCCESS;
}

read.go:

package main

/*
#include <stdint.h>
*/
import "C"

import (
    "os"
    "unsafe"
)

//export read_go
func read_go(buffer *C.uint8_t, bufferSize C.int) C.int {
    f, err := os.Open("filename")
    if err != nil {
        return C.int(-1)
    }
    defer f.Close()

    buf := (*[1 << 30]byte)(unsafe.Pointer(buffer))[:bufferSize:bufferSize]
    n, err := f.Read(buf)
    if err != nil {
        return C.int(-1)
    }
    return C.int(n)
}

func main() {}

Output (Linux):

$ cat filename
filedata 01234567890
$ export LD_LIBRARY_PATH=./:$LD_LIBRARY_PATH
$ go build -a -o libread.so -buildmode=c-shared read.go
$ gcc main.c libread.so -o read && ./read
21
66696C65646174612030313233343536
f i l e d a t a   0 1 2 3 4 5 6 
$

So just to clarify: You are casting the `buffer` pointer into a Go pointer to a byte array of size `1 << 30` (big but not the maximum, just a convenient shift size, and a constant) and then creating a slice of that imaginary buffer. — Zan Lynx, Nov 28 '18 at 16:16
@ZanLynx: Exactly. "The length is part of the [Go] array's type; it must evaluate to a non-negative [compile-time] constant representable by a value of type int.": [Array types, The Go Programming Language Specification](https://golang.org/ref/spec#Array_types). So we need some trickery. Array length constant `1 << 30` is a simple way of describing a very large Go `byte` array that is handled by 32-bit `int` implementations. — peterSO, Nov 28 '18 at 16:30

Is it safe to convert *C.uint8_t to []uint8?

1 Answers1