include_str for null terminated string

Question

I need to read a file into a null terminated string at compile time.

Working in Rust OpenGL. I have a shader source code stored in a separate file. The function that will eventually read the source is gl::ShaderSource from the gl crate. All it needs is a pointer to a null terminated string (the std::ffi::CStr type).

Typically the guides I have seen read the shader source file using include_str!, then at run time allocate a whole new buffer of length +1, then copy the original source into the new buffer and put the terminating 0 at the end. I'd like to avoid all that redundant allocating and copying and just have the correctly null terminated string at compile time.

I realize it is somewhat petty to want to avoid an extra allocation for a short shader file, but the principle could apply to many other types of larger constants.

---

While scrolling through suggested questions during the preview I saw this: How do I expose a compile time generated static C string through FFI?

which led me to this solution:

    let bytes1 = concat!(include_str!("triangle.vertex_shader"), "\0");
    let bytes2 = bytes1.as_bytes();
    let bytes3 = unsafe {
        CStr::from_bytes_with_nul_unchecked(bytes2)
    };
    println!("{:?}", bytes3);

Does this accomplish avoiding the runtime allocation and copying?

Answer 1

Your code is unsound. It fails to verify there are no interior NUL bytes.

You can use the following function to validate the string (at compile time, with no runtime cost):

pub const fn to_cstr(s: &str) -> &CStr {
    let bytes = s.as_bytes();
    let mut i = 0;
    while i < (bytes.len() - 1) {
        assert!(bytes[i] != b'\0', "interior byte cannot be NUL");
        i += 1;
    }
    assert!(bytes[bytes.len() - 1] == b'\0', "last byte must be NUL");
    // SAFETY: We verified there are no interior NULs and that the string ends with NUL.
    unsafe { CStr::from_bytes_with_nul_unchecked(bytes) }
}

Wrap it in a little nice macro:

macro_rules! include_cstr {
    ( $path:literal $(,)? ) => {{
        // Use a constant to force the verification to run at compile time.
        const VALUE: &'static ::core::ffi::CStr = $crate::to_cstr(concat!(include_str!($path), "\0"));
        VALUE
    }};
}

Then use it:

let bytes = include_cstr!("triangle.vertex_shader");

If there are interior NUL bytes the code will fail to compile.

When CStr::from_bytes_with_nul() becomes const-stable, you will be able to replace to_cstr() with it.

Answer 2

Yes that should work as intended. If you want you can even bundle it into a simple macro.

macro_rules! include_cstr {
    ($file:expr) => {{
        // Create as explicit constant to force from_bytes_with_nul_unchecked to
        // perform compile time saftey checks.
        const CSTR: &'static ::std::ffi::CStr = unsafe {
            let input = concat!($file, "\0");
            ::std::ffi::CStr::from_bytes_with_nul_unchecked(input.as_bytes())
        };
        
        CSTR
    }};
}

const VERTEX_SHADER: &'static CStr = include_cstr!("shaders/vert.glsl");
const FRAGMENT_SHADER: &'static CStr = include_cstr!("shaders/frag.glsl");