How to deal with assets in rust?

Question

I have several json files which contain objects that need to be exported from the module to be used(read only) in various places in the code base. Exporting a function that reads the files and parses them and invoking it every time the objects are needed seems very wasteful. In go I'd export a global variable and initialize it in init function. So how do I go about doing it in rust?

Answer 1

I guess you are using this for interface definitions between different system parts. This is a known and well understood problem and is usually solved with a build script, such as in the case of protobuf.

There is a very good tutorial about how to use the build script to generate files.

This is how this could look like in code.

(All files are relative to the crate root directory)

shared_data.json:

{
    "example_data": 42
}

build.rs:

use std::{
    env,
    fs::File,
    io::{Read, Write},
    path::PathBuf,
};

fn main() {
    // OUT_DIR is automatically set by cargo and contains the build directory path
    let out_path = PathBuf::from(env::var("OUT_DIR").unwrap());

    // The path of the input file
    let data_path_in = "shared_data.json";

    // The path in the build directory that should contain the generated file
    let data_path_out = out_path.join("generated_shared_data.rs");

    // Tell cargo to re-run the build script whenever the input file changes
    println!("cargo:rerun-if-changed={data_path_in}");

    // The actual conversion
    let mut data_in = String::new();
    File::open(data_path_in)
        .unwrap()
        .read_to_string(&mut data_in)
        .unwrap();

    {
        let mut out_file = File::create(data_path_out).unwrap();
        writeln!(
            out_file,
            "::lazy_static::lazy_static! {{ static ref SHARED_DATA: ::serde_json::Value = ::serde_json::json!({}); }}",
            data_in
        )
        .unwrap();
    }
}

main.rs:

include!(concat!(env!("OUT_DIR"), "/generated_shared_data.rs"));

fn main() {
    let example_data = SHARED_DATA
        .as_object()
        .unwrap()
        .get("example_data")
        .unwrap()
        .as_u64()
        .unwrap();

    println!("{}", example_data);
}

Output:

42

Note that this still uses lazy_static, because I didn't realize that the json!() macro isn't const.

One could of course adjust the build script to work without lazy_static, but that would probably involve writing a custom serializer that serializes the json code inside the build script into executable Rust code.

EDIT: After further research, I came to the conclusion that it's impossible to create serde_json::Values in a const fashion. So I don't think there is a way around lazy_static.

And if you are using lazy_static, you might as well skip the entire build.rs step and use include_str!() instead:

use lazy_static::lazy_static;

lazy_static! {
    static ref SHARED_DATA: serde_json::Value =
        serde_json::from_str(include_str!("../shared_data.json")).unwrap();
}

fn main() {
    let example_data = SHARED_DATA
        .as_object()
        .unwrap()
        .get("example_data")
        .unwrap()
        .as_u64()
        .unwrap();

    println!("{}", example_data);
}

However, this will result in a runtime error if the json code is broken. With the build.rs and the json!(), this will result in a compile-time error.

Answer 2

The general way of solving this problem in Rust is to read the assets in a single place (main(), for example), and then pass a reference to the assets as needed. This pattern plays nicely with Rust's borrow checker and initialization rules.

However, if you insist on using global variables:

When we apply Rust's initialization and borrow checker rules to global variables one can see how the compiler might have a hard time proving (in general) that all accesses to a global variable are safe. In order to use global variables the unsafe keyword might need to be used when accessing the variable, in which case you are simply asserting that the programmer is responsible for manually verifying that all accesses to the global variable happen in safe way. Idiomatic Rust tries to build safe abstractions to minimize how often programmers need to do this. I wouldn't consider the lazy_static! macro as a hack, it is a abstraction (and a very commonly used one) that transfers the responsibility from the programmer to the language to prove that the global access is safe.