Is it possible to compile a Vulkano shader at runtime?

Question

I've been using Vulkano in order to get some simple 3D graphics going on. Generally, I like to write my GLSL shaders in text and restart my program, or even changing shaders while the program is running. The examples given in Vulkano appear to use a macro to convert the GLSL to some form of SPIR-V based shader with Rust functions attached, but the GLSL is actually compiled into the binary (even when using a path to a file).

I've managed to get the crate shaderc to build my SPIR-V on the fly:

let mut f = File::open("src/grafx/vert.glsl")
         .expect("Can't find file src/bin/runtime-shader/vert.glsl 
                This example needs to be run from the root of the example crate.");

 let mut source = String::new();
 f.read_to_string(&mut source);

 //let source = "#version 310 es\n void EP() {}";
 let mut compiler = shaderc::Compiler::new().unwrap();
 let mut options = shaderc::CompileOptions::new().unwrap();
 options.add_macro_definition("EP", Some("main"));
 let binary_result = compiler.compile_into_spirv(
 &source, shaderc::ShaderKind::Vertex,
 "shader.glsl", "main", Some(&options)).unwrap();
 assert_eq!(Some(&0x07230203), binary_result.as_binary().first());

 let text_result = compiler.compile_into_spirv_assembly(
     &source, shaderc::ShaderKind::Vertex,
     "shader.glsl", "main", Some(&options)).unwrap();

 assert!(text_result.as_text().starts_with("; SPIR-V\n"));
 //println!("Compiled Vertex Shader: {}", text_result.as_text());

 let vert_spirv = { 
     unsafe { ShaderModule::new(device.clone(), binary_result.as_binary_u8()) }.unwrap()
 };
vert_spirv

So far, so good, we have a ShaderModule which seems to be the first step. However, we we actually need is a GraphicsEntryPoint which we can then put into our GraphicsPipeline. Apparently, GraphicsPipeline is where we string together our shaders, triangles and depth maps and all that lovely stuff.

Trouble is, I've no idea what is going on with the code that performs this feat:

pub fn shade_vertex <'a, S> (vert_spirv: &'a Arc<ShaderModule>) ->

 GraphicsEntryPoint<'a, S, VertInput, VertOutput, VertLayout>  {
 let tn = unsafe {
     vert_spirv.graphics_entry_point(
         CStr::from_bytes_with_nul_unchecked(b"main\0"),
         VertInput,
         VertOutput,
         VertLayout(ShaderStages { vertex: true, ..ShaderStages::none() }),
         GraphicsShaderType::Vertex
     )
 };
 tn
}

Specifically, what is VertInput and VertOutput? I've copied them from the example.

This is the closest example I could find that deals with loading Shaders on the fly. It looks like Input and Output are looking for entry points into the SPIR-V or something but I've no idea what to do with that. I'm hoping there is a function somewhere in the existing macro that will just take care of this for me. I've gotten this far but I seem a little stuck.

Has anyone else tried loading shaders at runtime?

Answer 1

I'm using wgpu, I've made my device, render_pipeline multithreaded like this:

let rx = Arc::new(Mutex::new(rx));
let window = Arc::new(Mutex::new(window));
let fs = Arc::new(Mutex::new(fs));
let fs_module = Arc::new(Mutex::new(fs_module));
let render_pipeline = Arc::new(Mutex::new(render_pipeline));
let device = Arc::new(Mutex::new(device));

used notify to listen to change events:

notify = "4.0.15"

use notify::{RecommendedWatcher, Watcher, RecursiveMode};
//mainxx
let (tx, rx) = mpsc::channel();
let mut watcher: RecommendedWatcher =
    Watcher::new(tx, Duration::from_millis(500)).unwrap();

log::info!("Starting watcher on {:?}", *FRAG_SHADER_PATH);
watcher.watch((*FRAG_SHADER_PATH).clone(), RecursiveMode::NonRecursive).unwrap();

Then spawn a thread that listens to changes:

thread::spawn(move || {
    log::info!("Shader watcher thread spawned");
    loop {
        if let Ok(notify::DebouncedEvent::Write(..)) = rx.lock().unwrap().recv() {
            log::info!("Write event in fragment shader");
            window.lock().unwrap().set_title("Loading shader.frag...");
            *fs.lock().unwrap() = load_fs().unwrap();
            *fs_module.lock().unwrap() = load_fs_module(Arc::clone(&device), &Arc::clone(&fs).lock().unwrap());
            *render_pipeline.lock().unwrap() = create_render_pipeline_multithreaded(Arc::clone(&device), Arc::clone(&fs_module));
            render.lock().unwrap().deref_mut()();
            window.lock().unwrap().set_title(TITLE);
        };
    }
});

where load_fs is a closure that uses glsl_to_spirv:

let load_fs = move || -> Result<Vec<u32>, std::io::Error> {
    log::info!("Loading fragment shader");
    let mut buffer = String::new();
    let mut f = File::open(&*FRAG_SHADER_PATH)?;
    f.read_to_string(&mut buffer)?;

    // Load fragment shader
    wgpu::read_spirv(
        glsl_to_spirv::compile(
            &buffer,
            glsl_to_spirv::ShaderType::Fragment
        ).expect("Compilation failed")
    )
};

Answer 2

There is an updated example for this in the vulkano repository.

I followed that and the example for shaderc-rs to get to this:

fn compile_to_spirv(src: &str, kind: shaderc::ShaderKind, entry_point_name: &str) -> Vec<u32> {
    let mut f = File::open(src).unwrap_or_else(|_| panic!("Could not open file {}", src));
    let mut glsl = String::new();
    f.read_to_string(&mut glsl)
        .unwrap_or_else(|_| panic!("Could not read file {} to string", src));

    let compiler = shaderc::Compiler::new().unwrap();
    let mut options = shaderc::CompileOptions::new().unwrap();
    options.add_macro_definition("EP", Some(entry_point_name));
    compiler
        .compile_into_spirv(&glsl, kind, src, entry_point_name, Some(&options))
        .expect("Could not compile glsl shader to spriv")
        .as_binary()
        .to_vec()
}


let vs = {
    unsafe {
        ShaderModule::from_words(
            device.clone(),
            &compile_to_spirv(
                "shaders/triangle/vs.glsl",
                shaderc::ShaderKind::Vertex,
                "main",
            ),
        )
    }
    .unwrap()
};

After this, vs can be used as in the example.