3

I am using glslang in my C++ based Vulkan project to deduce information about shaders that I am using to help automatically build my descriptor sets and descriptor set layouts. I have come to a point that I need to support unsized/runtime-defined arrays (See https://www.khronos.org/opengl/wiki/Data_Type_(GLSL)#Arrays_of_arrays).

The glslang library appropriately compiles the shader and when looking at the SPIRV does show that I am generating a TypeRuntimeArray.

However, when digging through the reflection all but the most simple of cases are not reflected.

I have taken steps to make sure that the reflection isnt optimized out or hidden. I have tried manually enabling the array of arrays extension, and have tried in vertex, compute and fragment shader stages. The shaders are compiled with #version 450. I've tried seeing if std430 vs 140 provides different results to no avail.

I've also made sure that when indexing into the unsized array I'm not indexing with a constant. I've tried providing index's from built in types such as gl_VertexIndex, and from the push_constant block or seperate uniform-blocks

Take the following example that does appropriately reflect:

layout(std430) buffer Buffer1
{
    float[] unsizedArray; 
} myBuffer1;

The reflection for the unsizedArray is available from both TProgram.dumpReflection() and getting the reflection and playing with it.

However, it will only work for that simple case. If we try either

layout(std430) buffer Buffer1
{
    float[][10] unsizedArray; 
} myBuffer1;

or

struct MyType{
    vec4 thingy;
    int otherThingy;
}
layout(std430) buffer Buffer1
{
    MyType[] unsizedArray; 
} myBuffer1;

No reflection of the inner unsized array is provided is provided. We can get reflection on the outer array from the first failed example, or on the variable from MyType, but there doesn't seem to be any way to get the inner array information. As far as I can tell there is no way to 'walk-up' the tree tree from the reflected variable.

Aside from parsing the spirv myself is there anyway to convince glslang to reflect the unsized array?

Edit: Examples to give a little more info

#version 450

struct MyStruct{
    float z;
};

layout(binding = 5,std430) buffer StorageBlock{
    float[] myArray;
} storage;

layout(push_constant, std430) uniform Constant{
    int index;
} ps;

void main() {
    float test = storage.myArray[ps.index];
}
// A simple function I I built just to show this. There are quite a few other functions provided
// by both the reflection and glsl::TProgram object, but I haven't found anyway to get the inner array.

function parseUniform(glslang::TObjectReflection& reflection){
    reflection->dump(); // This just outputs a string to stdout
    auto type = reflection.getType();
    if( type->isArray() ) {
        bool isUnsizedArray = type->isUnsizedArray();
        bool containsUnsizedArray = type->containsUnsizedArray();
        bool isArrayOfArrays = type->isArrayOfArrays();
        bool isSizedArray = type->isSizedArray();

        auto arraySizes = (glslang::TArraySizes*)type->getArraySizes();
        bool isInnerUnsized = arraySizes->isInnerUnsized();
        bool hasUnsized = arraySizes->hasUnsized();
        int  numDims = arraySizes->getNumDims();
        bool isVariablyIndexed = arraySizes->isVariablyIndexed();

        std::cout << "isUnsizedArray(): " << isUnsizedArray << "\n";
        std::cout << "containsUnsizedArray(): " << containsUnsizedArray << "\n";
        std::cout << "isArrayOfArrays(): " << isArrayOfArrays << "\n";
        std::cout << "isSizedArray(): " << isSizedArray << "\n";
        std::cout << "isInnerUnsized(): " << isInnerUnsized << "\n";
        std::cout << "numDims: " << numDims << "\n";
        std::cout << "isVariablyIndexed(): " << isVariablyIndexed << "\n";
        std::cout << "hasUnsized(): " << hasUnsized << "\n";
    }
}

If we query reflection data using the function above on a storage block defined like

layout(binding = 5,std430) buffer StorageBlock{
    float[] myArray;
} storage;

The output is

StorageBlock.myArray: offset 0, type 1406, size 0, index 0, binding -1, stages 32, arrayStride 4, topLevelArrayStride 4  
isUnsizedArray(): 1  
containsUnsizedArray(): 1  
isArrayOfArrays(): 0  
isSizedArray(): 0  
isInnerUnsized(): 0  
numDims: 1  
isVariablyIndexed(): 1  
hasUnsized(): 1  

If we call on a simple multi-dimensional array:

    layout(binding = 5,std430) buffer StorageBlock{
       float[][5] myArray;
    } storage;
The singular reflected uniform that is provided outputs   
StorageBlock.myArray: offset 0, type 1406, size 1, index 0, binding -1, stages 32, arrayStride 4, topLevelArrayStride 40  
isUnsizedArray(): 0  
containsUnsizedArray(): 0  
isArrayOfArrays(): 0  
isSizedArray(): 1  
isInnerUnsized(): 0  
numDims: 1  
isVariablyIndexed(): 0  
hasUnsized(): 0  

And if we call using a struct we only get:

    layout(binding = 5,std430) buffer StorageBlock{
       MyStruct[] myArray;
    } storage;

We get:

StorageBlock.myArray.z: offset 0, type 1406, size 1, index 0, binding -1, stages 32, topLevelArrayStride 4

Because .z is of course a single float and not an array we don't get any of the array information. In all of these tests only one uniform is 'reflected'

  • What do you mean by "reflection" in this context? – Nicol Bolas Apr 08 '19 at 03:19
  • The glslang library provides an object they call "Reflection" it gives information about various items such as what uniform buffer blocks are available or what what variables are used that create the shader. The reflection dump that I mentioned just spits the info out to stout For example: `myBuffer1.y.z: offset 648, type 1406, size 1, index 0, binding -1, stages 32, topLevelArrayStride 32`. However you can get the info and dig down into the specifics from a reflection object that is created after you compile and link the shaders – user1712469 Apr 08 '19 at 03:27
  • spriv-cross can be used to generate reflection data from a spirv shader in a JSON format. Would that provide the information you need? – Jherico Apr 08 '19 at 03:49
  • @user1712469: Can you provide the actual reflection data, both for what you say works and what you say does not? – Nicol Bolas Apr 08 '19 at 04:21
  • @NicolBolas I have edited the post to incude some info, but looks like that library spirv-cross Jherico mentioned may be able to help and indeed https://github.com/KhronosGroup/SPIRV-Cross/wiki/Reflection-API-user-guide looks like its going to give me the info i need! Thanks so much for reaching out, I have heard bad things about posting as a new-comer so it was nice to have you take an interest! – user1712469 Apr 08 '19 at 06:23
  • @Jherico that library looks like it does everything I didn't want to do myself! I didn't know it spat out reflection too, although I guess it makes sense. Thanks so much – user1712469 Apr 08 '19 at 06:24
  • @user1712469 I'd recommend you make a self answer then on how exactly it solved your issue. Comments are not meant to be permanent on stack overflow and can be deleted at any time for a variety of reasons. – Krupip Apr 08 '19 at 14:27

1 Answers1

1

You can use the SPIRV-cross utility included with the Vulkan SDK to generate JSON reflection data for SPIR-V shaders. Alternatively, you can link against use the SPIRV-cross libraries if you want to build them and use the reflection data directly.

Jherico
  • 28,584
  • 8
  • 61
  • 87