10

I am getting a suprising warning from the native library I am using when passing a boolean value in a JNA structure:

value of pCreateInfo->clipped (-1) is neither VK_TRUE nor VK_FALSE

In this library VK_TRUE and VK_FALSE are #defined as 1 and 0 respectively.

The structure itself isn't particularly complex and everything else appears to be working (the native library seems to treat the 'undefined' boolean as false), but here it is anyway:

public class VkSwapchainCreateInfoKHR extends Structure {
    public int sType;
    public Pointer pNext;
    public int flags;
    public Pointer surface;
    public int minImageCount;
    public int imageFormat;
    public int imageColorSpace;
    public VkExtent2D imageExtent;
    public int imageArrayLayers;
    public int imageUsage;
    public int imageSharingMode;
    public int queueFamilyIndexCount;
    public Pointer pQueueFamilyIndices;
    public int preTransform;
    public int compositeAlpha;
    public int presentMode;
    public boolean clipped;       // <--------- this is the field in question
    public Pointer oldSwapchain;
}

If the clipped field is false there is no warning, if it's true then I get the warning - it appears JNA is mapping true to integer -1?

There are not many native boolean values used by this library but I get the same behaviour whenever one is set to true (and again everything else works fine).

In particular, if I change clipped to be an int and set the value explicitly to 1 or 0 everything works!

Is -1 the default value for JNA boolean true?

If so, how would I go about over-riding the type mapping?

Or should I just use int 'manually'?

stridecolossus
  • 1,449
  • 10
  • 24
  • Something smells fishy here. I would think that correct code would have the actual bit pattern of a boolean field never enter into the equation. A boolean value has two potential states...'true' and 'false'...period. So what is going on where the mapping is being made between boolean and integer values? Does that structure somehow actually contain a malformed bit pattern for 'true', or is something else converting the value to an int and then another something else is expecting a different integer representation? – CryptoFool Mar 18 '19 at 17:42
  • Given the constant definitions, is the code expecting that field to be an integer value? Maybe it just needs to be an int. what native library is defining VK_TRUE? - Could this be a function of whatever database definition is persisting this structure? – CryptoFool Mar 18 '19 at 17:46
  • @Steve No databases, JNA is used to invoke native libraries (Without having to go through the JNI hell that it entails). – stridecolossus Mar 19 '19 at 08:07

2 Answers2

10

JNA maps to native libraries via libffi. There is no bool type in libffi so other mappings must be used -- JNA's default type mapping chooses to map boolean to ffi_type_uint32. This works in the structure(s) because it happens to match the 32-bit mapping size, but not the definition: in C, 0 is false and anything nonzero is true. Only if the native type is also boolean does this 0/non-zero interpretation regain meaning as false/true.

A web search using FFI or JNI and boolean keywords can uncover multiple examples such as this one and this one where unpredictable results occur when libraries are accessed via FFI or JNI and do not conform to the 0 / 1 requirement for boolean values. The latter example appears very similar to this case where a true Java boolean is interpreted as a C int with a value other than 1.

Somewhere under the hood between FFI and your library, and possibly in compiled byte code and/or platform/compiler-dependent type conversions, it's likely that a bitwise "not" is being applied to 0x00000000, turning it into 0xffffffff which is still 'true' in C.

The bottom line is that JNA will by default map Java boolean false to a 32-bit native value of 0, and a Java boolean true to a 32-bit native value that is not 0, and that's all that can be assumed. If your library requires true to have an integer value of 1, either use an integer type that you can specifically set, or use a custom Type Mapping for boolean that sets an int to 0 or 1 for you. JNA's W32APITypeMapper has an example of this conversion to 1 or 0 for the Windows BOOL type.

In your case, assuming you are mapping the VkSwapchainCreateInfoKHR structure defined here, the type of clipped is VkBool32:

typedef struct VkSwapchainCreateInfoKHR {
    VkStructureType                  sType;
    const void*                      pNext;
    VkSwapchainCreateFlagsKHR        flags;
    VkSurfaceKHR                     surface;
    uint32_t                         minImageCount;
    VkFormat                         imageFormat;
    VkColorSpaceKHR                  imageColorSpace;
    VkExtent2D                       imageExtent;
    uint32_t                         imageArrayLayers;
    VkImageUsageFlags                imageUsage;
    VkSharingMode                    imageSharingMode;
    uint32_t                         queueFamilyIndexCount;
    const uint32_t*                  pQueueFamilyIndices;
    VkSurfaceTransformFlagBitsKHR    preTransform;
    VkCompositeAlphaFlagBitsKHR      compositeAlpha;
    VkPresentModeKHR                 presentMode;
    VkBool32                         clipped;
    VkSwapchainKHR                   oldSwapchain;
} VkSwapchainCreateInfoKHR;

Where...

typedef uint32_t VkBool32;

So int is the correct mapping here -- you need to map clipped to a 32-bit integer Edit: As you've pointed out in your answer, it is simple to add your own type mapper to better handle these int values!

(While I'm reviewing the type mappings, you might find IntByReference a better mapping than Pointer for the pQueueFamilyIndices field.) (Your mapping is correct for a variable length int array.)

Daniel Widdis
  • 8,424
  • 13
  • 41
  • 63
  • 1
    I feared that might be the case. The structure was code-generated from the C header where (as you point out) VkBool32 -> int, but I intentionally over-rode the mapping to VkBool32 -> Java boolean, hoping that JNA would then map the booleans the 0/1, or at least there was some way to override the resultant mapping, but such is life! As I said it's not a big deal (there aren't that many boolean values), I posted the question mainly for background interest. – stridecolossus Mar 19 '19 at 08:13
  • Your aside about the `pQueueFamilyIndices` is interesting - it's currently a `Pointer` because the indices is a Java `int[]`, i *thought* that the various by-reference types were only for parameters *returning* values using the C reference operator? – stridecolossus Mar 19 '19 at 08:15
  • Ah, you are correct that Pointer is better for a non-fixed length array. – Daniel Widdis Mar 19 '19 at 13:24
3

Actually as it turns out there are a lot of booleans in the various native library structures, several hundred of them in fact! It would be nice to preserve the intention of the boolean fields, rather than replacing them all with int just because the implementation enforces that restriction. So I spent some time looking into JNA type conversion...

JNA supports mapping of custom types using a TypeMapper passed as an additional argument to Native::load when the native library is created. Custom type mappings are defined using the Java-to/from-native converter interface TypeConverter.

Defining a custom boolean wrapper that maps Java boolean to/from a C int with 1=true and 0=false is fairly straight-forward:

public final class VulkanBoolean {
    static final TypeConverter MAPPER = new TypeConverter() {
        @Override
        public Class<?> nativeType() {
            return Integer.class;
        }

        @Override
        public Object toNative(Object value, ToNativeContext context) {
            if(value == null) {
                return VulkanBoolean.FALSE.toInteger();
            }
            else {
                final VulkanBoolean bool = (VulkanBoolean) value;
                return bool.toInteger();
            }
        }

        @Override
        public Object fromNative(Object nativeValue, FromNativeContext context) {
            if(nativeValue == null) {
                return VulkanBoolean.FALSE;
            }
            else {
                final int value = (int) nativeValue;
                return value == 1 ? VulkanBoolean.TRUE : VulkanBoolean.FALSE;
            }
        }
    };

    public static final VulkanBoolean TRUE = VulkanBoolean(true);
    public static final VulkanBoolean FALSE = VulkanBoolean(false);

    private final boolean value;

    private VulkanBoolean(boolean value) {
        this.value = value;
    }

    public boolean value() {
        return value;
    }

    public int toInteger() {
        return value ? 1 : 0;
    }
}

The type mapper(s) are registered thus:

final DefaultTypeMapper mapper = new DefaultTypeMapper();
mapper.addTypeConverter(VulkanBoolean.class, VulkanBoolean.MAPPER);
...

final Map<String, Object> options = new HashMap<>();
options.put(Library.OPTION_TYPE_MAPPER, mapper);
Native.load("vulkan-1", VulkanLibrary.class, options);

However this only works if the structure(s) in question is defined inside the JNA library interface - trivial if one is writing a small library with a handful of structures (which is usually the case) but a bit of a headache when you have several hundred methods and ~500 structures (that are code-generated).

Alternatively the type mapper can be specified in the structure constructor but this requires:

  1. instrumenting every structure that needs the custom mapping(s).

  2. every custom type has to additionally implement NativeMapped so that JNA can determine the native size of the custom type (no idea why essentially the same information has to be specified twice).

  3. each custom type must support a default constructor.

Neither of these are particularly pleasant options, it would nice if JNA supported global type mappings that covered both cases. Guess I need to re code-generate all the structures with the type-mapper. Sigh.

However this only works if the structure(s) in question are defined inside the JNA library interface. A simple workaround is to define a base-class structure within the library and extend all the others from that:

public interface Library {
    abstract class VulkanStructure extends Structure {
        protected VulkanStructure() {
            super(VulkanLibrary.TYPE_MAPPER);
        }
    }
...
}

public class VkSwapchainCreateInfoKHR extends VulkanStructure { ... }

I have used the same mechanism to auto-magically map the ~300 code-generated enumerations to native int that currently look like this:

public enum VkSubgroupFeatureFlag implements IntegerEnumeration {
    VK_SUBGROUP_FEATURE_BASIC_BIT(1),   
    VK_SUBGROUP_FEATURE_VOTE_BIT(2),    
    ...

    private final int value;

    private VkSubgroupFeatureFlag(int value) {
        this.value = value;
    }

    @Override
    public int value() {
        return value;
    }
}

Currently all structures that refer to an 'enumeration' are actually implemented as an int. With a custom type converter for IntegerEnumeration in place the field type can be the actual Java enumeration and JNA will handle the conversion to/from the integer value (which I currently have to by-hand). This obviously makes the structures slightly more type-safe, definitely clearer, and explicitly refers to the actual enumeration rather than an int - nice.

i.e.

public class VkSwapchainCreateInfoKHR extends VulkanStructure {
    ...
    public int flags;
    public Pointer surface;
    public int minImageCount;
    // The following fields were int but are now the Java enumerations
    public VkFormat imageFormat = VkFormat.VK_FORMAT_UNDEFINED;
    public VkColorSpaceKHR imageColorSpace;
    ...
}

(recently found an example doing exactly that here).

Hopefully all this waffling helps someone trying to get their head around the vagaries of JNA.

stridecolossus
  • 1,449
  • 10
  • 24
  • Nice solution! I may steal your enum idea elsewhere. – Daniel Widdis Mar 19 '19 at 13:27
  • Also I just realized JNA has a built in [W32APITypeMapper](https://github.com/java-native-access/jna/blob/master/src/com/sun/jna/win32/W32APITypeMapper.java) that already does the conversion to integer 1 / 0. – Daniel Widdis Mar 19 '19 at 23:10
  • 1
    @DanielWiddis - Thought you might like know - this morning I finally achieved the first step of the project I'm working on that has led to the various questions I've posted regarding JNA. A hard-coded triangle rendered using Java and Vulkan! Not much to show for a LOT of effort (several weeks hard slog) but it's definitely a result. Many thanks for the support. At some point I'll update the questions with links to markdown 'blog' I'm planning. – stridecolossus Mar 21 '19 at 11:18
  • Excellent! I look forward to looking at it! – Daniel Widdis Mar 21 '19 at 15:45
  • From the wiki link: "JNA provides the PointerType class but it is not clear what this actually achieves since we want compile-time type-safety." -- that's exactly what it achieves! `class ThingPtr extends PointerType { }` and you now have a type safe equivalent to `Pointer`. You can use `setPointer()` to assign it to any `Pointer` you get from elsewhere (e.g., a `Memory` buffer). See [here, for example](https://github.com/oshi/oshi/blob/342c8eed73fd8285a280794b852358d1462f5532/oshi-core/src/main/java/oshi/jna/platform/mac/CoreFoundation.java#L49). – Daniel Widdis Apr 02 '19 at 15:31
  • Ah nice, I obviously completely misinterpreted how it was used! Cheers – stridecolossus Apr 02 '19 at 17:16
  • Yep, the `Structure.ByReference` types also extend from this, with the added Structure overhead as well. Actually all the `ByReference` types extend from it, with their own customized accessor methods. But if all you want to do is dress up a `Pointer` with strict typing, it's a nice middle ground. – Daniel Widdis Apr 02 '19 at 19:03