0

I have an unmanaged struct containing fields whose byte offsets within it I want to determine. However, some of these fields are value tuples, and because value tuples are generic (EDIT: This is actually because ValueTuple uses LayoutKind.Auto; see answer below), I can't use Marshal.OffsetOf to determine the offset of any of the fields within the struct. (Even though it's not "marshal-able", it IS unmanaged and blittable to unmanaged memory.) Here's a trivialized example showing what I'm trying to do:

using System;
using System.Runtime.InteropServices;

IntPtr offset = Marshal.OffsetOf<TestStruct>(nameof(TestStruct.B)); // Throws exception
Console.WriteLine(offset);

[StructLayout(LayoutKind.Sequential, Pack=4)]
struct TestStruct
{
    public int A;
    public (int, int) B;
}

The above code throws an exception when calling Marshal.OffsetOf:

Type 'TestStruct' cannot be marshaled as an unmanaged structure; no meaningful size or offset can be computed.

So how can I programmatically determine the offset of all of the fields within an unmanaged struct, but without using Marshal.OffsetOf? I'm looking for a solution that will work on an arbitrary (unmanaged) struct type supplied via either generic parameter or Type object.

(Obviously, the above example is trivially easy to compute by hand. This is not my real-world example. My real-world use case is a library that uses reflection to iterate over all fields within arbitrary structs defined by the client application, and thus it is not possible to compute the offsets by hand.)

Walt D
  • 4,491
  • 6
  • 33
  • 43
  • What do you expect this hypothetical “OffsetOf” implementation to return as the offset of `TestStruct.B` on x64? – Iridium Aug 03 '22 at 22:27
  • Clearly you are not marshalling it using the standard interop marshaller, because it won;t handle it. If you are using `Unsafe.Copy` etc then why not use `Unsafe.ByteOffset(ref yourstruct.A, ref yourstruct.B)` – Charlieface Aug 03 '22 at 22:39
  • @Iridium My guess was B would have an offset of 4, though interestingly, when using an (int, int) tuple as it type, it's actually 8, for reasons I don't understand. (If I use a custom struct of two ints, the offset is 4.) – Walt D Aug 03 '22 at 22:43
  • @Charlieface `Unsafe.ByteOffset` requires both parameters to be of the same type, so I can't use it. Though I can do basically the same trick by getting a pointer to the struct and a pointer to each field and measuring the address offset. The problem with that is I don't know how to make it work if the struct type is supplied via generic parameter. – Walt D Aug 03 '22 at 22:46
  • 1
    It's still not really clear why you need to do this at all - accessing the fields by reflection shouldn't require knowledge of their offsets. This feels a bit like an XY problem, so if you can give some more details of what you're *actually* trying to do, it may be possible to find a more effective approach. – Iridium Aug 03 '22 at 23:09
  • The reason you don't understand is called "packing" which means that each value is a minimum of 8 bytes on x64 and 4 bytes on x86. I agree it would be easier if we understood what you are ultimately trying to do. – Charlieface Aug 03 '22 at 23:42
  • @Iridium I need to know the offsets because they get reported to the GPU driver. These structs store GPU vertex information and they get copied to GPU memory using Unsafe.CopyBlock, and I have to provide the GPU with a list of fields and their offsets so that it knows how to parse the memory. – Walt D Aug 04 '22 at 00:18
  • @Charlieface I know what packing is. But regardless of whether the program is running on x86 or x64, an `int` in C# is *always* 4 bytes. – Walt D Aug 04 '22 at 00:20
  • It makes no difference how big it is, if it's less than 8 bytes then the next field is at the next 8 byte alignment. Consider using `Unsafe.Copy` to just copy the struct as a whole, rather than going down the rabbit hole of working out the offsets – Charlieface Aug 04 '22 at 00:25
  • @Charlieface That is demonstrably untrue. Proof: https://pastebin.com/JcW4gA51 (Regardless, this has gone off-topic from my original question, which is about how to determine the offsets regardless of alignment.) – Walt D Aug 04 '22 at 00:31
  • You may find https://dotnetfiddle.net/E2SsyY and https://stackoverflow.com/questions/24742325/ interesting. Be that as it may, I still don't understand why you would want to compute the offsets of struct fields, why not just copy the struct as a whole? – Charlieface Aug 04 '22 at 00:51
  • @Charlieface As I mentioned to the other user above, it's because I need to provide the individual field offsets to the GPU driver. (The struct *does* get copied as a whole but the GPU still needs to know the offsets.) – Walt D Aug 04 '22 at 00:55

1 Answers1

1

Using Marshal.OffsetOf<>() doesn't work because System.ValueTuple<,> has automatic layout, (this is also likely why the packing you specified in your example is ignored, resulting B having an offset of 8 on x64).

It's not clear why you need to use tuples specifically, rather than e.g. defining your own structs with defined layout and using these instead, e.g.:

[StructLayout(LayoutKind.Sequential, Pack = 4)]
public struct TestStruct
{
    public int A;
    public InnerStruct B;
}

[StructLayout(LayoutKind.Sequential, Pack = 4)]
public struct InnerStruct
{
    public int A;
    public int B;
}

With the above definitions, Marshal.OffsetOf<TestStruct>(nameof(TestStruct.B)) works fine, and returns 4 as expected.

If there is some specific reason that using tuples is necessary, then you could provide your own System.ValueTuple<...> implementations with layout, though it feels like a bit of a hack.

For example, if you place the following definition somewhere within the same project that defines TestStruct:

namespace System
{
    [StructLayout(LayoutKind.Sequential, Pack = 4)]
    public struct ValueTuple<T1, T2>
    {
        public T1 Item1;
        public T2 Item2;

        // Appropriate implementation to match the behaviour of the framework's ValueTuple here...
    }
}

Then the using the original definition of TestStruct from your question:

[StructLayout(LayoutKind.Sequential, Pack = 4)]
public struct TestStruct
{
    public int A;
    public (int, int) B;
}

Will use the locally defined System.ValueTuple<,> implementation. Since the whole struct now has layout, it can then be used with Marshal.OffsetOf<>(), which will again return 4 for the offset of TestStruct.B.

Iridium
  • 23,323
  • 6
  • 52
  • 74
  • Thanks for the answer! Looks like I was incorrect about generics being the culprit behind Marshal not working and the real culprit was that ValueTuple uses auto layout. (Why that overrides my explicit packing in the containing struct I'm still curious about, but that's a question for another day.) – Walt D Aug 04 '22 at 12:55
  • I don't strictly *need* to support tuples here, it would just be nice for the library's completeness if it worked, and it would be convenient (and avoid reinventing the wheel) to not have to define custom vector types for these. Replacing ValueTuple is a clever solution, though I suspect inadvisable in most cases -- I assume there's a good reason why it uses auto layout, and changing it to sequential may have unintended consequences. – Walt D Aug 04 '22 at 12:59