1

for something I'm building I need to be able to store runtime created structs in an array, I also want these structs to be stored contiguously in memory. Is there a way to store a sequence of objects in an array, without knowing what objects they will be at compile time? I will know every possible object that could be used and their sizes. Basically when I define one of these arrays I will create a template of exactly what objects will be stored and in what order and after that it will not change for the lifetime of the array, meaning that at runtime the stride and positions of objects will be known. Could this be done by allocating something like a std::byte array and then accessing/initializing the variables by casing their memory positions to the desired type using a pointer? I know that it's not the cleanest solution but could it work?

Clarification: This would be used for ECS components defined by a custom scripting system at runtime, so contiguous memory and dynamic allocation is a must, I realize that I'm going to need to create class to handle this and it's going to be advanced, that's the fun of it.

WireWhiz
  • 43
  • 5
  • What are the original requirements? What' the analysis of those requirements? The design from that analysis? What original requirement, analysis or design decision lead to you needing to store contiguously? What is the actual underlying problem that your program is supposed to solve? – Some programmer dude Apr 16 '21 at 13:41
  • This is possible but advanced. You can look into placement new and `std::aligned_storage`. – François Andrieux Apr 16 '21 at 13:45
  • 2
    Casting byte (array) pointers to higher types (like structures) will likely cause alignment/aliasing issues. But `std::memcpy` should get round that. Probably best to define your own class to handle this. – Adrian Mole Apr 16 '21 at 13:48
  • 1
    It's possible but pretty difficult to get right. My first approach would be to see if there is any way at all to avoid it. – molbdnilo Apr 16 '21 at 13:49
  • 1
    Sounds like an *array* of [std::variant](https://en.cppreference.com/w/cpp/utility/variant) may be helpful? – Galik Apr 16 '21 at 13:55
  • I'm not sure how you'd implement all those requirements/constraints in C++. Does it have to be an array? Can it be a `std::vector` or `std::array`? How would you index such a beast? Why do they need to be contiguous? Non-homogenous arrays that are contiguous in memory will be tricky, you may need a `union` or `std::variant` to hold the possible structs. However, I don't see how you can do that runtime synthesizes the structs. That's a lot of tricky. – Eljay Apr 16 '21 at 14:01
  • Yes, you can invent this yourself by using a byte array and casting pointers. Watch out for alignment and don't forget to call placement new to create objects in the array. – user253751 Apr 16 '21 at 14:07
  • Yes, it would work. Depending on hardware and speed, watch your alignment. Suggest a constant Alignment=1 (or 2, 4, 8 ... bytes) – Pete D. Apr 16 '21 at 14:15

1 Answers1

0

So I think I found an answer:

std::vector<std::byte> bytes;
void addFloat(float value)
{
    size_t index = bytes.size();
    bytes.resize(bytes.size() + sizeof(float), std::byte(0));
    *(float*)(&bytes[index]) = value;
}
float readFloat(size_t index) const 
{
    return *(float*)(&bytes[index]);
}

This can read and write floats (tested) from the vector, and I'm assuming it will work for any value type I make a function for. So now all I need to do is write a class that handles multiple value types. When the class is created I'll calculate the the size of the virtual float and store that, then I can look up a float by index using that value. Then to retrieve a component of that struct value I'll have another list that stores the offset of each variable from the 0 index of the struct. So index + offset[variable index] = index of variable, then I can cast that index to the correct type and there's my value.

Edit: See comments bellow for possible improvements.

WireWhiz
  • 43
  • 5
  • you could implement proper lookup using a `map` or `unodered_map` – Nsikan Sylvester Apr 16 '21 at 14:40
  • @NsikanSylvester, what's the advantage to using a map over just a vector, and what are you suggesting I use it for? Looking up structs? or their variables? – WireWhiz Apr 16 '21 at 15:23
  • If the `bytes` has (in order) a `float`, a `double`, a `char`, and a `tuple`, how do you access them? – Eljay Apr 16 '21 at 16:10
  • @Eljay How I would do it is have an enum that has a value for each type, then store enum values in an array so that you can get the type at a certain index, then have a different function to call for each, most likely in a switch statement. – WireWhiz Apr 16 '21 at 20:08
  • Do you care about the *undefined behavior* aspect? For example, fixing this one: `float readFloat(size_t index) const { float rv; auto p = (float*)(&bytes[index]); memcpy(&rv, p, sizeof rv); return rv; }` – Eljay Apr 16 '21 at 21:29
  • @Eljay what makes the original undefined? As far as I can tell it would only be undefined if the index was out of range or less then sizeof(float) away from the end of an array. I'm still learning c++ and I don't know all the edge cases yet. – WireWhiz Apr 16 '21 at 22:36
  • Because it hasn't had a `float` constructed there, and "there" may be an invalid location if it doesn't meet the alignment requirements (if any) for a `float` on your platform. That's where `memcpy` can come in handy for the kind of purpose you are envisioning to copy the bytes to-and-fro. – Eljay Apr 16 '21 at 23:40