2

I'm somewhat new to programming in general and I've run into an issue with declaring 3D and 4D arrays. I have several declarations like this at the start of my main function, but I've narrowed the problem down to these 4:

string  reg_perm_mark_name[64][64][64];
short   reg_perm_mark_node_idex[64][64][64];
short   reg_perm_mark_rot[64][64][64][4];
short   reg_perm_mark_trans[64][64][64][3];

When I run my program with these, I get "System.StackOverflowException" in my executable. I would much prefer a way to allocate them dynamically, The way I have it now was meant to be temporary anyway and I'm not sure how to declare array pointers properly.

The 4 elements I'm using in the 4D array reg_perm_mark_trans, for example, are [node index][region index][marker index][xyz coordinates]. Also there's a total of 35 multidimensional arrays being declared at once. (most of them are 1D and 2D) I'm not sure if that helps.

Can someone show me how to make these 4d arrays work or maybe how to make them dynamically allocating with pointers or vectors? Be descriptive please, I'm still learning.

  • 2
    The space for such "simple" variables is intentionally very limited. Your arrays are too big. Use a `vector` etc. (which uses `new` inside, this isn´t limited.). There are enough examples how to use `std::vector` in the Internet. – deviantfan Jul 11 '15 at 01:06
  • Standard containers are a poor substitute for multidimensional arrays, though. – celticminstrel Jul 11 '15 at 01:08
  • 1
    @celticminstrel `std::array` is a great substitute for naked multidimensional arrays. Same goes for `std::vector` when you're working with dynamically sized multidimensional arrays. – Captain Obvlious Jul 11 '15 at 01:27
  • Oh, you have a point - `std::array` at least is a decent substitute for multidimensional arrays. Not so much with `std::vector` though - you have to choose between manually flattening the index, or using a "jagged array" instead of a true multidimensional array. – celticminstrel Jul 11 '15 at 01:59
  • 1
    Please note the _dynamically sized_ portion of my comment on `std::vector`. You'll have even more overhead trying to manage a _dynamically sized_ multidimensional array yourself than you will using `std::vector`. The really nice thing is you can mix and match them with the greatest of ease so you get the best of both worlds without all the overhead and headache you get by dismissing the usefulness of standard containers. – Captain Obvlious Jul 11 '15 at 02:04
  • The only way to use `std::vector` to make a dynamically sized multidimensional array is to use the technique Chris Drew posted in his answer - use a single 1D vector and calculate the index from the coordinates. It certainly works great, but the syntax isn't so nice and convenient (unless you write a wrapper class as I did). You can certainly create a vector of vectors, but that's not a true multidimensional array, and to get MD-array behaviour you'll need to loop through and resize each inner vector. (Or use inner `std::array` instead, I guess, as Chris Drew suggested.) – celticminstrel Jul 11 '15 at 02:33
  • @celticminstrel And what is wrong with my suggestion to use vector of array of array? That is not a "jagged array" and it has nice syntax. – Chris Drew Jul 11 '15 at 02:40
  • It's a little weird, and actually it's a "semi-jagged" array, but I guess it works. – celticminstrel Jul 11 '15 at 02:54
  • @celticminstrel I'm not sure I understand how it is "semi-jagged". Every row and column has the same size. Isn't that the definition of a true multidimensional array? Multidimensional vectors only get difficult if the number of rows or columns is not fixed at compile time and in that case c-style arrays are equally difficult, if not more so. – Chris Drew Jul 11 '15 at 03:26
  • Hm... you do have a point. The vector could still be resized, but I guess that wouldn't actually make it no longer truly multidimensional. – celticminstrel Jul 11 '15 at 03:49

3 Answers3

4

Assuming for simplicity that sizeof(string) == 2 (it's probably more), you're trying to allocate (64^3)*9*2 bytes on the stack. That comes out to 4,718,592 bytes, or approximately 4.5 MiB. Most likely, you just don't have 4.5 MiB available on your stack.

Since these variables are declared in main(), you have two possible solutions:

  1. Declare them static.

  2. Declare them outside main(), as global variables.

This will cause them to be allocated before the program starts, rather than on the stack. The only difference between the two approaches is whether they'll be visible in other functions.

There may also be a way to tell your compiler that the program needs more stack space, but I think making them static is probably the better solution here. If they were in a function other than main() though, you'd probably need to do something else.

celticminstrel
  • 1,637
  • 13
  • 21
  • Something is very wrong about your byte calculation. It´s more like MB, not GB. And where did that 9 come from? Btw. a `std::string` can´t use less than pointer (and in reality, depending on the implementation, it´s far more) – deviantfan Jul 11 '15 at 01:12
  • ...you're right. It's MiB, not GiB. And the 9 comes from the sum of his fourth dimensions (4 and 3) plus the fact that the other two are three-dimensional. I'm only estimating `sizeof(string) == 2` because it simplified the calculation by making all the base types the same size. It's an estimate only. – celticminstrel Jul 11 '15 at 01:13
  • 1
    There are plenty of contexts where you can't declare variables `static` or make them global (e.g multi-threaded) and you would need to allocate dynamically. – Chris Drew Jul 11 '15 at 02:03
  • 1
    I won't down vote because the answer isn't wrong, and it's reflects effort, and I hate down voting, but this really isn't the best answer. Putting giant arrays into static storage is hardly idiomatic. Also, creating non function-local static variables of non trivial type is incredibly dangerous; code written like this calls constructors before main. Constructors before main can cause innocent code to segfault depending how things are linked, and is banned in many style guides (e.g. llvm). – Nir Friedman Jul 11 '15 at 02:38
  • `string` is a class so it's size must be much larger than 4 bytes because it must store string length along with allocated memory – phuclv Jul 11 '15 at 03:20
  • Yes. I just made a quick, rough estimate there. Of course the `string` isn't just 2 bytes. – celticminstrel Jul 11 '15 at 03:21
  • I think that string is probably 3 words, like vector, though that's just a guess, which would be 24 bytes. – Nir Friedman Jul 11 '15 at 03:22
2

This line:

string  reg_perm_mark_name[64][64][64]

declares 64*64*64 = 262144 strings on the stack. A std::string is typically about 32 bytes so thats about 8MB. The maximum stack size is typically about 1MB.

To declare the array dynamically you could use std::vector. Generally, multidimensional std::vectors can be a bit cumbersome and it is often better to declare a single dimensional vector and convert to a single index when you access an element:

std::vector<std::string> reg_perm_mark_name(64*64*64);

int i = 13;
int j = 27;
int k = 7;
reg_perm_mark_name[i + 64*j + 64*64*k] = "Hello world!";

But in this case you can declare a multi-dimensional std::vector quite efficiently by using std::array instead of std::vector for the inner types. The use of std::array avoids too many memory allocations as they have a fixed size. I would use typedefs or using aliases to make the declaration clearer:

using StrArray = std::array<std::string, 64>;
using StrArray2D = std::array<StrArray, 64>;

std::vector<StrArray2D> reg_perm_mark_name(64);

reg_perm_mark_name[3][4][7] = "Hello world!";
Chris Drew
  • 14,926
  • 3
  • 34
  • 54
  • It's completely irrelevant whether the inner types fit on the stack. If the outer type is a vector, everything except that one vector (3 words usually) will be on the heap. In fact, if the outer most type is a vector but all the inner types are array, you get the best of everything: single allocation, almost no overhead, no possibility of stack over flow, and nice indexing notation. – Nir Friedman Jul 11 '15 at 02:14
  • 1
    @NirFriedman Yeah, I've used the wrong term. I was just trying to say the inner types are of suitable size to be chunks for allocation. I'll try and edit to make it more accurate... – Chris Drew Jul 11 '15 at 02:20
  • The best answer, hope it overtakes the other two as they have serious issues. – Nir Friedman Jul 11 '15 at 02:44
  • Well, even though I have my own answer, this one's worth an up-vote. I'm a bit dubious about his second option, mind you. I'd instead suggest writing a wrapper class (though that's a fair bit of work even for 2D, and even more for 3D and 4D.) – celticminstrel Jul 11 '15 at 03:14
  • The default stack size on Windows is 1MB but on Linux it's typically 4 or 8 bytes depending on distro. Anyway it's still not enough to store such large arrays – phuclv Jul 11 '15 at 03:21
  • I highly doubt it would be just 4 or 8 bytes. That's probably not even enough room for one stack frame. You probably meant MB or maybe kB or something? – celticminstrel Jul 11 '15 at 03:22
  • 1
    Why is the second option dubious? – Nir Friedman Jul 11 '15 at 03:24
2

A simple solution is to use static allocation (i.e. move your arrays outside of any function, or mark them as static).

Note that if you use C++ arrays then the usage and footprint are the same but then they behave like proper containers:

array<array<array<string,64>,64>,64> reg_perm_mark_name;

To use dynamic allocation unsafely, you could write:

auto reg_perm_mark_rot = new short[64][64][64][4];
// ...
delete[] reg_perm_mark_rot;

To use it safely, since C++14 (note that the innermost dimension gets special treatment):

auto reg_perm_mark_rot = std::make_unique<short[][64][64][4]>(64);

Of course you can use array instead of C-style arrays with the dynamic options, but then you would have an extra level of indirection to use the array.

M.M
  • 138,810
  • 21
  • 208
  • 365
  • 1
    There's no extra level of indirection between a c array and a std array. You should use std::array. – Nir Friedman Jul 11 '15 at 02:20
  • Also, please never tell people to use globals of non trivial type, it's wildly dangerous, see my other comment. – Nir Friedman Jul 11 '15 at 02:42
  • @NirFriedman there is if you are dynamically allocating it. If you disagree then please show your code. – M.M Jul 11 '15 at 02:44
  • These global strings are not dangerous at all, let alone wildly. The standard requires that they are initialized before first use (unless another unit accesses them via `extern`, which I'm not suggesting) – M.M Jul 11 '15 at 02:46
  • It is in fact dangerous, because depending on the linking structure of the project, they can get double destructed at exit. See e.g. http://stackoverflow.com/questions/6714046/c-linux-double-destruction-of-static-variable-linking-symbols-overlap. If you write out a trivial program, you will see such variables construct before main and destruct after, this turns out to not be very robust in the face of linking pieces of code together. – Nir Friedman Jul 11 '15 at 03:13
  • @MattMcNabb also, since you asked for code: https://goo.gl/PtfYV0. Identical assembly for a unique_ptr to a c array and a unique_ptr to a std::array. – Nir Friedman Jul 11 '15 at 03:25
  • @NirFriedman destruction errors would be a compiler bug. The question you linked to is a mess. Your "identical assembly" link illustrates my point perfectly. The `std::array` version has to write `*x` as opposed to the C-style array writing `x`. – M.M Jul 11 '15 at 04:36
  • It's nothing to do with the compiler, it's the linker. And that's how it is, it's a known thing, it's not changing, so such variables are de facto dangerous. It's irrelevant what * there is, the assembly is what's actually executed. Identical assembly means identical behavior, and identical behavior means the same amount of indirection – Nir Friedman Jul 11 '15 at 04:42
  • @NirFriedman by "compiler" I mean the implementation. And if a broken implementation is in use, then *any* construct is dangerous. I used a compiler once where `memcpy(a, b, 0)` would copy 65536 bytes. Does that mean `memcpy` with third argument `0` is dangerous? – M.M Jul 11 '15 at 04:44