2

I program embedded systems. One golden rule is that we never call malloc(); all data must be statically allocated at compile time.

Hence, I am not really familiar with Variable Length Arrays, which were introduced with C99.

The concept seems clear enough, and I don't need it explaining. My question is what happens at run time if there isn't enough free memory for such an array?

I would imagine that it is o/s dependent, maybe compiler dependent, to what would GCC/Linux do, and MS visual Studio C on Windows? Any X99 or Posix definitions?

Deduplicator
  • 44,692
  • 7
  • 66
  • 118
Mawg says reinstate Monica
  • 38,334
  • 103
  • 306
  • 551
  • Mawg I think you've answered your own question. It will result in undefined behaviour so you'll need to see how the given platform your developing for deals with this. I'd guess you'll have to try and capture this before calling the function that uses the VLA. I'd guess that if malloc is to be avoided so should VLAs. – cdcdcd Jan 15 '17 at 16:08
  • The same rules as for any stack overflow apply, likely crashing uncontrollably on smaller embedded MCUs and trapping via memory protection on larger devices (including Linux/Windows). More to the point stack allocation is also dynamic allocation, and so cannot really be avoided in C. About the best you can do is to use an embedded compilers with supports computing an upper stack bound for your program's call graph if you avoid function pointers/recursion/VLAs/alloca (or perhaps provide suitable manual hints, say lists of possible function pointer targets). – doynax Jan 15 '17 at 16:11
  • doynax, I could be wrong here but on many platform/environments the stack is fixed per process at compile time. For example, it is very easy to run out of stack on windows (but then doesn't mean it isn't dynamic just the upperlimit is static), but in saying that you can get round the problem by starting a new thread with an arbitrary stack space. – cdcdcd Jan 15 '17 at 16:13
  • @cdcdcd: Yes, but only up to available memory. The question concerns an embedded device so presumably this space is rather limited and what is worse is liable to be entirely unchecked depending on the specific platform. Usually the requirement is that such systems (your toaster or whatever) be designed so as to never run out of memory, in which case stack allocation must be handled with care. Nominally the same rules would apply on Windows/Linux as well but since the OS itself lacks any such allocation guarantees you are forced to settle for best-effort solutions. – doynax Jan 15 '17 at 16:27
  • doynax - point taken. Just added point as a caveat. – cdcdcd Jan 15 '17 at 17:03
  • 2
    A VLA usually (depends on compiler) allocates memory from the stack, similar to alloca() or _alloca(), in which case not having a heap or malloc() isn't a problem. On an embedded system, the stack space (per thread) may be very limited. – rcgldr Jan 15 '17 at 17:56
  • I didn't know that (+1). So, if I create such an array at power up it will permanently consume a chunk of my stack. As you say, probably best to avoid. – Mawg says reinstate Monica Jan 15 '17 at 18:08

2 Answers2

1

From the point of view of the Standard, an attempt to allocate a VLA with a size the implementation cannot accommodate invokes Undefined Behavior. Because the Standard provides no means of discovering what size array an implementation could safely create, and does not mandate that implementations allow any particular size, any attempt to create a VLA object with a size greater than 1 should be regarded as invoking Undefined Behavior except in cases where one happens to know enough about implementation's inner workings to determine the size of VLA it will be able to handle.

If malloc() is unavailable, one's best bet may be to define a large array of whatever type has the coarsest alignment requirement, store its address into a volatile-qualified pointer [the storage in which the pointer itself resides should be thus qualified] read it back, and interpret that as the start of a memory pool. No other use should be made of the original array object. While the Standard wouldn't guarantee that a compiler wouldn't decide that it should generate code that checks whether the pointer still identifies the original object and, if it does, skipping any code that would use that pointer to access anything other than the original object's type, the use of volatile on the pointer should make that really unlikely.

Once a memory pool is created, you can write your own memory-management functions to use it, though any time a pointer is returned to the pool it may be necessary to use the volatile-pointer-laundering hack to prevent compilers from using type-based aliasing to justify treating the last uses of storage as its old type as unsequenced relative to the first uses of storage as a new type.

supercat
  • 77,689
  • 9
  • 166
  • 211
  • *all data must be statically allocated at compile time* means "all data must be statically allocated at compile time". Not "you may allocate your own data at run time as long as you are using your own functions for that". – n. m. could be an AI Jan 15 '17 at 15:53
  • @n.m.: The array would be allocated at compile time; the user functions would merely make return pointers to portions of the *already-allocated* storage. – supercat Jan 15 '17 at 16:12
  • "Allocating" doesn't mean "asking the OS, or the comoiler, or God for whatever resiurce". It is not necessarily a single-level affair. The compiler allocates a chunk of memory for your program, (statically), then your memory manager uses this chunk to allocate smaller chunks (dynamically) – n. m. could be an AI Jan 15 '17 at 16:44
  • supercat are you basically saying if you want a type of dynamic allocation don't use VLA or malloc (they're unsafe). Instead statically allocate resources that you know to be safe at compile time (you know the platform limits) then write your own version of a "TLB" for this resource and manage. – cdcdcd Jan 15 '17 at 17:01
  • Who has promised thay size 1 will necessarily succeed? I wouldn't trust him. – n. m. could be an AI Jan 15 '17 at 17:04
  • @cdcdcd: Basically. The aliasing rules in the Standard make it absurdly difficult to write such managers in a way that implementations would be forbidden from messing up--likely because compiler writers used to have the common sense to avoid breaking such things even when the Standard would allow it. Today's mentality is that if the Standard would permit a compiler to impose a stupid code-breaking "optimization", that means the code was "broken". – supercat Jan 15 '17 at 17:43
  • @cdcdcd: If one can convince a compiler to guarantee that it won't try to apply aliasing optimizations across certain places in the code, however, parceling out pieces of static allocations can make it possible for programs to reap many of the benefits of dynamic allocation while maintaining sufficient constraints on how things might be allocated to ensure that a program will work with all possible sequences of inputs. – supercat Jan 15 '17 at 20:24
  • @n.m.: Nothing in the Standard promises that *any* particular operation will succeed without overflowing the stack, beyond the fact that for each implementation there must be at least one program (possibly contrived) which nominally exercises the implementation limits given in the Standard without dying. On the other hand, an implementation should be able to create a variable length array of 1 in any case where it could create any such array at all, and a non-snarky interpretation of the Standard would suggest that the authors wouldn't have required implementations to accept code... – supercat Jan 15 '17 at 20:27
  • ...which makes use of VLAs if they intended that compilers should be able to treat *all* such code as invoking Undefined Behavior. While it's often useful to allow programs to contain code that could not possibly execute without invoking UB, I don't see much point to requiring that a compiler accept *syntactic forms* which have no required meaning. – supercat Jan 15 '17 at 20:35
  • @n.m.: On further reflection, I'll agree that assuming that creation of even one single-element VLA within main() should be reliable is overly optimistic, since I know of at least one embedded compiler where it's likely to fail (the Keil/ARM compiler uses malloc/free for VLAs, but unless the startup code has been configured to set up the heap--something the Standard does not require for freestanding implementations--the malloc() call will fail at runtime). – supercat Jan 16 '17 at 16:00
  • It's not UB to declare a large VLA. The standard does not permit automatic allocation to fail. If an implementation exhibits stack overflow then it's non-conforming. – M.M Sep 30 '17 at 01:38
  • @M.M: If conformance would require that VLA allocations always succeed, that would make the Standard essentially impossible to implement in conforming fashion on real-world hardware. What would be the point of such a standard? – supercat Sep 30 '17 at 15:17
  • I know what your answer to your last question would be :) – M.M Sep 30 '17 at 21:20
1

Variable length arrays are typically allocated on the stack. Like any other variable allocated on the stack this is normally done by subtracting from the stack pointer (or adding to it for an upwards-growing stack). A frame pointer is likely to be used so that the function can keep track of it's stack frame in the face of dynamically determined changes to the stack pointer. As with other stack allocations there is typically no error checking in this process.

This brings a couple of dangers.

  1. The space allocated to the stack may be overflowed. Depending on the platform this may result in some kind of memory error from the kernel, it may result in the platform dynamically allocating more stack space or it may result in overwriting of other memory.
  2. On platforms that use protection pages beyond the stack for automatic stack growth and/or detecting stack overflows a sufficiently large stack allocation may "skip over" those pages. This may lead to memory protection errors or worse memory corruption in cases where the stack overflow would normally have been caught.

Neither of these risks is specific to variable length arrays but variable length arrays make them far more likely to lie hidden until the code is invoked with particular parameters.

plugwash
  • 9,724
  • 2
  • 38
  • 51