1

One can safely allocate x elements of size y in C by using calloc(x, y) and calloc() will take care of the multiplication x*y.

However realloc() for example only takes the new size as parameter and I was wondering how I could safely realloc x*y bytes using realloc().

What if x*y doesn't fit in size_t? How does calloc() handle this?

Sourav Ghosh
  • 133,132
  • 16
  • 183
  • 261
user194192
  • 21
  • 3
  • Just a thought: If you need to worry about overflow for allocating size, probably you need to break down the algo into smaller sub-parts? – Sourav Ghosh Apr 03 '17 at 11:41
  • @SouravGhosh This is part of a datatype library for other persons to use and they only need to provide element size and number of elements they want to store. How can I use `realloc` to do this safely (I already have some memory - so `calloc` is not an option)? – user194192 Apr 03 '17 at 11:43
  • If you need more than 4GB of memory for your algorithm, then there are serious problems :D – unalignedmemoryaccess Apr 03 '17 at 11:44
  • @tilz0R This is used in a datatype. What is wrong with me about caring about overflow? – user194192 Apr 03 '17 at 11:46
  • Nothing is wrong with you, just mention it :) – unalignedmemoryaccess Apr 03 '17 at 11:49
  • @SouravGhosh The usual reason to worry about overflow during this multiplication is security. There have been numerous exploits where a protocol/user input provides a far too large number of elements in an array and the allocation succeeds because the multiplication wraps to a small number. – Art Apr 03 '17 at 12:22

5 Answers5

5

The size_t is an unsigned type, and the maximum value of size_t is the absolute maximum size of object that can be allocated with realloc or malloc; this is available in the macro SIZE_MAX. On 32-bit personal computers the size_t is often 32 bits; 64 bits on 64-bit computers. It should be enough.

To ensure that the calculation of item_size * n_items does not overflow, you can divide the SIZE_MAX by item_size and ensure that the resulting value is greater than or equal to n_items:

size_t max_items = SIZE_MAX / item_size;
if (max_items < n_items) {
    // an overflow would occur
}
else {
    // it is ok
}

calloc must return NULL if the allocation did not succeed, so calloc most probably has a check similar to one above.

1

The short answer is that you can't do it safely. The most you can do is limit the amount of memory you attempt to allocate so it does not exceed SIZE_MAX.

SIZE_MAX is the maximum value that can be produced by the sizeof operator.

Every data type in C must have a size that can be computed using sizeof, including arrays, and any contiguous block of memory allocated using malloc(), calloc(), or realloc().

If x*y is mathematically greater than SIZE_MAX, then it is not possible to allocate that amount of memory by any means. Even if the underlying system supports that, the C program will not be able to use that memory block entirely.

There is also a concern that computing x*y (assuming x and y are of type size_t) will use modulo arithmetic, so will actually give the result mathematically equivalent to (x*y)%(SIZE_MAX + 1).

Peter
  • 35,646
  • 4
  • 32
  • 74
1

What if x*y doesn't fit in size_t? How does calloc() handle this?

realloc() and malloc() are limited in that the size argument passed to them is limited to SIZE_MAX. Not so with calloc()

A compliant C implementation is not required to limit calloc() to allocating memory of only SIZE_MAX. The following may work. An single type can have a maximum size of SIZE_MAX and an array size can be as large as SIZE_MAX SIZE_MAX-1 "bytes", yet iptr below is not an array, but a pointer.

// Assume sizeof(double) == 8
double *iptr = calloc(SIZE_MAX, sizeof *iptr);

Re-allocating such large pointers is problematic as it requires using another call to calloc()


How to avoid overflow in realloc?

OP's problem is not so much of what realloc() can handle but of how code calculates the values passed to it may overflow.

To insure an unsigned type, like size_t, does not overflow multiplication:

  if (b && a > SIZE_MAX/b) Handle_Overflow();
  prod = a*b;
chux - Reinstate Monica
  • 143,097
  • 13
  • 135
  • 256
  • Can't an array size be as large as `SIZE_MAX`, with indices as large as `SIZE_MAX-1`? – ad absurdum Apr 03 '17 at 16:46
  • @DavidBowling Certainly an array size can be `SIZE_MAX-1` and I now certain you are correct that the size can even be 1 more or: `SIZE_MAX`. Yet I was not focusing on that issue nearly so much as that `calloc()` can allocate more memroy in one call than `SIZE_MAX`. Unfortunately I do find many implementations do not properly detect/handle overflow in `calloc()` and so such large allocations are thus inherently ID, even if the spec does not allow it. – chux - Reinstate Monica Apr 03 '17 at 16:56
  • The issue with `calloc()` was what I took to be the salient point; I had not noticed this difference between `calloc()` and `malloc()` before, but obvious now. Documentation about maximum size of arrays seems scattered in the Standard, but I found a reference in §6.5.3.4 of [revision 5.10 of the C99 Rationale](http://www.open-std.org/jtc1/sc22/wg14/www/C99RationaleV5.10.pdf) (PDF) that seemed informative. – ad absurdum Apr 03 '17 at 17:12
  • @DavidBowling The maximum size of array has nothing directly to do with`*alloc()` . Which is your concern, arrays, or allocation functionality/limitations? – chux - Reinstate Monica Apr 03 '17 at 17:16
  • Arrays. The statement "an array size can be as large as `SIZE_MAX-1`" had me looking for a reference. The best I could find in the C11 Standard was §6.7.6.2/1: "If they delimit an expression (which specifies the size of an array), the expression shall have an integer type." This had me wondering if an array could have more than `SIZE_MAX` elements, since this is only guaranteed to be at least 65535. It seems that `size_t` would need to be comparable to the widest `unsigned` integer type, but I don't see this in the Standard. What am I missing? – ad absurdum Apr 03 '17 at 18:11
  • @DavidBowling An array may have more than 65535 if `SIZE_MAX` is larger. As the `sizeof` yields the _size_: "The sizeof operator yields the size (in bytes) of its operand,", and its type is `size_t` and the max value of a `size_t` is `SIZE_MAX`, all arrays are <= `SIZE_MAX` in "byte" size. – chux - Reinstate Monica Apr 03 '17 at 19:13
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/139800/discussion-between-chux-and-david-bowling). – chux - Reinstate Monica Apr 03 '17 at 19:15
0

size_t is an unsigned type.

For unsigned types overflow behaviour is deterministic, as mentioned in C11

[....] because a result that cannot be represented by the resulting unsigned integer type is reduced modulo the number that is one greater than the largest value that can be represented by the resulting type.

So, as long as you make sure the operation which produces the value to be stored in the variable type size_t does not overflow in the process, passing that variable to realloc() has no impact. At most, realloc() will fail to allocate that memory.

Sourav Ghosh
  • 133,132
  • 16
  • 183
  • 261
  • I'm trying to stay `C99` compliant. Is this also in the `C99` standard? Do you imply I should just check `x*y < x || x*y < y` and safely fail in such a case? – user194192 Apr 03 '17 at 11:48
  • This should be the case in C99 also, yes. – Sourav Ghosh Apr 03 '17 at 11:51
  • Thank you! I'll give you my upvote as soon as I'm able to (currently not enough rep). Now I'm wondering how `calloc` handles this case? Will it fail as well of an overflow of `size_t` or will it use some kind of magic? – user194192 Apr 03 '17 at 11:56
  • @user194192 What do you mean how calloc handles? The approach should be same, it takes the value provided as `size_t` and multiply with block number. If the result is huge, probably `calloc(0` will signal failure. That's it. – Sourav Ghosh Apr 03 '17 at 11:59
  • @user194192 : `x*y < x || x*y < y` is *NOT* a good check for overflow. Take eg. `x = 3; y = 0x80000000;` (with a 32bit `size_t`). – Sander De Dycker Apr 03 '17 at 12:11
0

If x * y overflows size_t, realloc will try to allocate that value which was overflown:

bytes = (x * y) % 2^(sizeof(size_t) * 8)

So realloc will "see" bytes amount of bytes. It does not take care of anything.

unalignedmemoryaccess
  • 7,246
  • 2
  • 25
  • 40