disable the randomness in malloc

Question

I'm running this following simple C program:

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char** argv){
  void *p = malloc(4);
  fprintf (stderr, "p==%p\n", p);
  return 0;
}

Different runs give different results:

p==0x101c010

then: p==0x1ad9010

then: p==0xe77010

and so.

As I remember, in the past malloc was fully deterministic. So probably from some version, some randomness was added to malloc. I am using now gcc-4.6.3 on Ubuntu.

Is there a way to eliminite that randomness ?

You definitely remember incorrectly. And no, there is no way to eliminate it. — Puppy, Nov 15 '13 at 14:36
@The downvoters: why the downvotes? I can't imagine ever wanting to do this, and OP almost certainly doesn't need to, but the question isn't off topic or anything, is it? — anjruu, Nov 15 '13 at 14:37
Think of all the bugs that would be hidden if malloc was predictable. — Charlie Burns, Nov 15 '13 at 14:37
You could not even guarantee that malloc/free/malloc within one process function would return two identical pointers. — Martin James, Nov 15 '13 at 14:40
With deterministic adresses, it's easier to debug. Of course, I'm talking about the virtual adress as the process see, and not the physical address. If I'll look at it using a dibugger, will it become deterministic ? — user2996526, Nov 15 '13 at 14:40
No. And what on earth debugging technique do you have in mind that would only be applicable to deterministic addresses? — Puppy, Nov 15 '13 at 14:41
@BartekBanachewicz I don't think he does. For better or for worse, program execution is fully deterministic, unless randomness is introduced artificially. On a given system, with a given implementation of malloc, his code would have, in the past, been fully deterministic, and always display the same value. Variation was introduced intentionally, for security reasons, but I'm sure that there are still implementations of malloc out there that don't introduce it. — James Kanze, Nov 15 '13 at 14:43
@BartekBanachewicz: Memory allocation used to be deterministic in practice, even though not guaranteed. This was simply a function of software behaving in a mechanistic manner. — Eric Postpischil, Nov 15 '13 at 14:43
@HAL: Being able to repeat program execution with identical memory allocations (and other values) can be useful for debugging. — Eric Postpischil, Nov 15 '13 at 14:44
@DeadMG: Here is a debugging technique that is applicable to deterministic addresses: A program is run, and, after a time, it is observed that incorrect data has been written to address x at some time in the past. If addresses are deterministic, then program execution can be repeated with a watchpoint set to address x, so that the operations that writes incorrect data can be detected. — Eric Postpischil, Nov 15 '13 at 14:47
Suppose I have some memory corruption. So I edit the source code and print the pointer values. I look at it, and then want to change some things in my code. Definetily I want see the privious values I saw before. Even more important, determinicity of the program is crucial for debug. I don't want a situation that on one run the exe crash, and on the other run it works fine. — user2996526, Nov 15 '13 at 14:49
@EricPostpischil You could also just watch the variable that's being modified. — Bartek Banachewicz, Nov 15 '13 at 14:49
@EricPostpischil - if only process execution was always determininistic. The number of apps that I write where that is the case is approximately zero. — Martin James, Nov 15 '13 at 14:51
@BartekBanachewicz: What variable? It’s an address, not a named object. It is some dynamically allocated object, perhaps part of a linked list or some other data structure. Or, even if it is a named object, it was not written to through the name but by an incorrectly used pointer somewhere else in the code. It is a **bug** we are looking for, so you cannot rely on orderly behavior. — Eric Postpischil, Nov 15 '13 at 14:51
@MartinJames: That is irrelevant. The fact that eliminating a source of non-deterministic behavior leaves other non-deterministic behavior still to be dealt with does not mean that eliminating this source is not useful. — Eric Postpischil, Nov 15 '13 at 14:53
@EricPostpischil - I cannot remember any time when such behaviour would have been useful to me, for debugging or any other purpose. — Martin James, Nov 15 '13 at 14:55
@MartinJames: What is useful to you is not a good basis for determining what is useful to other people who have specifically asked for something. There are people and situations for which this is useful (which is of course why there exists a mechanism to turn off address space layout randomization). — Eric Postpischil, Nov 15 '13 at 14:56
@EricPostpischil - I accept your argument there. If some developer wnats such behaviour, then they could be allowed the option of turning it on. OTOH, the number of apps where behaviour is repeatable for anything but a very short time must be zer.. very small:) — Martin James, Nov 15 '13 at 15:00

score 9 · Answer 1 · edited Apr 13 '17 at 12:22

9

If the variation is caused by address space layout randomization, then, according to this page, you can disable it with:

echo 0 | sudo tee /proc/sys/kernel/randomize_va_space

This should be done only temporarily for debugging purposes.

This is a good paper on interposing functions: Intercepting Arbitrary Functions on Windows, UNIX, and Macintosh OS X Platforms by Daniel S. Myers and Adam L. Bazinet. This would allow you to replace the malloc behavior with a completely controlled implementation.

edited Apr 13 '17 at 12:22

Community

1
1

answered Nov 15 '13 at 14:40

Eric Postpischil

195,579
13
168
312

Note that this will not always return the exact same addresses for the exact same malloc/free calls, as the userspace allocator itself might do something, and there are many other things that could influence placing of anonymous mmap regions, which are usually at least a part of how the allocators get memory from the OS (besides sbrk) – PlasmaHH Nov 15 '13 at 14:43
splendid! However notice that the malloc hooks are from glibc, not from gcc, and that they are very unfortunately deprecated, and no immediate replacement planned. One can however always link against a different allocator, or use LD_PRELOAD to preload one (at least on linux). – PlasmaHH Nov 15 '13 at 16:00
@PlasmaHH: Okay, I have deleted the old hook information. The paper is a good starting point for interposing functions. – Eric Postpischil Nov 15 '13 at 16:11

score 7 · Answer 2 · answered Nov 15 '13 at 14:37

First of all, there is absolutely no guarantee of memory manager determinism in C. You cannot rely on it even if you see it, period.

Second of all, nowadays a lot of environments employ address layout randomization primarily to ensure that exploits of various buffer overflows cannot rely on a deterministic addresses and thus use them to execute arbitrary code.

score 0 · Answer 3 · answered Nov 15 '13 at 14:38

0

Malloc obtains memory from the operating system, which assigns memory that is not in use.

Which addresses that will be returned will depend heavily on the prior use of the computer.

answered Nov 15 '13 at 14:38

Edwin Buck

69,361
7
100
138

Each process has a virtual address space separate from others. – Eric Postpischil Nov 15 '13 at 14:39
1

Huh? Most modern OS use _virtual_ address space for programs. – Alexander L. Belikoff Nov 15 '13 at 14:39
C is used in far more environments than the _current_ set of modern operating systems. – Edwin Buck Nov 15 '13 at 14:41
The OP specifically asks about Ubuntu. It is clear the reason stated in this answer is not the reason for the observations reported by the OP. – Eric Postpischil Nov 15 '13 at 14:41
@AlexanderL.Belikoff It is still memory that is not in use. It is still assigned by the operating system. I didn't make any indication as to the offsets or mappings to physical hardware addresses (or even mentioned physical hardware addresses). You are reading far more into my answer than I wrote. – Edwin Buck Nov 15 '13 at 14:44
@AlexanderL.Belikoff At which point this answer states that OS doesn't use virtual address space? – BartoszKP Nov 15 '13 at 14:45

Peter Bloomfield · Answer 4 · 2013-11-15T14:50:59.373

0

Given precisely the same conditions, the result of malloc() will always be the same. (Like anything in computing, there's no such thing as true randomness.) However, the determining conditions extend beyond your program's state or control.

It should never matter though. Unless you're attempting some very weird form of memory mapping, there's really no reason to care where an allocation takes place.

edited Nov 15 '13 at 14:50

answered Nov 15 '13 at 14:41

Peter Bloomfield

5,578
26
37

I totally don't agree. A program with deterministic behaviour is much easier to debug. Consider a bug that one writes an array, and overflows it. Now, in different allocations, different things will be corrupted, and each time you run it, you see different buggy behaviour, and maybe sometimes, a clean run. Even if malloc get some randomness (from the clock for example), I think some info need to be given (like a seed), that enebles the developer to reproduce the exact situation. – user2996526 Nov 15 '13 at 15:05

score 0 · Answer 5 · answered Nov 15 '13 at 14:41

0

A direct answer to your question: no. The C standard for malloc says nothing about it being deterministic, so the implentors are free to write it any way they think will make it most efficient.

answered Nov 15 '13 at 14:41

Barzee

897
3
15
30

I don't want C to guarantee anything. Obviously the malloc implementation access the clock, or some global information taken from the OS. otherwise it would have been deterministic. I would expect 1. A flag to disable it 2. In case that malloc uses randomness, some seed that enable me to reproduce the specific run. Otherwise how canI possibly debug the program ? – user2996526 Nov 15 '13 at 15:13

score 0 · Answer 6 · answered Nov 15 '13 at 14:47

It's perfectly possible for a given malloc implementation to be deterministic in response (generally when talking about whether an implementation of malloc is or isn't "deterministic" it's in terms of time taken that people care, as in whether it could be used in RT programming). It would depend on the virtual memory system in use, the code itself, and how deterministic the question of calls to it or an underlying memory manager by other code in the same process space was, but given the right conditions for all of those, you can have a deterministic malloc.

All that said though, there's no reason why it should be. After all, any code that was depending on particular addresses being used could just ignore malloc and write the the memory directly - it knows it's free to use, after all!

As such, since there's zero value in maintaining such deterministic behaviour, if it does happen it's purely an accident of several different implementation choices, and while they would be a bit interesting (it would be interesting to know just how such an implementation ended up being deterministic) they certainly aren't what one would expect.

disable the randomness in malloc

6 Answers6

Linked

Related