Segmentation fault while accessing particular array location/index

Question

I used the malloc function for allocating 10^9 memory locations as part of an array.

The Code 1 gets executed successfully.

Code 1:

int main(){

    int size = (int)(1e9);

    int* arr = (int*) malloc( size * sizeof(int) );

    for(int i=0;i<size;i++){

        arr[i] = i;
    }    

    return 0;
}

But when I tried to access the particular memory location or index value = 12345678 (which is < 1e9), I got segmentation fault

Code 2:

int main(){

    int size = (int)(1e9);

    int* arr = (int*) malloc( size * sizeof(int) );

    for(int i=0;i<size;i++){

        arr[i] = i;
    }    

    cout<<arr[12345678]<<endl; //added this line of code, which gives segmentation fault

    return 0;
}

My guess is, this occurs due to memory fragmentation, but I am not sure about this. Can anyone kindly explain the correct reason.

`std::vector` is your friend. `malloc` and `new` are not your friends. — NathanOliver, Dec 16 '19 at 14:33
Welcome to over-committing! On many operating systems, you can allocate as much as you want and only get errors when you actually _access_ more than there is available. See also https://stackoverflow.com/questions/48585079/malloc-on-linux-without-overcommitting and https://www.win.tue.nl/~aeb/linux/lk/lk-9.html#ss9.6. — Max Langhof, Dec 16 '19 at 14:34
@Max Langhof, My concern is, that the loop got executed successfully. We were accessing the memory their as well, but it got executed. — Deepak Tatyaji Ahire, Dec 16 '19 at 14:35
@DeepakTatyajiAhire Are you building this with optimizations on? Because a compiler can trivially see that the loop does nothing in the first program and just eliminate it (and the rest of the program too): https://godbolt.org/z/7q3ofp. But in the second program that's not so easy, as it has to actually output one of the values. — Max Langhof, Dec 16 '19 at 14:38
In your "Code 1" the compiler may well completely optimize-away the `for` loop, as it doesn't do anything that has any further noticeable effect. — Adrian Mole, Dec 16 '19 at 14:38
@MaxLanghof, I executed it on Hackerrank platform. Not sure if optimisations are on. Can u kindly tell how to check it? — Deepak Tatyaji Ahire, Dec 16 '19 at 14:40
@DeepakTatyajiAhire You did not check to see if `malloc` returned NULL. Thus the posted program's behavior cannot be determined after the call to `malloc`. — PaulMcKenzie, Dec 16 '19 at 14:59
I fiddled with the platform you mentioned and it seems there're are limits on amount of memory that you can allocate. If you carefully check return value of malloc you'll get NULL, plus with out print statement it runs because compiler optimisation removes the entire for loop. — moghya, Dec 16 '19 at 15:06

score 2 · Accepted Answer · answered Dec 16 '19 at 15:08

2

It's definitely

Compiler optimisation
Limit on memory allocation as per standard sandbox

answered Dec 16 '19 at 15:08

MC_user

48
3

score 1 · Answer 2 · answered Dec 16 '19 at 14:40

Modern operating systems are doing something called "lazy allocation", this means that when you ask to allocate memory, they give it to you without actually allocating the memory.

This is exactly the reason for the Virtual memory vs Physical memory taken you see while using top.

The memory is only really allocated when you try to use the buffer itself.

In your example, you've allocated a lot of memory (4GB), the OS let you allocate it but did not actually free up the memory needed for it, once you've tried using it, it tried to actually provide the memory you asked for, couldn't do it, and therefore crashed.

Segmentation fault while accessing particular array location/index

2 Answers2