new and malloc allocates extra 16 bytes

Question

I'm writing on c++ in VS2010 Windows 7. I try to read file of size 64 bytes. Here's the code:

    BYTE* MyReadFile(FILE *f)
{
    size_t result;
    BYTE *buffer;
    long lSize;
    if (f == NULL) 
    {
        fputs ("File error", stderr); 
        exit (1);
    }

    fseek (f, 0, SEEK_END);
    lSize = ftell (f);
    rewind (f);

    //buffer = (BYTE*) malloc (sizeof(char)*lSize);
    buffer = new BYTE[lSize];
    if (buffer == NULL) 
    {
        fputs ("Memory error", stderr); 
        exit (2);
    }

    result = fread (buffer, 1, lSize, f);
    if (result != lSize) 
    {
        fputs ("Reading error",stderr); 
        exit (3);
    }

    fclose (f);
    return buffer;
}

When I get file size it is 64, but when I allocate memory for it with new BYTE[lSize] I get 80 bytes of space and thus strange sequence ээээ««««««««оюою is added to the end of buffer. Can you please tell me how to handle this?

How are you printing it out? If it's a C-string, you need to NULL-terminate. — Mysticial, Feb 22 '12 at 07:51
Insert `int buffSize = ftell(buffer)` after you the `result = fread (buffer, 1, lSize, f);` and tell me the value of buffSize. Also why do you mix c (fseek, ftell, etc) and c++ (new)? Why don't you stick to one or the other? — Alexander, Feb 22 '12 at 07:56
@Mysticial: null-terminate, not NULL-terminate (`NULL` is a null pointer constant). — Keith Thompson, Feb 22 '12 at 08:07
@KeithThompson Sigh... Haha, this isn't the first time I've been called out for that. Knowing myself, it's gonna take another 5 times before I finally remember... :-P — Mysticial, Feb 22 '12 at 08:16
You may use "`buffer, 64`" in watch window to see first 64 elements of `buffer`. — Ajay, Feb 22 '12 at 08:30

score 4 · Answer 1 · answered Feb 22 '12 at 07:57

4

What is behind and above is called sentinel.It is used to check if your code does not exceed boundary of allocated memory.When your program overwrite this values, CRT library will report debug messages when you release your buffer.

Look here : http://msdn.microsoft.com/en-us/library/ms220938%28v=vs.80%29.aspx

enter image description here

answered Feb 22 '12 at 07:57

rkosegi

14,165
5
50
83

Correct, but not relevant for this problem. The real problem is the nul-terminator. – Patrick Feb 22 '12 at 08:08

score 4 · Accepted Answer · answered Feb 22 '12 at 07:58

There is an important difference between the number of bytes you have allocated, and the number of bytes that you see.

If lsize is 64, you have indeed allocated yourself 64 bytes. This does not mean that behind the screen the C++ run time will have asked exactly 64 bytes to Windows. In practice memory managers ask slightly more memory so they are able to do their own homework. Often these extra bytes are allocated BEFORE the pointer you get back from new/malloc so you will never see them.

However, that is not your problem. The problem is that you read 64 bytes from file using fread. There is no way that fread knows what kind of data you are reading. It could be a struct, a char buffer, a set of doubles, ... It just reads these bytes for you.

This means that if the file contains the characters "ABC" you will get exactly "ABC" back. BUT, in C, strings should be nul-terminated, so if you pass this buffer to printf, it will continue to scan memory until it finds a nul-character.

So, to solve your problem, allocate 1 byte more, and set the last byte to the nul character, like this:

buffer = new BYTE[lSize+1]; 
buffer[lSize] = '\0';

+1 I see your point. He is relying on there being a `\0` later in some random piece of memory. If there were not he would get a bus error trying to print it. — Peter Lawrey, Feb 22 '12 at 08:08
It worked. But can you tell me the proper way to read/write data to/from files in c++? — forik, Feb 22 '12 at 08:55
use std:ifstream and std::ofstream. See http://www.cplusplus.com/reference/iostream/ for an introduction and examples. — Patrick, Feb 22 '12 at 09:16

Necrolis · Answer 3 · 2012-02-22T08:03:57.473

3

Although this may look like a memory problem, its actually a printing problem (as @Mystical pointed out). You need to put a null termination if you are going to print out anything as a string, else memory will be wildy read till one is encountered (which is UB).

Try this instead:

buffer = new BYTE[lSize + 1];
if (buffer == NULL) 
{
    fputs ("Memory error", stderr); 
    exit (2);
}

result = fread (buffer, 1, lSize, f);
if (result != lSize) 
{
    fputs ("Reading error",stderr); 
    exit (3);
}

buffer[lSize] = '\0';

It'll ensure there is a null terminator at the end of the returned buffer.

edited Feb 22 '12 at 08:03

answered Feb 22 '12 at 07:57

Necrolis

25,836
3
63
101

NULL check after new is bogus. operator new throws exception, and wont return null if memory allocation fails. – Ajay Feb 22 '12 at 08:28
@Ajay: I merely copied his code and fixed it for this problem, also, certain compilers/CRT impl's *will* return `NULL` and throw an exception: http://msdn.microsoft.com/en-us/library/kftdy56f(v=vs.71).aspx – Necrolis Feb 22 '12 at 08:35

score 1 · Answer 4 · answered Feb 22 '12 at 07:52

1

When memory is allocated, it is not on a per byte basis. Instead it is allocated in aligned blocks of 8 or 16 bytes (possibly with a header at the start, before the pointer). This is usually not a problem unless you create lots (many millions) of little objects. This doesn't have to be a problem in C and isn't even a major problem in Java (which doesn't support array of objects or objects allocated on the stack).

answered Feb 22 '12 at 07:52

Peter Lawrey

525,659
79
751
1,130

1

Correct, but not relevant for this problem. – Patrick Feb 22 '12 at 07:59
There are two solutions; don't worry about it, you don't need to know that more space is used (16 bytes or temporary space is trivial on any PC), or allocate on the stack which doesn't have this problem. – Peter Lawrey Feb 22 '12 at 08:06

new and malloc allocates extra 16 bytes

4 Answers4

Linked