0

I'm working on Solaris 5.8, C++, using the Json parser.

The problem is: while parsing a file of size greater than 700 MB, the process crashes with core dump error. It roughly occurs at below code point - 

int printbuf_memappend(struct printbuf *p, char *buf, int size)
{
    char *t;
    if(p->size - p->bpos <= size)
    {
        int new_size = json_max(p->size * 2, p->bpos + size + 8);
        if (!(t = realloc(p->buf, new_size)))
            return -1;
        p->size = new_size;
        p->buf = t;
    }
    memcpy(p->buf + p->bpos, buf, size); // CORE DUMP HERE
    p->bpos += size;
    p->buf[p->bpos]= '\0';
    return size;
}

Could you please help to identify the problem? The core dump file contain only the data being copied. Can increase of RAM be a solution ? Or do I need to limit the file size to 700MB ?

WhozCraig
  • 65,258
  • 11
  • 75
  • 141
user1805010
  • 1
  • 1
  • 2
    If running out of memory were the problem, `realloc()` would fail. The problem is probably elsewhere in your program, and it corrupted memory so things are failing here. Have you used any debugging tools like `valgrind`? – Barmar Nov 07 '12 at 04:34
  • No I do not have much idea about debugging. I'd done several tests using files of sizes: 250MB, 500MB, 650MB, 700MB, 750MB. The process terminates at 700MB. Moreover the process terminates during the memcpy function only. – user1805010 Nov 07 '12 at 04:42
  • 1
    Just because it terminates here doesn't mean the problem is in this function. Memory corruption leaves a timebomb that can go off practically anwhere in the program. This function by itself looks fine, so you need to use debugging tools to catch the problem. – Barmar Nov 07 '12 at 04:45
  • What signal does it crash with? – Barmar Nov 07 '12 at 04:46
  • Only a core dump file is generated. Tail of that file comprises of the data to be copied. – user1805010 Nov 07 '12 at 04:59
  • That's meaningless. When the program crashes, you should see an error message with the signal that killed it, such as `Segmentation violation'. Or when you load the core dump into `gdb` it will tell you the reason the program stopped. Please go back and start debugging your program with proper tools, there's nothing we can do here to help you until you analyze it properly. – Barmar Nov 07 '12 at 05:02

1 Answers1

0

If crash happened in memcpy, you have two variants something wrong with input or output.

To test the second variant add memset after realloc:

    int new_size = json_max(p->size * 2, p->bpos + size + 8);
    if (!(t = realloc(p->buf, new_size)))
        return -1;
    p->size = new_size;
    p->buf = t;
    memset(p->buf + p->bpos, 0, size);

On Linux (depend on configuration) possible to allocate not existing virtual memory. The real allocation happens after the first usage. May the same happens on your Solaris? relloc return ok, but system really have no enought memory? memset should give answer to this question.

fghj
  • 8,898
  • 4
  • 28
  • 56