0

I write a test program with aio, but I find that when the main process of writing data is over, the memory usage of my process never falls down. Can any one give some tips about the problem of my program, or about aio cache?

My program:

#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <stdlib.h>
#include <aio.h>

FILE *db_file;

void aio_completion_handler(sigval_t sigval)
{
    int ret;
    struct aiocb *pb;  

    pb = (struct aiocb *)sigval.sival_ptr;  
    if ((ret = aio_return(pb)) > 0) {}

    printf((char*)pb->aio_buf);
    free((void*)pb->aio_buf);
    free((void*)pb);
}

int write_1000W()
{
    for(int i = 0; i < 10000000; ++i)
    {
        char *buf;
        const int BUFLEN = 1024*1024;
        struct aiocb *pb;

        buf = (char*)malloc(BUFLEN);
        bzero((void*)buf, BUFLEN);
        pb = (struct aiocb*)malloc(sizeof(struct aiocb));
        bzero((void*)pb, sizeof(struct aiocb));

        sprintf(buf, "{%d %d %d}\n", i, i, i);

        pb->aio_buf = buf;
        pb->aio_fildes = fileno(db_file);
        pb->aio_nbytes = strlen(buf);
        pb->aio_offset = 0;

        pb->aio_sigevent.sigev_notify = SIGEV_THREAD;
        pb->aio_sigevent._sigev_un._sigev_thread._function = aio_completion_handler;
        pb->aio_sigevent._sigev_un._sigev_thread._attribute = NULL;
        pb->aio_sigevent.sigev_value.sival_ptr = pb;

        if (aio_write(pb) < 0)
        {   
            free(buf);
            free(pb);
        }
    }
}

int main()
{
    db_file = fopen ("./1000W.txt", "a+");
    if(!db_file)
    {
        printf("Cannot open file for append.\n");
        exit(1);
    }
    write_1000W();
    printf("0000000000000\n");
    sleep(300);
    return 0;
}

The memory usage rises to more than 4G (some times 3G, may change), but the malloc and free functions both run 10,000,000 times, so I don't think its memory leak issue.

Ouroborus
  • 16,237
  • 4
  • 39
  • 62
Bruce
  • 1
  • If you remove all the aio specific code, does the memory issue still occur? – MrEricSir Nov 28 '16 at 02:16
  • Complie the pragram with "g++ file.c -lrt -o aio". – Bruce Nov 28 '16 at 02:25
  • Thank you for your inspiration. I'm trying this. @MrEricSir – Bruce Nov 28 '16 at 02:40
  • I remove the code from buf = (char*)malloc(BUFLEN); in for loop to the end of for loop, the memory issue does not accur. @MrEricSir – Bruce Nov 28 '16 at 02:45
  • Once memory is allocated, it is not returned to the o/s until the program exits. It is stashed away for reuse when it is freed; it will be supplied when a new request for memory is made. So, if you create a million AIO requests before any are completed, you will have allocated a million lots of the memory that's needed for each request — and that space won't be returned to the o/s until your program exits. Even if only a few thousand requests are created before you get held up, your program will have allocated a lot of memory. – Jonathan Leffler Nov 28 '16 at 03:27
  • I'm sure my callback function aio_completion_handler freed the buffer I allocate in for loop. You mean this memory block doesnot return to os? Or the aio have a cache to match my memory usage. @Jonathan Leffler – Bruce Nov 28 '16 at 04:01
  • Calling `free()` does not return the memory to the o/s; it stays allocated as a part of the memory available for use by the program. – Jonathan Leffler Nov 28 '16 at 04:03
  • You mean that my program frees memory and os marks it reusable for my program, when next request come, os allocates memory from the resuable memory. So the usage of memory will not expand too large. This makes sense. @Jonathan Leffler – Bruce Nov 28 '16 at 04:31
  • Yes, that is what I mean. – Jonathan Leffler Nov 28 '16 at 04:40
  • See http://stackoverflow.com/questions/1421491/does-calling-free-or-delete-ever-release-memory-back-to-the-system. – Mike Andrews Nov 29 '16 at 15:26

0 Answers0