0

My coworker and I are having a killer of a time trying to figure out this problem. We want to parse a file (custom made 3d object file). The problem is in using the stdio function calls like fread, fseek, etc.

In the below code, when I check the virutal memory before call and after call I am missing like 1MB each time the function is run and I have no idea why this happens.

FILE *fp= fopen([path UTF8String], "rb");
char *buffer = (char *) malloc(MAX_STRING_LENGTH);
while(!feof(fp))
{
    fgets(buffer, MAX_STRING_LENGTH, fp);
}

if(0 == fclose(fp))
    NSLog(@"File is closed");

free(buffer)

Path is generated from ios Library and MAX_STRING_LENGTH is an arbitrarily large number (eg 2000000)

Also used the function from http://www.cplusplus.com/reference/clibrary/cstdio/fread/ in the example and still the memory error persists.

I know that buffer is not the problem because if I use just create the buffer and free it then no memory is taken but if I use the while loop then after the file is closed and I check the memory available I have one less MB than before. it has been driving me and my coworker crazy trying to fix this for the past week.

EDIT::

- (float)print_usage_memory
{    
vm_statistics_data_t vmStats;
mach_msg_type_number_t infoCount = HOST_VM_INFO_COUNT;
host_statistics(mach_host_self(), HOST_VM_INFO, (host_info_t)&vmStats, &infoCount);

const int availablePages = vmStats.free_count;
float availableMemory = [self pagesToMB:availablePages];

/*
 */     
const int totalPages = vmStats.wire_count + vmStats.active_count + vmStats.inactive_count + vmStats.free_count;
const int activePages = vmStats.active_count;
const int wiredPages = vmStats.wire_count;
const int purgeablePages = vmStats.purgeable_count;   

NSMutableString* txt = [[NSMutableString alloc] initWithCapacity:512];
[txt appendFormat:@"\nTotal: %d (%.2fMB)\n", totalPages, [self pagesToMB:totalPages]];
[txt appendFormat:@"                              nAvailable: %d (%.2fMB)\n", availablePages, availableMemory];
[txt appendFormat:@"                                 nActive: %d (%.2fMB)\n", activePages, [self pagesToMB:activePages]];
[txt appendFormat:@"nWired: %d (%.2fMB)\n", wiredPages, [self pagesToMB:wiredPages]];
[txt appendFormat:@"nPurgeable: %d (%.2fMB)\n", purgeablePages, [self pagesToMB:purgeablePages]];

NSLog(@"%@", txt);
[txt release];
txt = nil;

return availableMemory;
}


- (float)pagesToMB:(const int)pages
{
    return pages*4/1024;
}

Thanks for any help you might be able to give, or any insights into this.

Raigex
  • 1,205
  • 12
  • 32
  • 3
    How do you measure the memory? I don't know about iOS, but many implementations of free marks the freed memory as available for re-use in the current process, but doesn't return it to the operating system. In that case, a process monitoring tool in iOS would still consider that memory as used by the process. – Thomas Padron-McCarthy Nov 12 '12 at 16:21
  • 1
    As an aside, I'd be inclined to use [`stringWithContentsOfFile:encoding:error`](https://developer.apple.com/library/mac/documentation/Cocoa/Reference/Foundation/Classes/NSString_Class/Reference/NSString.html#//apple_ref/doc/uid/20000154-BAJIAAFJ) or [`dataWithContentsOfFile`](https://developer.apple.com/library/mac/documentation/Cocoa/Reference/Foundation/Classes/NSData_Class/Reference/Reference.html#//apple_ref/doc/uid/20000172-CIHCGGDJ) rather than `stdio` functions. – Rob Nov 12 '12 at 16:24
  • well the thing is when I use free it does release my memory, I can see it being returned (plus or minus a couple kilobytes). The problem is when I use the fread function. It does not release release the memory. Although it might have something to do (that i just thought of) that even when I malloc the memory is not actually allocated until something is written into it (i think that is how it works) and I posted the code of how i find memory usage above. – Raigex Nov 12 '12 at 16:28
  • @Rob I tried those functions before but they have even worse memory leaks then these and are slower to parse the way I need them. – Raigex Nov 12 '12 at 16:29
  • wait, 1) why do you expect fread to release memory and 2) why do you keep saying fread when i see fgets – im so confused Nov 12 '12 at 16:30
  • sorry, it should be fgets, I had the wrong title but I also had the same problem with fread. As for memory I know that fread allocates memory in the kernel (so does fgets i belive) and I think it might not be released properly but I am not sure. Thats why I am asking if anyone can see anything wrong with my above code. – Raigex Nov 12 '12 at 16:32
  • Do you have an autorelease pool set up around this? (There's not much but there is the UTF8String call at least). Also, "File fp = fopen(...)" should "FILE *fp = fopen(...)". What's this "File" thing? – Graham Perks Nov 12 '12 at 16:34
  • I'd be tempted to write this off to caching, but you're saying that it leaks 1MB each time you call this code? – Graham Perks Nov 12 '12 at 16:35
  • wait, is C on iOS different from regular C? fgets and fread do not allocate memory to my knowledge (which is limited, I admit) – im so confused Nov 12 '12 at 16:36
  • @GrahamPerks I only have the regular iOS autorelease pool (the one that is created when application starts) and yes it is FILE *fp, i wrote the above code freehand and must have missed the capslock on it. will fix now and yes there is around 1MB less memory available each time this code is called. – Raigex Nov 12 '12 at 16:38
  • @AK4749 I have read that fgets and fread allocated their own buffers (on stackoverflow in several threads), in the kernal, thus I thought the still did that. – Raigex Nov 12 '12 at 16:40
  • @Raigex huh, interesting, I was unaware of that – im so confused Nov 12 '12 at 16:43

2 Answers2

0

A couple thoughts:

  1. If you're running your test in a loop, make sure to drain the autorelease pool inside the loop. Otherwise, it won't get drained until after your test, and it will mess up your measurements. For example:

    for(int i = 0; i < TEST_COUNT; i++)
{
    // If you're using ARC, you can instead replace this with an
    // @autoreleasepool block
    NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
    run_my_fopen_fgets_fclose_test();
    [pool release];
}

  2. Try using unbuffered I/O by calling setbuf(fp, NULL) on the FILE* object immediately after opening it. Normally, stdio buffers the data by allocating a buffer with malloc so that it can read the file data in large chunks, and this buffer is deallocated with free when you call fclose. You are closing the file, so this shouldn't be leaking, but just in case it's screwing with your memory measurements, it may be worth a try to use unbuffered I/O to get cleaner results.

Adam Rosenfield
  • 390,455
  • 97
  • 512
  • 589
  • Ok I will try that. Even though my loop is not that kind (eg I use Augmented Reality and scan a target and show a model) but I will try the autorelease pool. Aswell as option 2. – Raigex Nov 12 '12 at 17:04
  • I tried both and at first I thought setbuff(fp, null) would work but after 2 scans i saw a 2mb drop. And NSAutoreleasePool did nothing to help, but thanks for your info. – Raigex Nov 12 '12 at 17:55
0

I think we have figured it out. The problem is with the Autorelease pool used by ios which actually releases the memory either when it wants to, the application is turned off, and or when something actually requires the memory that was set for release to actually be released. Noticed it after getting the a ton of memory back after getting low (about 3mb left) to almost what it started with (eg it started with 230mb and i now have 225). This was a weird bug because we expected all this to be released right when it was done not marked for release and acutally released when needed.

Raigex
  • 1,205
  • 12
  • 32