How to read text chunks from a huge text file?

Question

I am trying to read a text file containing characters in billions. Using the function contentOfFile is not working, as my application get crashed due to it.

So anybody please send me the sample code so that I get the chunks according to my requirement.Whichever i need i wanna get that one only. please reply as soon as possible.

I know this is an old post, but your questions feels more like a command. Next time ask for help politely. — AndaluZ, Jun 08 '18 at 14:48

score 15 · Accepted Answer · edited Jun 12 '13 at 12:57

I'm guessing this is an iOS app. In that case, you are likely hitting the memory limit by calling contentsOfFile: because that method is trying to read the entire contents of the file into a variable (memory). Remember that on iOS your app must play nice and if it decides to consume too much memory, then the watchdog process will kill your app to save the device from rebooting (which happens because there is no disk to swap to on iOS devices).

Have you had a look at NSFileHandle? NSFileHandle supports seeking within a text a file. With some simple iteration you can use the following to methods to seek within the file and read chunks of data:

- (NSData *)readDataOfLength:(NSUInteger)length;
- (void)seekToFileOffset:(unsigned long long)offset;

It might look something like this. Assume pathToFile is an NSString containing the path to the text file to be read in.

uint64 offset = 0;
uint32 chunkSize = 1024;     //Read 1KB chunks.
NSFileHandle *handle = [NSFileHandle fileHandleForReadingAtPath:pathToFile];
NSData *data = [handle readDataOfLength:chunkSize];

while ([data length] > 0)
{
     //Make sure for the next line you choose the appropriate string encoding.
     NSString *dataString = [[[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding] autorelease];

     /* PERFORM STRING PROCESSING HERE */

     /* END STRING PROCESSING */
     offset += [data length];

     [handle seekToFileOffset:offset];
     data = [handle readDataOfLength:chunkSize];
}

[handle closeFile];

Thanks for the response, but you know what when i increase the value of `uint64 offset = 0;` from '0' to any upper integer i.e. 10, then it respond in some weird correctors, which are not the English letters — rptwsthi, Sep 21 '11 at 07:05
Please check this question, here first st2= is desired response, then after i have change the `offset` from `0 to 100` — rptwsthi, Sep 21 '11 at 07:09
http://stackoverflow.com/questions/7487761/problem-in-reading-from-file-recursively — rptwsthi, Sep 21 '11 at 07:23

score 1 · Answer 2 · answered Sep 20 '11 at 13:32

1

A good idea is to look at the textedit source because I've opened massive files with it before and there should be a way to do it. Not sure why your app is crashing though. It shouldn't have a problem.

answered Sep 20 '11 at 13:32

TheAmateurProgrammer

9,252
8
52
71

How to read text chunks from a huge text file?

2 Answers2