0

I have a large (text) file that is divided into d "chunks". The user can read the next line from either of these chunks - the order is not specified. The file is not changed in the process.

There are two obvious solutions: Maintaining d input streams (one per chunk) or maintaining the current stream positions and jumping from chunk to chunk via the seekg-method with a single stream.

Is there any reason to assume that either method is (much) better than the other (in terms of efficiency)?

mike
  • 75
  • 2
  • I imagine the answer depends upon usage and how many chunks you're talking about. You might have to test it. I also imagine it varies by platform. – Joseph Larson Feb 19 '21 at 14:35
  • (1) From a design point of view, seek does not sound promising - evil centralization. I would use different streams. (2) If the performance is so critical, just run some tests. – zdf Feb 19 '21 at 15:08
  • You could also use functions depending on platform specific features, like boost's [memory mapped files](https://www.boost.org/doc/libs/release/libs/iostreams/doc/classes/mapped_file.html). Just memory map the file and jump back and forth using iterators and rely on the platform for efficiency. Posix's `mmap` does a good job and I assume the Windows variant does too. – Ted Lyngmo Jun 09 '23 at 12:44

0 Answers0