I have a file which is stored on disk and the records are written to the file with a custom record format. A bunch of records will represent a block. When you perform a read in the disk, you can only fetch a block instead of a record.
Block = physical record.
Data = logical record.
Every logical record has a unique offset in the file. It's closer to leveldb's record format.
Given a bunch of offsets, how would you efficiently read a logical record?
In practice, on most current operating systems and hardware, it does not matter much.
Read Operating Systems: Three Easy Pieces for more.
First, the operating system maintain a page cache, which contains recently read or written pages. See also linuxatemyram. So many disk IO operations are not immediately done by the hardware, and they are not done in the order you requested them. Read more about hard disk drive performance.
Then, recall that more and more disks are SSDs (without any rotational delay, and a constant access time). And current hard disk drives don't use CHS but LBA so the block address is not immediately related to its physical position (and the disk controller itself handles the mapping).
In some cases, the kernel disk driver and file system would reorganize pending disk requests.
However, you'll better do disk IO on large enough buffers (typically, several pages of 4Kbytes at least for each read(2) or write(2)).
On Linux, see also the sync(2), fsync(2), posix_fadvise(2), readahead(2), mmap(2), madvise(2) system calls
Given a bunch of offsets, how would you efficiently read a logical record?
You might order these offsets, but that practically won't change much and probably not at all. Be sure to benchmark.
In practice, use some library like leveldb or gdbm (indexed files) or sqlite. You could tune some parameters (buffer sizes), but you should trust the library for the rest.
