0

I want to write a program that writes data as one contiguous block of data to disk, so that when I read that data back from the disk, I can just read one long series of bytes without stopping. Are there any references I can be directed to regarding this issue?

I am essentially asking whether or not it is possible to write data for multiple files contiguously and read past an EOF, or many, to retrieve the data written.

I am aware of fwrite and fopen, I just want to be sure that the data being written to disk is contiguous.

dbc
  • 104,963
  • 20
  • 228
  • 340
Novice Programmer
  • 63
  • 2
  • 7
  • `fwrite` and `fread` – user3528438 Aug 28 '15 at 13:23
  • http://stackoverflow.com/questions/15752339/contiguously-space-on-hard-disk-ntfs – Sorin Aug 28 '15 at 13:24
  • Questions asking us to recommend or find a book, tool, software library, tutorial or other off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it. – too honest for this site Aug 28 '15 at 13:24
  • You did not even try to read a C book, right? – too honest for this site Aug 28 '15 at 13:25
  • I am aware of fread and fwrite, but I don't know how the internals of the functions work. How does fwrite guarantee that what is being written to disk will be one contiguous block of data. – Novice Programmer Aug 28 '15 at 13:25
  • I have access to the K&R book, if that is what you are referring to. – Novice Programmer Aug 28 '15 at 13:29
  • `Internally, the function interprets the block pointed by ptr as if it was an array of (size*count) elements of type unsigned char, and writes them sequentially to stream as if fputc was called for each byte.` http://www.cplusplus.com/reference/cstdio/fwrite/ – user3528438 Aug 28 '15 at 13:32
  • Not much of anything in this world would work if `fwrite` didn't produce contiguous data when the file is read back. Whether the data is physically contiguous on the disk is another question that would require a device driver tag. – Carey Gregory Aug 28 '15 at 13:38
  • This is one of the simplest requirements I've ever seen:) – Martin James Aug 28 '15 at 13:38
  • @kdibene1 Files are a sequential stream of bytes. All the file related APIs and functions read/write data sequentially to or from a file, you will need to explicitly go out of your way in order to not do that. – nos Aug 28 '15 at 13:41
  • @nos yeah but he wants to burst read the whole file – Pyjong Aug 28 '15 at 13:43
  • Well, that's up for debate. It's not in any way clear what the OP actually wants to do. – nos Aug 28 '15 at 13:44
  • @user3528438, thank you for your reply. Is there any way to know whether fwrite is writing to disk or memory? – Novice Programmer Aug 28 '15 at 14:30
  • @nos, I am trying to create a large storage structure with fast retrieval. To optimize the retrieval process, I want to be able to read, say for example, files 1-7. Instead of doing 7 read operations, I would just simply like to be able to read the contiguous blocks starting from file1 to file7 and recompose them on my end. I hope that is clear. I am basically trying to minimize I/O overhead. – Novice Programmer Aug 28 '15 at 14:33
  • @stupid_idiot I would like to burst read many files at once, not simply one file. If this were a question about just one file, it would be silly and trivial. – Novice Programmer Aug 28 '15 at 14:34
  • @kdibene1 It writes to a `FILE *`, which is the return value of a successful call `fopen()`. – user3528438 Aug 28 '15 at 14:44
  • @kdibene1 You should update your question to make it clear that you want to open one file and read past `EOF`, which I think is impossible. – user3528438 Aug 28 '15 at 14:47
  • @user3528438 I understand the basics of writing to a file in C, I am just asking about specific low-level implementation details. Is there any way I could use GDB to verify that the data is being written contiguously? – Novice Programmer Aug 28 '15 at 14:48
  • @user3528438 I updated my question, thanks. – Novice Programmer Aug 28 '15 at 14:51
  • @nos So, essentially this is not possible? If I use fwrite for multiple files, there is no guarantee that it will be contiguously written to disk? – Novice Programmer Aug 28 '15 at 16:41

2 Answers2

0

It depends on what the underlying filesystem is, as this is filesystem-dependent. You'll want to look at extents, which are a contiguous area of storage reserved for a file.

Rob Percival
  • 58
  • 5
  • Thanks for your reply. There isn't a way to make it universal for any file system? I would like it to be file system independent. – Novice Programmer Aug 28 '15 at 14:21
  • @kdibene1, no there is no way to make this universal to any file system. And even then the media itself can move things around. For example SSD drives routinely re-map blocks on writes. – josh poley Aug 28 '15 at 16:08
0

On Windows you can open an unformatted volume with CreateFile and then WriteFile a contiguous block of data. It won't be a file, but you will be able to read it back as you stated.

According to this NTFS tries to allocate contiguous space if possible, your chances are lower when appending though.

Pyjong
  • 3,095
  • 4
  • 32
  • 50
  • This is specific to Windows though, I am searching for a generic way to do it in C. – Novice Programmer Aug 28 '15 at 14:46
  • C by itself does not offer anything beyond it's runtime library. You will have to do it like the rest of us. Write up those generic functions yourself and then their implementations for all the platforms you want to deploy to. – Pyjong Aug 28 '15 at 17:27