2

I wanted to speed up compilation so i was thinking i could have my files be build on a ramdisk but also have it flushed to the filesystem automatically and use the filesystem if there is not enough ram.

I may need something similar for an app i am writing where i would like files to be cached in ram and flushed into the FS. What are my options? Is there something like this that already exist? (perhaps fuse?) The app is a toy app (for now) and i would need to compile c++ code repeatedly. As we know, the longer it takes to compile when there is a specific problem to solve before progressing. the less we can get done.

3 Answers3

2

Ram-disks went the way of the dodo with the file system cache. It can make much better decisions than a static cache, having awareness of RAM usage by other programs and the position of the disk write head. The lazy write-back is for free.

Hans Passant
  • 922,412
  • 146
  • 1,693
  • 2,536
  • are you saying that windows will cache my files so i wouldnt need to implement this? –  Feb 26 '09 at 18:53
  • Is this true in general, or does the developer who wrote the software have to do anything to tell Windows it should use the file system cache? In other words, will my simplistic C code benefit from system cache when it writes large files? And if so, then why do I see a dramatic speedup when I write to a RAMDISK? (Note I am using OpenCV to write out large video files one frame at a time). – AndyL May 03 '13 at 14:41
1

Compilation is CPU-bound, not disk bound. If you utilize all your CPU cores using the appropriate build flag, you can easily saturate them on typical PCs. Unless you have some sort of supercomputer, I don't think this will speed things up much.

For VS2008 this flag is /MP. It also exists on VS2005.

Nick
  • 13,238
  • 17
  • 64
  • 100
1

Not sure it helps answering a question from almost 12 years ago, but just now I was looking for some software to sync file systems or directories to do exactly that. Both answers are correct, in the sense that the file system cache will attempt to predict what you need and make available in RAM, and, generally, building a software benefits a lot of multithreading, possibly maxing out your CPU. But right now I'm building with Unity and I don't have control over build optimization.

That said, a virtual RAM disk has advantages, because those files are always in RAM, different from file system cache (FSC) that has to deal with resources and applications competing for disk access.

Another difference is that when an application closes a file handle or forces a sync, FSC will attempt to get that files as soon as possible to the disk, in order to avoid problems (power failure and so on). I believe you can alter the FSC behavior on Linux. A sync to a RAM disk does not write to disk, that may be responsible for the difference of performance you said in your comment.

That said, I still need to look for something to auto-sync my two file systems!

João Carrion
  • 11
  • 3