5

For the game Minecraft, the general approach when running the server application is to run it in a RAMDisk, as it is uses hundreds of tiny files for world generation, and the I/O speeds are the major bottleneck.

In a recent attempt, I tried to use Dokan/ImDisk to create a RAMDisk programatically for the Server Application. Dokan was considerably slower than the average Hard-Drive, and I was unable to get ImDisk to function properly. Since these are the only 2 Filesystem Drivers I know of that have a .NET API, i'm looking into alternatives now.

It was mentioned to me previously to try Memory-Mapped Files. My approach currently is to Create RAMDisk, Create Symbolic Link between Data Folder for Game Server and the RAMDisk, then launch the Game Server process.

  • Can Memory-Mapped Files function the same way, I.E. creating a virtual drive which I can create a symbolic link to? Such as G:\Data_Files\?

  • Are there any other alternatives to Dokan/ImDisk with a .NET API/Bindings floating around?

  • Not really sure where C#/.net comes into this. – CodesInChaos Aug 13 '11 at 10:58
  • But how does your application need to interact with the RAM disk driver? Can't you just load an existing RAM disk driver. Not really sure what you need a C# application for if you want to run a minecraft server. And if it's your own application not minecraft , why are you using many little files in the first place? – CodesInChaos Aug 13 '11 at 11:27
  • RAM disks are a relic from the DOS days. It is a precious resource on a modern multitasking operating system, far too precious to waste on storing files. Such operating systems have a file system cache which works just like a RAM disk. But with the considerable added advantage that the operating system can dynamically adjust its size as necessary to find a good balance between the file system and processes that need memory. If there's a ram disk driver that can do a better job than Windows then it is a well kept secret. – Hans Passant Aug 13 '11 at 12:25
  • @Hans this is not exactly so. You can have RAM disk which is several Gb large, and cache is much smaller. There indeed exist several RAMDisk products, some of which even support more than 4Gb on 32-bit systems, and they perform perfectly, especially in tasks like keeping temporary files. – Eugene Mayevski 'Callback Aug 16 '11 at 18:16
  • 1
    Just use a 64-bit operating system. Your advice is clouded by your business interests. – Hans Passant Aug 16 '11 at 19:36

2 Answers2

3

After looking at a bunch of solutions and doing a few benchmarks, we couldn't pass up RAMDisk from DataRam. We kicked around a bunch of the Windows driver stuff and some other freebie solutions and ultimately couldn't justify the expense compared to the tiny price tag of a commercial solution.

Jonathan Oliver
  • 5,207
  • 32
  • 31
  • Instead of using a Windows comercial RAMDISK, if using Win8 or Server 12 or newer could one run a linux VM, mount a tmpfs (RAMDISK), then share that mounted RAMDISK over SAMBA http://serverfault.com/questions/267116/improving-exim-performance-mounting-message-queue-on-ramdisk – yzorg Jul 17 '13 at 17:36
  • 3
    Ouch! The overhead to make that work is incredible! You're almost better off figuring out how to get the software running on Linux. Some RAM disk solutions are free, while others are free up to N GB of RAM. If it's really that critical to the application, get a native driver for Windows or run the software on Linux. – Jonathan Oliver Jul 18 '13 at 02:24
3

There are several approaches that depend on specifics of your task.

If you need to work with file system (i.e. via filesystem API functions and classes), and you want it fast, then (as I suggested in reply to your previous question) you'd need to create a RAMDisk driver. Windows Driver Kit includes a sample driver, which (coincidence?) has the name "RamDisk". Driver development, though, is tricky, and if something goes wrong with the sample or you need to extend it, you would need to dig deep into kernel-mode development (or hire someone to do the job). Why kernel mode? Cause as you could see with Dokan, switching back to user mode to store the data causes major slowdown.

If all you need is a handy management of bunch of files in memory using Stream class (with possibility to flush all of this to the disk), then you can make use of one of virtual file systems. Our SolFS (Application Edition) is one of such products that you can use (I can also remember CodeBase File System, but they don't seem to provide an evaluation version). SolFS seems to fit your task nicely so if you think so too, you can contact me privately (see my profile) for assistance.

To answer your questions:

  1. No, memory-mapped files (MMF) are literally files on the disk (including a virtual disk if you have one), which can be accessed not via filesystem API but directly using in-memory operations. MMFs tend to be faster for most file operations, that's why they are frequently mentioned.

  2. Our Callback File System or CallbackDisk products (see virtual storage line) are an alternative, however, as I mentioned in the first paragraph, they won't solve your problem due to user-mode context switch.

Update: I see no obstacles for the driver to have a copy in memory and perform writes to disk asynchronously when needed. But this will require modifying sample RAMDisk driver (and this involves quite a lot of kernel-mode programming).

With SolFS or other virtual file system you can have a copy of the storage on the disk as well. In case of virtual file system it might appear that working with container file on the disk will give you satisfactory results (as virtual file system usually has a memory cache) and you won't need to keep in-memory copy at all.

Eugene Mayevski 'Callback
  • 45,135
  • 8
  • 71
  • 121
  • Excellent answer, thank you very much. So there is an alternative to backup the data to a physical disk rather than switching back to user-mode? –  Aug 13 '11 at 11:34
  • Thanks for all the help. I actually just did some Read/Write test using standard I/O and ImDisk, and was shocked at the speed increase, so I believe I will pursue getting the API working for that. Much appreciate the information you provided! –  Aug 13 '11 at 12:29