1

I'm reading file 800 Mb via FileStream (.NET 3.5, Console application). The hard drive has SATA 3 bus (600 Mb/s), but I can't figure out: why the time to read the file is only 486 ms.

Things get even worse when I try to read it asynchronously - it takes about 100-150 ms.

The file is not cached - I can see 800 Mb increase in consumption of RAM only when the file is actually loaded into the memory.

Please, help me, this things seem very weird to me

Dragonthoughts
  • 2,180
  • 8
  • 25
  • 28
JohnIdlewood
  • 356
  • 3
  • 12
  • It may not be cached within your application, but the Windows file system operates its own internal file cache in the Windows Cache Manager. see: https://learn.microsoft.com/en-us/windows/desktop/fileio/file-caching Also it can allocate the buffer and pass you a handle to it while it reads the rest asynchronously. – Dragonthoughts Oct 16 '18 at 09:19
  • 1
    "not cached"? Are you sure? Observing RAM usage in task manager is a terrible way to measure what's going on. The operating system probably holds most of the file in memory if you're hitting it up more than once. How does your app hold-up after a cold-reboot? – spender Oct 16 '18 at 09:20
  • Ok, I made some differences to the file, renamed it and it's still about 470-490 ms. – JohnIdlewood Oct 16 '18 at 09:22
  • 2
    You're performing other operations on the file that I'm almost certain the caching system is aware enough to keep track of. Changing the file and renaming it doesn't (necessarily) mean that it's no longer in cache. – Damien_The_Unbeliever Oct 16 '18 at 09:25
  • Ok, right now tried to read file 26 Gb for the first time, that definetely exceeds the RAM and can't be cached - sometimes it's read speed is 90 Mb/s, sometimes it's lower that 60 Mb/s – JohnIdlewood Oct 16 '18 at 09:31
  • The whole file may not be cached, but again pages of it could be. – Dragonthoughts Oct 16 '18 at 14:15

1 Answers1

2

When you read a file it's also copied into the file system cache by the operating system. If the file has not been modified since the last read and it's still cached, the read is served entirely from RAM. That's why after reading a file once you can bypass the disk completely for subsequent reads.

You don't see the cache as memory consumption either. The cache is the entirety of the free memory, something that Task Manager will also point out:

enter image description here

Joey
  • 344,408
  • 85
  • 689
  • 683