0

I'm using fs.readFileSync almost exactly as this answer suggests.

var fileContents;
try {
  fileContents = fs.readFileSync('foo.bar');
} catch (err) {
  // Here you get the error when the file was not found,
  // but you also get any other error
}

However, while I'm playing around getting my code to work and handle missing files, and as I haven't got jest.mock('fs', ...) working yet, I was wondering if there was a way of configuring fs or NodeJs, or the WSL1 I'm using to take less time before throwing ENOENT errors?

As an example, time type foo.bar takes 0m0.000s but the rough equivalent (bear in mind my actual code is a little different to above, i.e. it dynamically determines what my actual foo.bar is) takes 39 seconds.

When I run my code above, it looks like NodeJs is waiting on the file system, perhaps polling for ~5-10 seconds, and then it bails and throws an error with ENOENT? I don't know how it works under the covers.

Can I configure how long it takes/waits for the file system to respond? Or is it just reporting the system error?

AncientSwordRage
  • 7,086
  • 19
  • 90
  • 173
  • "wait for less time"? What do you think is waiting? I'm not aware of anything in `fs.readFileSync()` that "waits". It first calls `fs.open()` and that either works or fails. There's no built-in retry mechanism that I'm aware of that would be waiting for anything. So, if it's not "waiting", I don't think you can make it wait less. – jfriend00 Jan 05 '22 at 18:25
  • @jfriend00 but it's not instant either, if you know what I mean? I've tried to make it clearer in my question, perhaps there's a misconception I have that an answer could dispell. – AncientSwordRage Jan 05 '22 at 18:31
  • readFileSync is synchronous so you cannot do anything else while this runs. So the short answer is No, you cannot. – Laurent Dhont Jan 05 '22 at 18:39
  • synchronous api blocks the event loop and further javascript execution, so in case the hdd is busy you have to wait with everything. Is there a reason you cannot use the async way? – Estradiaz Jan 05 '22 at 18:40
  • Instead of adding a timeout to a synchronous process, why not use the non-blocking, asynchronous version? `import * as fs from 'node:fs/promises'; fileContents = await fs.readFile('foo.bar');` ? Not even sure why the sync versions still exist, probably for compatibility/legacy reasons – Jeremy Thille Jan 05 '22 at 18:45
  • @LaurentDhont I am just personally waiting on the code to run. It sounds like the bottle neck is entirely the file system, and not how NodeJs interacts with it. – AncientSwordRage Jan 05 '22 at 18:52
  • @Pureferret How big is the file? Maybe consider using readStreams? – Laurent Dhont Jan 05 '22 at 18:54
  • It is waiting on the file system. It's waiting for the file operation to be completed. You asked to execute a synchronous file operation which means the file has to be opened, contents read into a buffer, file closed, buffer returned. It takes however long to return as the file system takes to execute those operations. If your hard disk is asleep and has to be spun up or the file contents are large, it may take a little while to execute. There's no polling involved. It uses OS file system calls and they return when they are complete. – jfriend00 Jan 05 '22 at 18:54
  • @Estradiaz if it's just the HDD then that's understandable, if node is doing anything to manage that, i.e. as IO isn't instantaneous, I was hoping to configure that. – AncientSwordRage Jan 05 '22 at 18:55
  • @JeremyThille because in my use case I'd just be putting `await` in front of it. – AncientSwordRage Jan 05 '22 at 18:56
  • @LaurentDhont one is a few bytes and the other is a few MB – AncientSwordRage Jan 05 '22 at 18:57
  • @jfriend00 that's what I thought it did, but at some point it throws an ENOENT how do I make that happen sooner? – AncientSwordRage Jan 05 '22 at 18:59
  • What OS and what kind of disk? So, you're saying it takes 5 seconds to get a ENOENT error? That does not happen on my Windows 11 system at all so that isn't something that is generically built into nodejs. When I run this on a non-existent file on my system, it takes 1ms to generate ENOENT. – jfriend00 Jan 05 '22 at 19:01
  • @jfriend00 WSL1 on windows 10 I believe? The laptop is about 15 years old though. – AncientSwordRage Jan 05 '22 at 19:04
  • 1
    Your code takes less than 1ms to generate ENOENT on my Windows 11 system. So, something is likely wrong with your system. – jfriend00 Jan 05 '22 at 19:06
  • If you read [this doc](https://learn.microsoft.com/en-us/windows/wsl/compare-versions) comparing WSL1 to WSL2 it sounds like there are major performance issues with file system access in WSL1. – jfriend00 Jan 05 '22 at 19:13
  • @jfriend00 I'll have to do some timing, and see what if anything the verbose flag of node reports. Thanks for confirming there wasn't anything obvious I had missed. – AncientSwordRage Jan 05 '22 at 19:13
  • 1
    Here's one relevant quote: ***File intensive operations like git clone, npm install, apt update, apt upgrade, and more are all noticeably faster with WSL 2. The actual speed increase will depend on which app you're running and how it is interacting with the file system. Initial versions of WSL 2 run up to 20x faster compared to WSL 1 when unpacking a zipped tarball, and around 2-5x faster when using git clone, npm install and cmake on various projects.***. Which all points to how slow WSL1 can be. – jfriend00 Jan 05 '22 at 19:13
  • This is a very basic rule in OS design: network accesses are protected by timeout as the network is susceptible to outages, I/O internal to the host is considered reliable and it is not protected by timeout - resulting in a freeze in the very rare case where it does not work - all current systems follow this design – mmomtchev Jan 09 '22 at 22:56
  • @mmomtchev it's odd that my I/O times out then...then again the ENOENT error exists for a reason. Maybe how long Node waits to respond is set somewhere, just not configurable. – AncientSwordRage Jan 09 '22 at 23:01
  • `ENOENT` is not a timeout, it is a file not found error. How much time does it take to read that same file from a command prompt with `type foo.bar`? – mmomtchev Jan 09 '22 at 23:13
  • @mmomtchev on my WSL1 instance I ran `time type foo.bar` and it reported `0m0.000s` I haven't timed *just* the call to `readFileSync` but my code starts the timer works out the file name tries to access it and then in the catch block ends the timer. That take 39 seconds. I'm confident all that time isn't take up by the bits of code around the `readFileSync`. – AncientSwordRage Jan 10 '22 at 08:52

0 Answers0