9

I just learned that Node.js crown jewel libuv uses blocking system calls for file operations. The asynchronous behavior is implemented with threads! That raises two questions (I only care about Unix):

  1. Why is it not using the non-blocking filesystem calls like it does for networking?
  2. If there are one million outstanding file reads, it probably does not launch one million threads... What does libuv do??
Robert Siemer
  • 32,405
  • 11
  • 84
  • 94
  • 2
    Non-blocking I/O may not be possible always or be of significant use. Read http://www.remlab.net/op/nonblock.shtml – user568109 Nov 17 '13 at 17:28

2 Answers2

5

  1. Most likely to support synchronous operations such as fs.renameSync() vs fs.rename().

  2. It uses a thread pool, as explained in the "Note" at the link you provded.

    [...] but invoke these functions in a thread pool and notify watchers registered with the event loop when application interaction is required.

    So, it creates a limited number of threads and reuses them as they become available.

Also, regarding the quip of "crown jewel:" Node.js and libuv aren't magic. They're good tools to have at your disposal, but certainly have their limitations.

Though, the hyperbole of "one million file reads" would be a stretch for any platform to manage without constraint.

Jonathan Lonowski
  • 121,453
  • 34
  • 200
  • 199
  • 1) I didn’t meant to include rename() operations. I mean file descriptor based operations (i.e. what the kernel can do non-blocking). 2) So libuv doesn’t handle all operations at once. It just takes the first bunch to put it the thread pool and keeps the rest in a FIFO buffer? – Robert Siemer Nov 18 '13 at 01:42
3

  1. The same non-blocking API can not be used, as O_NONBLOCK and friends don’t work on regular files! For Linux AIO is available, but it has it’s own quirks (i.e. depends on the filesystem, turns silently blocking for some operations).

  2. I have no idea.

Robert Siemer
  • 32,405
  • 11
  • 84
  • 94
  • Yeah, O_NONBLOCK just doesn't work on files with linux so uv simulates the non blocking behaviour with a thread pool. – Andrius Bentkus May 26 '14 at 09:09
  • So because O_NONBLOCK not working on linux, windows does not have the benefit to Non-blocking file APIs when using Node? How about .Net Core? – InsaneRabbit Jan 10 '20 at 06:01