4

I am creating an append/rw file (depending if the download is resumed or not)

this.syncFile = await fs.promises.open(this.syncPath, 'a+');
this.syncFile = await fs.promises.open(this.syncPath, 'r+');

later I create a write stream for that FD 

let writeStream = fs.createWriteStream(null, {fd:this.syncFile.fd, autoClose:false, highWaterMark: 1024*1024});

after downloading the remaining data I may or may not create a readStream for the same FD to calculate checksum of full file

let readStream = fs.createReadStream(null, {fd:this.syncFile.fd, autoClose:false, highWaterMark: 1024*1024, start:0});

for finalizing I would like to close the FD.

fs.closeSync(this.syncFile.fd);

I have tried numerous combinations with close() or dispose() on the readStream/writeStream I keep getting uncaught exceptions

[Window Title]
Error
[Content]
Uncaught Exception:
Error: EBADF: Closing file descriptor 5 on garbage collection failed, close

or console warnings about closing file descriptor on gc always two in a row sometimes followed by

Error: EBADF: bad file descriptor, write

Why is this happening? The read/write streams have option autoClose:true Also, before, when I only had a single writeStream some combination that I cannot re-create worked.

What is the correct way of closing all the streams and the FD in this scenario? I cannot wait for the GC to close the file descriptor because it holds an exclusive lock on the file, a file whose attributes I am changing (child_process.execSync('attrib -h +r "' + this.finalPath + '"');) Also the file is unusable by other programs until GC is triggered.

If I only do

delete this.syncFile; I get a beatiful spam of 

(node:25180) Warning: Closing file descriptor 10 on garbage collection
(node:25180) Warning: Closing file descriptor 10 on garbage collection
(node:25180) Warning: Closing file descriptor 8 on garbage collection
(node:25180) Warning: Closing file descriptor 8 on garbage collection
(node:25180) Warning: Closing file descriptor 7 on garbage collection
(node:25180) Warning: Closing file descriptor 7 on garbage collection
(node:25180) Warning: Closing file descriptor 9 on garbage collection
(node:25180) Warning: Closing file descriptor 9 on garbage collection

But no exceptions, handled or otherwise, but this is still not good enough because of the issues listed above. My only option seems to be to force a GC collection?

n00b
  • 5,642
  • 2
  • 30
  • 48
  • OK, you're not using `autoClose` so you have to manually close. You will need to hook the correct events in order to be able to close it yourself. For a readStream, that would usually be the `end` event as that signals the end of serially reading content. For a writeStream, it depends upon how you're using it as only you know when you're doing writing to it. We need to see your entire flow of the real code in order to know how to help you here. This is specific to your use of the streams and your specific code. There is no generic answer here. – jfriend00 Dec 18 '19 at 18:39
  • 1
    Also, if you're opening the file with `fs.promises.open()`, then you're getting the newer FileHandle object (not a regular file descriptor) and you will need to call close on that object as in [`fileHandle.close()`](https://nodejs.org/api/fs.html#fs_filehandle_close), not `fs.close(fd)`. – jfriend00 Dec 18 '19 at 18:43
  • @jfriend00 If I do not close the `autoClose:false` streams, they will attempt to close the FD during garbage collection anyway, and fail with a nice error messagebox since its an uncaught exception in the main loop. If I call `destroy()` or `close()` on any of them all other attempts to close the FD will cause exceptions. If I `delete readStream;` them and the main `FileHandle` it will work, but the file will not be released until the next GC run, blocking all other programs from using it. – n00b Dec 18 '19 at 18:44
  • @jfriend00 The second comment is correct, `this.syncFile.close()` instead of `fs.closeSync(this.syncFile.fd)` used after `delete writeStream/readStream` seems to work properly, pls post as answer, thanks ! :) – n00b Dec 18 '19 at 18:49
  • (However calling `close()` or `destroy()` on the streams or not deleting them before calling it will still lead to random errors.) – n00b Dec 18 '19 at 18:51
  • Glad, you got it figure out. FYI, for the future, this would have been easier to see what was going on if you included more code context including the specific code you were using to try to close the file. Apparently, the GC warnings were because even though you were trying to close the file, you were doing it the wrong way and probably you were ignoring errors on the close too. We can always help in a more complete way if we can see more of your real code. – jfriend00 Dec 18 '19 at 19:16

1 Answers1

8

If you're opening the file with fs.promises.open(), then you're getting the newer FileHandle object (not a regular file descriptor) and you will need to call close on that object as in fileHandle.close(), not fs.close(fd).

jfriend00
  • 683,504
  • 96
  • 985
  • 979