2

I want to insert a string in any position (beginning, end, middle) without overlapping/overwriting the existing data.

I've tried using

fs.createWriteStream(file, {start: 0, flags: 'r+'}) but this overwrites the existing data and it does not insert it literally.

I've seen solutions of reading data into buffers then rewrite it again into the file, which won't work for me because I need to handle even large data and buffer has its limits.

Any thoughts on this?

Astral Cloud
  • 99
  • 1
  • 9
  • The **standard** solution for this in all programming language is to save to another file then when complete delete the original file and rename the new file. For example, if you have foo.txt, save to foo.tmp then when completed delete foo.txt and rename foo.tmp to foo.txt – slebetman Sep 25 '20 at 01:07

1 Answers1

1

The usual operating systems (Windows, Unix, Mac) do not have file systems that support inserting data into a file without rewriting everything that comes after. So, you cannot directly do what you're asking in a regular file.

Some of the technical choices you have are:

  1. You rewrite all the data that comes after to new locations in the file essentially moving it up in the file and then you can write your new content at the desired position. Note, it's a bit tricky to do this without overwriting data you need to keep. You essentially have to start at the end of the file, read a block, write it to a new higher location, position back a block, repeat, until you get to your insert location.

  2. You create your own little file system where a logical file consists of multiple files linked together by some sort of index. Then, to insert some data, you split a file into two (which involves rewriting some data) you can then insert at the end of one of the split files. This can very complicated very quickly.

  3. You keep your data in a database where it's easier to insert new data elements and keep some sort of index that establishes their order that you can query by. Databases are in the business of managing how to store data on disk while offering you views of the data that are not directly linked to how it's stored on the disk.

Related answer: What is the optimal way of merge few lines or few words in the large file using NodeJS?

jfriend00
  • 683,504
  • 96
  • 985
  • 979