Java: How can I update the existing csv file and not create a new one?

Question

I am writing to a csv file and then later updating it. But with. my update code, a new same file is generated. Below is my code and I was wondering how I can update without creating a new file.


public void updateFile(List<String> fileNames, String key, String value) throws IOException {
        for (String file: fileNames) {

            InputStreamReader inputStreamReader = new InputStreamReader(new FileInputStream(file), StandardCharsets.UTF_8);
            CSVReader reader = new CSVReader(inputStreamReader);

            String[] header = reader.readNext();
            int columnNum = Arrays.asList(header).indexOf(key);

            List<String[]> data = reader.readAll();
            for (String[] row : data) {
                row[columnNum] = value;
            }
            reader.close();

            OutputStreamWriter outputStreamWriter = new OutputStreamWriter(new FileOutputStream(feedFile), StandardCharsets.UTF_8);
            CSVWriter writer = new CSVWriter(outputStreamWriter);
            writer.writeNext(header);
            writer.writeAll(data);
            writer.flush();
            writer.close();
        }
    }

By new file, I meant, 2 copies of files are created now with different timestamps.

In general, you can't, because the number of bytes per row isn't fixed. — chrylis -cautiouslyoptimistic-, Jul 20 '21 at 21:13

score 2 · Answer 1 · answered Jul 20 '21 at 21:25

Not how it works. Think of a disk: It's a bit like a giant, single-floor hotel with a few billion rooms.

Now imagine that in this hotel you have all the olympians signed in, and they're all in consecutive rooms, sorted on country and then name.

And then a new olympian signs in as a late arrival. You can't just 'update' your hotel and give this olympian a room. Nono, you have to tell about half of the olympians to move one room over to the right, to make room.

Disks are the same. If you want to 'overwrite in place' (say, swap one of the olympians out with another one, in disk terms: Change the value of an existing byte within a file with another value), that's possible, and you'd need to use java's RandomAccessFile and a few other rather more specialized classes to do so, it's not a common task.

But for the vast majority of file updates, you don't get to do it in terms of fixed sizing, and then the notion of 'update in place' is out the window, as far as the disk is concerned.

What you CAN do, however, is this somewhat common task, which ensures that other processes running on that hardware always observe consistent data:

Let's say the hotel is represented by a file on disk that just lists the name of the olympian on a line, and then the next olympian on the next line, and so forth.

To add a new olympian so that the rest of the system always gets a consistent view:

Make a new file named e.g. olympians.txt.new.123981329183 (that number is just randomly generated to avoid conflicts), and start copying the data over from the old file.
During the copy operation, 'fix' the file, as in, insert the new olympian in the right spot.
When you're done close the file, and then use Files.move() with the ATOMIC_MOVE flag, in order to ask java to ask the OS to rename olympians.txt.new.123981329183 to olympians.txt, in an atomic fashion: Any process that opens olympians.txt either opens the old file or the new file, and regardless of which one they got, they always get the complete view; it's not possible to see the new file but only halfway through, or what not.

In that model, as far as the harddisk is concerned, you have 2 utterly different files, but paths (filenames) are just pointers to a spot on the disk, that's all they are. As far as software running on that hardware is concerned, there is only ever one file, it is called olympians.txt, if you open it is it always complete and consistent. When new olympians show up, eventually opening that file will include their name, and it'll be in the right place.

Summary:

If you want to replace values without changing any sizes, this is technically possible but not in an atomic fashion, and requires specialized JDK classes, such as RandomAccessFile.
If you want to remove or add data (change sizes), what you want is, as far as the harddisk is concerned, impossible.
You can, however, make it look like there is just the one file that is always consistent and gets updated from time to time, copy the file over and rotate it back into the right name using java.nio.file.Files's move method, making sure you use the StandardCopyOperation.ATOMIC_MOVE flag.

Java: How can I update the existing csv file and not create a new one?

1 Answers1