1

I modified my code:

private static final String SAMPLE_CSV_FILE_PATH = "src/main/resources/testCSV.csv";
private static final String OUT_PUT_CSV_PATH = "src/main/resources/outCSV.csv";

public static void main(String[] args) throws IOException {

    Reader reader = Files.newBufferedReader(Paths.get(SAMPLE_CSV_FILE_PATH));
    CSVReader csvReader = new CSVReader(reader);



    List<String[]> records = csvReader.readAll();
    Writer writer = Files.newBufferedWriter(Paths.get(OUT_PUT_CSV_PATH));
    CSVWriter out = new CSVWriter(writer);

    int i = 1;
    int total = 0;
    while(i < records.size()){
        String[] result = records.get(i);
        for(int j =1; j<= Integer.parseInt(result[1]); j++){
            String pattern="00000";
            DecimalFormat myFormatter = new DecimalFormat(pattern);
            String output = myFormatter.format(j);
            writer.append(result[0]+output+"\n");
            total++;
        }
        i++;
    }
    out.flush();
    out.close();
    System.out.println(total);
}

First CSV

Now I am using the first CSV file to generate the serial number, Something like:

NAIS00001
NAIS00002
...
NAIS00625

Then I write these serial numbers into a new CSV file. But there is only one column. 6 millions data in one column... How can I star a new column?

LOTR94
  • 27
  • 7

1 Answers1

3

Your Filewriter is not writing in append mode, so your file is being overwritten each time it goes through the outer loop. It's not a problem with the file size.

Try this:

FileWriter fileWriter = new FileWriter("src/main/resources/testOutPut.json", true);

Documentation

NAMS
  • 983
  • 7
  • 17
  • Thanks, that works. But I ran it for 5 second, and my json file become 100mb. Does it suppose to be so big? And I have 6 millions numbers. Do I need to change to another method? – LOTR94 May 01 '18 at 21:42
  • For 6 million numbers, I wouldn't be surprised if the file turned out to be that large since there's a lot of extra overhead due to all the extra brackets and quotes and such, in addition to all of the consecutive numbers you have to generate. If file size is an issue, you could just keep the information in the csv file and convert it as needed, instead of saving the data in 2 different formats. – NAMS May 01 '18 at 21:47
  • Yea, but the problem is the CSV file only show the Model and total Quantity, I need combine Model and Quantity to generate the serial number. Can I create a new CSV file to store the serial number? will it be smaller to store 6 millions number in CSV than store in json? – LOTR94 May 01 '18 at 21:56
  • At the very least, you'd cut out all of the extra formatting that goes along with JSON, so yes, I'd expect you'd save a bit of space that way. But, if all of your serial numbers are 9 characters long like in your example, and you have ~6 million serial numbers, and assuming you're using UTF-8, that's ~54 million bytes of data at minimum, which is roughly half the 100mb you got before. – NAMS May 01 '18 at 22:01