How to read multiple line value CSV files in Java?

Question

Since a valid JSON file may contain only one JSON I intend to put multiple JSONs into a CSV file separated by a ";" and to read it using OpenCSV. (It's for putting those JSON data in file apart from the JUnit class to make it better readable.)

Is there a way to read multiple lines until the delimiter ";" occurs? Maybe it would be a good idea to enhance the JSON by an additional field _name which could be used as a key for a Map to read the JSON into!?? I tried it this way first to read into a List but it reads all lines into the list just eliminating the ";". I need a way to read until ";" occurs, store all lines read before into the Map<String, String> or better Map<String, JsonNode> and continue with the next.

    final ClassLoader classLoader = CrmServiceUT.class.getClassLoader();
    final File file = new File(classLoader.getResource("data/UpdatedDeal1.csv").getFile());
    final List<List<String>> records = new ArrayList<List<String>>();
    final CSVParser parser = new CSVParserBuilder().withSeparator(';').build();
    try (final BufferedReader br = Files.newBufferedReader(file.toPath(), StandardCharsets.UTF_8);
         final CSVReader csvReader = new CSVReaderBuilder(br).withCSVParser(parser).build()){
        String[] values = null;
                List<String> list = null;
        while ((values = csvReader.readNext()) != null) {
            if( !Stream.of(values).anyMatch(x -> x.contains(";")) ) {
                list = Arrays.asList(values);
            }
            if(list.contains(";")) {
                list.remove(";");
            }
                records.add(list);
        }
    } catch (CsvValidationException e) {
        e.printStackTrace();
    } catch (CsvException e) {
        e.printStackTrace();
    }

The JSON may be totally different with a few or a lot of lines:

{
  "_nodeName": "updatedDeal1",
  "results": [
    184896670
  ],
  "hasMore": false,
  "offset": 184896670
}
;
{
"_nodeName": "updatedDeal1",
"results": [
184896670
],
"hasMore": false,
"abcde": "xyz",
"offset": 184896670
}

With LHCHIN's answer I ended up in this solution to additionally handle the case that the delimiter (';') may be followed by the next JSON in one line:

@BeforeAll
public static void beforeAll() throws IOException {
    final ClassLoader classLoader = CrmServiceUT.class.getClassLoader();
    final File file = new File(classLoader.getResource("data/jsonsUT.csv").getFile());
    final StringBuilder sb = new StringBuilder();
    for (final String line : Files.readAllLines(file.toPath(), StandardCharsets.UTF_8)) {
        sb.append(line);
        if (line.endsWith(";")) {
            putJsonToMap(sb.substring(0, sb.length()-1));
            sb.delete(0, sb.length());
        }
    }
    putJsonToMap(sb.substring(0, sb.length()));
}

/**
 * Fills a map with JSON used throughout the unit tests.
 *
 * @param jsonStr       The JSON to be stored.
 * @throws IOException
 */
private static void putJsonToMap(final String jsonStr) throws IOException {
    final JsonNode node = mapper.readTree(jsonStr);
    jsons.put(node.get("_nodeName").asText(), node);
    ((ObjectNode) node).remove("_nodeName");
}

This seems very cumbersome. JSON and CSV don't mix very well. Why don't you put your JSON objects in a JSON object with an array instead? You could give it the single element documents with an array of whatever it is you want to read. The individual documents don't even need to be similar. — ewramner, Dec 23 '19 at 16:18

score 1 · Answer 1 · answered Dec 25 '19 at 02:21

For your case, I cannot see any advantage to use opencsv, so why not just handle it by yourself!

Following example shows how to read a given CSV file line by line and put each line into a StringBuilder for concatenation, then once you find a line which ends with ;, add the content of StringBuilder to a list of String.

Code snippet

List<String> records = new ArrayList<>();
StringBuilder sb = new StringBuilder();
for (String line : Files.readAllLines(file.toPath(), StandardCharsets.UTF_8)) {
    sb.append(line);
    if (line.endsWith(";")) {
        records.add(sb.substring(0, sb.length()-1));
        sb.delete(0, sb.length());
    }
}

After that, each element in the List is a JSON string, you can access them directly for further manipulation.

score 0 · Answer 2 · answered Dec 23 '19 at 16:17

0

You can read CSV file just like this and can get each row and column

BufferedReader csvReader;
csvReader = new BufferedReader(new FileReader(file.getPath())); // csv file path
while ((row = csvReader.readLine()) != null) {
String[] data = row.split(",");  // in data you have each column value for each row
}

answered Dec 23 '19 at 16:17

user5943862

39
5

And how does this address JSON documents that are split over multiple lines? You explicitly read one line at a time! – ewramner Dec 23 '19 at 16:19

score -1 · Answer 3 · answered Dec 23 '19 at 16:21

-1

I think, just do it.

json.replace('\n', '').replace(';','\n');

answered Dec 23 '19 at 16:21

seunggabi

1,699
12
12

My problem is that I don't know how to read in from the CSV file. – du-it Dec 23 '19 at 20:45
@du-it give me a sample csv files.. before -1 score – seunggabi Dec 24 '19 at 05:30

How to read multiple line value CSV files in Java?

3 Answers3