Ignore extra columns in csv - SuperCSV

Question

I am using SuperCSV to parse CSV records into Object. My CSV files have extra column in the end and I want to process only first X columns. So I define String[] mapping for first X columns and CellProcessor[] of same size. But it does not seem to work and throws exception that number of cell processors should be exactly same as number of columns.

Can somebody tell me if I am missing something. Do I need to define mapping array to have exact same columns as in five even if I don't want them?

  public CsvToBeanParser(Reader reader, Class<T> type, CsvPreference preference, CellProcessor[] cellProcessors, String[] mapping, boolean skipHeader)
        throws IOException {
    this.beanReader = new CsvBeanReader(reader, preference);
    this.mapping = mapping;
    if (skipHeader) {
        beanReader.getHeader(true);
    }
    this.cellProcessors = cellProcessors;
    this.type = type;

}

/**
 * Parse and return record.
 * 
 * @return
 * @throws Exception
 *             if there is any parsing error
 */
public T getItem() throws Exception {
    try {
        return (T) beanReader.read(type, mapping, cellProcessors);
    } catch (Exception e) {
        LOG.error("Error parsing record", e);
        throw e;
    }
}

Here are my mapping and cell processors

String[] mapping = {"column1", "column2"};
CellProcessor[] cellProcessors = {null, null};

This works for file

column1, column2
1,2

but fails for (where I want to ignore column3 )

column1, column2, column3
1,2,3

Could you maybe post a bit of code for us to look at? – Houdini Jun 14 '13 at 01:19 — Houdini, Jun 14 '13 at 01:19
Also, are you using the `AbstractCsvReader`? – Houdini Jun 14 '13 at 01:21 — Houdini, Jun 14 '13 at 01:21
I am using CsvBeanReader – RandomQuestion Jun 14 '13 at 01:24 — RandomQuestion, Jun 14 '13 at 01:24

score 5 · Accepted Answer · answered Jun 14 '13 at 11:25

If you're using the header of your CSV file as the nameMapping then you'll never need to know how many columns there are, you'd just use them. If they're not appropriate for mapping (their names don't correspond with the setters in your bean) then you need to supply your own nameMapping with the correct number of elements.

If your CSV file has a header and you know that you only ever want the first 2 columns, then you can simply provide a nameMapping array (and optionally a cell processor array) the same size as the header, but with only the first 2 elements populated.

String[] header = beanReader.getHeader(true);

// only interested in first 2 columns (rest set to null)
String[] nameMapping = 
    Arrays.copyOf(new String[]{"column1","column2"}, header.length);

// processors are optional, but you can populate these if you want
CellProcessor[] processors = new CellProcessor[header.length];

// TODO: read your CSV here...

Thanks, I will definitely try this. I can change my file to have headers. Curious, What would I do if my file didn't have header? — RandomQuestion, Jun 17 '13 at 05:09
You could read the file twice (first time just read the first line to see how many columns there are, and second to read the whole file). — James Bassett, Jun 17 '13 at 07:33

score 0 · Answer 2 · answered Dec 10 '13 at 13:07

If you don't have access to the header or parsing a very large file you can do the following:

Simply extend CsvBeanReader

public class FlexibleCsvBeanReader extends CsvBeanReader {

    public FlexibleCsvBeanReader(final Reader reader, final CsvPreference preferences) {
        super(reader, preferences);
    }

    @Override
    protected List<Object> executeProcessors(final List<Object> processedColumns, final CellProcessor[] processors) {
//      we have to make sure that processors.length is equal to columnSize (thats the real column-count of the csv)
        final int columnSize = getColumns().size();
//      resize the array
        final CellProcessor[] newProcessors = new CellProcessor[columnSize];
        System.arraycopy(processors, 0, newProcessors, 0, processors.length);
//      do default stuff
        return super.executeProcessors(processedColumns, newProcessors);
    }
}

The namemapping can be ignored as long as new columns are only at the end. If you think that is a bad practise you have to override the read method too.

You could also cache the resized array, because depending on your processors it may take some time. But this only makes sense if you apply for each row the same CellProcessor array.

Ignore extra columns in csv - SuperCSV

2 Answers2