Is there a way to skip an invalid record when unmarshalling csv records with apache camel bindy

Question

I have the following route

from("file:" + baseInputPath + "/exchangerates" + "?noop=true")
                                .idempotentConsumer(header(CAMEL_FILE_NAME),
                                                FileIdempotentRepository.fileIdempotentRepository(new File(
                                                                baseInputPath + "/tracker/exchangeratefiles.txt")))
                                .log("Processing exchange rate file: ${header.CamelFileName}").unmarshal()
                                .bindy(BindyType.Csv, ExchangeRateBindy.class).process(exchange -> {
                                        ExchangeRateBindy exchangeRateBindy = exchange.getIn()
                                                        .getBody(ExchangeRateBindy.class);
                                        if (exchangeRateBindy != null) {
                                                exchangeRateService.save(exchangeRateBindy.toExchangeRate());
                                        } else {
                                                log.error("Invalid exchange rate: {}", exchange.getIn().getBody());
                                        }
                                });

and the following bindy class

@Data
@CsvRecord(separator = "\\*", skipFirstLine = false)
public class ExchangeRateBindy {
    @DataField(pos = 1)
    private String currencyId;
    @DataField(pos = 2)
    private String currencyName;
    @DataField(pos = 3)
    private double buyRate;
    @DataField(pos = 4)
    private double midRate;
    @DataField(pos = 5, defaultValue = "0")
    private double sellRate;

    public ExchangeRate toExchangeRate() {
        return ExchangeRate.builder().currencyId(currencyId).currencyName(currencyName).buyRate(buyRate)
                .midRate(midRate).sellRate(sellRate).build();
    }
}

and csv data

CAD*Canadian Dollar.*1.0332*1.0332*1.0332
AUD*Australian Dollars.*812.7459194*837.88239115*Test
BWP*BOTSWANA PULA*90.29159051*93.084113925*95.87663734

How do I allow my route to skip the second record which has an invalid last field with the value 'Test' instead of a numeric value? As it is the code just throws a NumberFormatException and does not process any of the records.

score 0 · Answer 1 · answered Aug 17 '23 at 13:13

There can be various options like:

Implement custom data format that wraps Bindy data format and handles parting exception e.g. returning null on invalid record
use filter or validator that validates the record before processing it e.g. using regular expression before it's unmarshalled. This can look like

from("file:...")
   .idempotentConsumer(...)
   .log(...)
   .split(body().tokenize("\n"))
      .filter().regex("[A-Z]{3}\\*[^*]+\\*[0-9.]+\\*[0-9.]+\\*[0-9.]+") // put the valid regex here
      .unmarshal().bindy(BindyType.Csv, ExchangeRateBindy.class)
      .process(...);

Is there a way to skip an invalid record when unmarshalling csv records with apache camel bindy

1 Answers1