minOccurs attribute in @Group annotation causes UnexpectedRecordException

Question

I am new to Bean-IO and I was trying to configure a validation logic for occurrence of group when a particular record type is available in the file. For example if there are three records in flat file as shown below.

560866
670972
57086659

I am trying to setup the following logic

Both 56 and 67 lines together form a multi line record
56&67 records can come independently of record 57,but 57 record cannot come without 56&67.

I was successful in creating the first validation using minOccurs attribute in @record annotation, but was not able to do the same for 56&67 using a group.

Please find the sample code setup below.

HeaderRecord class holds the 56&67 record details

@Group
public class HeaderRecord {
    @Record(minOccurs = 1)
    public TX56 tx56;
    @Record(minOccurs = 1)
    public TX67 tx67;
}

RecordObject is used to hold the headers and line items

public class RecordObject {
@Group(collection = List.class, minOccurs = 1)
    List<HeaderRecord> headerRecords;
    @Record(collection = List.class)
    List<TX57> tx57s;
}

@Record(maxLength = 10, name = "TX56")
public class TX56 {
    @Field(ordinal = 0, at = 0, length = 2, rid = true, literal = "56", trim = true)
    protected int id;
    @Field(ordinal = 1, at = 2, length = 4, trim = true)
    protected int number;
}

@Record(maxLength = 31, name = "TX67")
public class TX67 {
    @Field(ordinal = 0, at = 0, length = 2, rid = true, literal = "67", trim = true)
    protected int id;
    @Field(ordinal = 1, at = 2, length = 4, trim = true)
    protected int number;
}

@Record(maxLength = 71, name = "TX57")
public class TX57 {
    @Field(ordinal = 0, at = 0, length = 2, rid = true, literal = "57", trim = true)
    protected int id;
    @Field(ordinal = 1, at = 2, length = 4, trim = true)
    protected int number;
}

with the above configuration when I try to parse the file with records given below, it throws UnexpectedRecordException.
560866
670972
57086659

Stack trace:

2018-07-17 15:22:07,778[http-nio-8080-exec-2]ERROR org.apache.catalina.core.ContainerBase.[Tomcat].[localhost].[/].[dispatcherServlet]-Servlet.service()for servlet[dispatcherServlet]in context with path[]threw exception[Request processing failed;nested exception is org.beanio.UnexpectedRecordException:End of stream reached,expected record'tx56']with root cause org.beanio.UnexpectedRecordException:End of stream reached,expected record'tx56' at org.beanio.internal.parser.UnmarshallingContext.newUnsatisfiedRecordException(UnmarshallingContext.java:367)~[beanio-2.1.0.jar:2.1.0] at org.beanio.internal.parser.Group.unmarshal(Group.java:127)~[beanio-.1.0.jar:2.1.0] at org.beanio.internal.parser.DelegatingParser.unmarshal(DelegatingParser.java:39)~[beanio-2.1.0.jar:2.1.0] at org.beanio.internal.parser.RecordCollection.unmarshal(RecordCollection.java:42)~[beanio-2.1.0.jar:2.1.0] at org.beanio.internal.parser.Group.unmarshal(Group.java:118)~[beanio-2.1.0.jar:2.1.0] at org.beanio.internal.parser.BeanReaderImpl.internalRead(BeanReaderImpl.java:106)~[beanio-2.1.0.jar:2.1.0] at org.beanio.internal.parser.BeanReaderImpl.read(BeanReaderImpl.java:67)~[beanio-2.1.0.jar:2.1.0] at dk.coop.integration.fileconversion.service.sampleapplication.createFixedLengthFile(sampleapplication.java:32)~[classes/:?]

Note: with the above configuration, following scenarios works

56&67comes independently
560866
670972
57cannot come independently
57086659:this flat file fails with a proper exception
56&67should always come as a single record.
this also works fine.

Additional Details: Sample Flatfile

560866
670972
560866
670972
560866
670972
57086659
57086659
57086659
57086659
52022
560866
670972
57086659

As seen above, in the flat file there is a possibility that multiple header records and TX57 record can come as a single entity. Also there can be other type of records that can come in between, in which case I have to treat second occurrence of TX56,67 and 57 as a different item.
In the above example first 10 records will form a single recordObject, then the second occurrence of these records will form a second record object. Sorry for not sharing earlier, but there is another wrapper class which holds a list of recordObject.

I am giving the working maven project Github URL below. https://github.com/Jayadeep2308/FlatFileParser

I have trouble reproducing your problem, I get other errors, which I'm trying to figure out as I go along. I don't know much about using the annotations on BeanIO. Could you perhaps show us how you configure the `StreamFactory` and then how you create the `BeanReader` to read the data from the file. — nicoschl, Jul 17 '18 at 21:18
Sorry for the delayed response and I am updating the question. with all the other relevant details. — Jayadeep23, Jul 18 '18 at 06:59

nicoschl · Answer 1 · 2018-08-05T16:34:35.500

EDIT: Updated after all requirements on 5 Aug 2018

I have made all the fields private inside the classes and assume that you have the getters + setters in place.

I have tried various combinations of settings on the @Group and @Record annotations so the code below might not be optimal, but should work.

First the main group (WrapperObject) that holds all your data:

@Group(minOccurs = 1, maxOccurs = 1)
public class WrapperObject {

  @Group(minOccurs = 0, maxOccurs = 1, collection = List.class)
  private List<RecordObject> recordObjectList;
  @Record(minOccurs = 0, maxOccurs = -1, collection = List.class)
  private List<TX52> tx52s;
}

EDIT: RecordObject updated to hold a list of HeaderRecord, also changed the @Group values.

@Group(minOccurs = 0, maxOccurs = -1)
public class RecordObject {

  @Group(minOccurs = 0, maxOccurs = -1, collection = List.class)
  private List<HeaderRecord> headerRecords;
  @Record(minOccurs = 0, maxOccurs = -1, collection = List.class)
  private List<TX57> tx57s;
}

@Group(minOccurs = 0, maxOccurs = -1)
public class HeaderRecord {

  @Record(minOccurs = 1, maxOccurs = 1)
  private TX56 tx56;
  @Record(minOccurs = 1, maxOccurs = 1)
  private TX67 tx67;
}

On the individual TX records I have added the required=true attribute on the @Field annotation for your record identifier fields. Edit: Added TX52

@Record(maxLength = 74, name = "TX52")
public class TX52 {

  @Field(ordinal = 0, at = 0, length = 2, rid = true, literal = "52", trim = true, required = true)
  private int id;
  @Field(ordinal = 1, at = 2, length = 3, trim = true)
  private int number;
}

@Record(maxLength = 10, name = "TX56")
public class TX56 {

  @Field(ordinal = 0, at = 0, length = 2, rid = true, literal = "56", trim = true, required = true)
  private int id;
  @Field(ordinal = 1, at = 2, length = 4, trim = true, required = true)
  private int number;
}

@Record(maxLength = 31, name = "TX67")
public class TX67 {

  @Field(ordinal = 0, at = 0, length = 2, rid = true, literal = "67", trim = true, required = true)
  private int id;
  @Field(ordinal = 1, at = 2, length = 4, trim = true)
  private int number;
}

@Record(maxLength = 71, name = "TX57")
public class TX57 {

  @Field(ordinal = 0, at = 0, length = 2, rid = true, literal = "57", trim = true, required = true)
  private int id;
  @Field(ordinal = 1, at = 2, length = 4, trim = true)
  private int number;
}

Lastly, my test code: (EDIT: Updated test data)

@Test
public void test() {

  final StreamFactory factory = StreamFactory.newInstance();

  final StreamBuilder builder = new StreamBuilder("Jaydeep23")
    .format("fixedlength")
    .parser(new FixedLengthParserBuilder())
    .addGroup(WrapperObject.class);

  factory.define(builder);

  final String scenario1 = "560866\n670972\n560866\n670972\n560866\n670972";
  final String scenario2 = "560866\n670972\n560866\n670972\n560866\n670972\n57086659\n57086659\n57086659\n" +
  "57086659\n560866\n670972\n57086659\n560866\n670972";
  // invalid
  final String scenario3 = "57086659\n57086659\n57086659\n57086659\n57086659";
  final String scenario4 = "52022\n52066\n52054\n52120";
  final String scenario5 = scenario1;
  final String scenario6 = "560866\n670972\n560866\n670972\n560866\n670972\n57086659\n57086659\n57086659\n" +
  "57086659\n52021\n52022\n52023\n560866\n670972\n57086659\n52023";

  final String message = scenario1;
  BeanReader beanReader = null;
  Object object = null;
  try (final Reader in = new BufferedReader(new StringReader(message))) {
    beanReader = factory.createReader("Jaydeep23", in);
    beanReader.setErrorHandler(new LoggingBeanReaderErrorHandler());
    while ((object = beanReader.read()) != null) {
      System.out.println("Object = " + object);
    }
  } catch (final Exception e) {
    fail(e.getMessage());
  } finally {
    if (beanReader != null) {
      beanReader.close();
    }
  }
}

Generates this output: (EDIT: using your toString() methods)

Scenario 1 = [[Record Type = 56, Store Number = 866
Record Type = 67, Store Number = 972
, Record Type = 56, Store Number = 866
Record Type = 67, Store Number = 972
, Record Type = 56, Store Number = 866
Record Type = 67, Store Number = 972
]null]null

Scenario 2 = [[Record Type = 56, Store Number = 866
Record Type = 67, Store Number = 972
, Record Type = 56, Store Number = 866
Record Type = 67, Store Number = 972
, Record Type = 56, Store Number = 866
Record Type = 67, Store Number = 972
, Record Type = 56, Store Number = 866
Record Type = 67, Store Number = 972
, Record Type = 56, Store Number = 866
Record Type = 67, Store Number = 972
][Record Type = 57, Store Number = 866
, Record Type = 57, Store Number = 866
, Record Type = 57, Store Number = 866
, Record Type = 57, Store Number = 866
, Record Type = 57, Store Number = 866
]]null

Scenario 3 - gives this error (which is correct according as TX57 is not allowed on its own:
Expected record/group 'tx56' at line 6

Scenario 4 = null[Record Type = 52, Store Number = 22
, Record Type = 52, Store Number = 66
, Record Type = 52, Store Number = 54
, Record Type = 52, Store Number = 120
]

Scenario 5 = [[Record Type = 56, Store Number = 866
Record Type = 67, Store Number = 972
, Record Type = 56, Store Number = 866
Record Type = 67, Store Number = 972
, Record Type = 56, Store Number = 866
Record Type = 67, Store Number = 972
]null]null

Scenario 6 = [[Record Type = 56, Store Number = 866
Record Type = 67, Store Number = 972
, Record Type = 56, Store Number = 866
Record Type = 67, Store Number = 972
, Record Type = 56, Store Number = 866
Record Type = 67, Store Number = 972
][Record Type = 57, Store Number = 866
, Record Type = 57, Store Number = 866
, Record Type = 57, Store Number = 866
, Record Type = 57, Store Number = 866
]][Record Type = 52, Store Number = 21
, Record Type = 52, Store Number = 22
, Record Type = 52, Store Number = 23
]

Hope this helps.

Let me know if it is now working for you.

Hi, Thank you very much for the response. The reason I have the headerRecord as a list is because there is a possibility that in a single set there can be multiple records. I have updated the question to reflect a sample working model of my actual requirement. I got it working for a single header, but I was looking at how it can work with a list of Header record. When you run the git Project, please updated the file location in the service class. I didn't have time to load the file In a nicer way than that(Chose not to use resource loader, since it requires restart on file change). — Jayadeep23, Jul 18 '18 at 08:09
I have updated my answer to reflect the changes in your question — nicoschl, Jul 18 '18 at 10:13
Hi, I have two quick questions, could you please share me your RecordObject class. Also I think that the StreamBuilder builder object should add the WrapperClass group rather than the RecordObject, Am I wrong? — Jayadeep23, Jul 18 '18 at 11:20
I have added the `RecordObject` to the answer, sorry, it got lost in the previous update. Yes, you are correct, the `WrapperObject` should be added to the `StreamBuilder`. It is now fixed in my answer as well. — nicoschl, Jul 18 '18 at 11:28
Hi Nicoschl, Thank you for the quick reply. But your code enforces RecordObject to be available at least once in the file. But my requirement is to have the HeaderObject at lease once if TX57 comes in flat file. But there can also be files which can have only TX52 records. That's why I was trying to enforce minOccurs on the headerRecords variable, but not on the RecordObject class itself. — Jayadeep23, Jul 18 '18 at 11:37
Just an addition, Please find the different scenarios for the flat file. 1. Just 56 &67 records - Valid. 2. 56 ,67 & 57 records - valid. 3. just 57 records - invalid(56&67 should be accompanied with 57). 4. just 52 records - valid. 5. list of 56 & 57 - valid. 6. combination of 56,67 & 57 and 52 records - valid Also other requirements is that 56 & 67 should always come together. — Jayadeep23, Jul 18 '18 at 11:37
I'm a little busy with a project at the moment, will look at it when I get a chance — nicoschl, Jul 23 '18 at 13:33

minOccurs attribute in @Group annotation causes UnexpectedRecordException

1 Answers1