0

I'm interested in the recordTerminator parser property of BeanIO. Does it apply to segments too, like "segmentTerminator"? Namely, I have a stream of fixedlength format, containing of a single record with repeatable segments, and all stream is a single line. Hence, I have set recordTerminator="", but it still gives me 

==> Invalid 'state':  Expected minimum 1 occurrences
==> Invalid 'city':  Expected minimum 1 occurrences
==> Invalid 'street':  Invalid field length, expected 35 characters
==> Invalid 'zip':  Expected minimum 1 occurrences

It doesn't complain about fields that precede to repeatable segment, and complaints about the fields in a repeatable segment are out of order defined in mapping.xml, that looks like this:

    <beanio  xmlns="http://www.beanio.org/2012/03" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.beanio.org/2012/03 http://www.beanio.org/2012/03/mapping.xsd">
      <stream name="employeeFile" format="fixedlength">
        <parser>
            <property name="recordTerminator" value="" />
        </parser>  
        <record name="employee" class="example.Employee">
          <field name="firstName" length="35" />
          <field name="lastName" length="35" />
          <field name="title" length="35" />
          <field name="salary" length="35" />
          <segment name="addressList" collection="list" minOccurs="1" maxOccurs="unbounded" class="example.Address">
            <field name="street" length="35" />
            <field name="city" length="35" />
            <field name="state" length="35" />      
            <field name="zip" length="10" />
          </segment>
        </record> 
      </stream>
    </beanio>

Class implementations is like this:

    package example;        
    public class Employee {
        String firstName;
        String lastName;
        String title;
        String salary;
        List<Address> addressList;

        // getters and setters not shown...
    }       

    package example;
    public class Address {
        private String street;
        private String city;
        private String state;
        private String zip;

        // getters and setters not shown...
    }

If I remove all preceding fields to repetitive segment both from mapping.xml and the input string, remaining string is properly unmarshalled, and marshalled to json afterwards, I even didn't change implementation of java classes, so the preceding fields stay uninitialized, as expected, but properly printed out after marshalling. Where did I go wrong?

OK, my camel code is in spring xml, looks like this:

    <route id="simple-route">
        <!-- from id="request-file" uri="file://C:/mqdocuments/?fileName=response464.txt"/-->
        <from id="request-file" uri="file://C:/mqdocuments/?fileName=request464.txt"/>
        <log id="route-log-request" message="request: ${body}"/>
        <setHeader headerName="CamelJmsDestinationName" id="_setHeader1">
            <constant>queue://QM_TEST/INPUTQ?targetClient=1</constant>
        </setHeader>
        <to id="_to1" pattern="InOut" uri="websphere:queue:SYSTEM.DEFAULT.LOCAL.QUEUE?useMessageIDAsCorrelationID=true&amp;replyTo=REPLYQ"/>
        <log id="route-log-response" message="response: ${body}"/>
                    <transform>
                        <simple>${body}\0</simple>
                </transform>
        <unmarshal ref="parseTransactions464"/>
        <marshal ref="jack"/>
        <log id="route-log-json" message="jackson: ${body}"/>
</route>

So basically, when I uncomment input from file, in which the reponse is saved, and place in comment mq to endpoint, unmarshalling is OK, but if I put a request to a queue, and get response, then I hope to rectify the problem by a transform that simply adds EOF character, because without it, it gives me error that I reported in the first place. And transform doesn't help, because I don't know how to write EOF (ascii 26), but even if I figure that out, I'm not sure it will help.

hdjur_jcv
  • 686
  • 1
  • 12
  • 30
  • Actually, if I return all preceding fields back to input stream and mapping.xml, but I read the stream from the file, instead from a queue, it is OK unmarshalled from fixed length to POJO, and then marshalled to json with jackson. So, the difference must be in EOF treatment for the stream. – hdjur_jcv Feb 16 '18 at 17:43
  • So the question now becomes how to concatenate EOF char to ${body} in Apache Camel route before unmarshalling. – hdjur_jcv Feb 16 '18 at 18:12
  • I don't know Camel at all, but does this perhaps help - https://stackoverflow.com/a/41450736/2020886 – nicoschl Feb 16 '18 at 18:25
  • Also, then if this is now more about Camel, then I suggest you provide the relevant Camel code as well – nicoschl Feb 16 '18 at 18:27
  • I tried with ${in.body}\0 and ${in.body}\n , but it didn't work. – hdjur_jcv Feb 16 '18 at 18:38

2 Answers2

0

I'm going to attempt this as an answer, unfortunately I can't test this, I have nothing setup for use with Camel. First I would not change the default recordTerminator value for BeanIO and leave it to use the default which is any of CR, LF or CRLF.

Then in the transformation of the message, I'll append a newline (\n) instead of the \0. I don't quite understand why you would want to use the EOF character if you have control over it. Instead of:

<transform>
  <simple>${body}\0</simple>
</transform>

I would go for:

<transform>
  <simple>${body}\n</simple>
</transform>

See the section on "Using New Lines or Tabs in XML DSLs" close to the bottom of the page:

Using New Lines or Tabs in XML DSLs

Available from Camel 2.9.3

From Camel 2.9.3: it is easier to specify new lines or tabs in XML DSLs as you can escape the value now xml

<transform> <simple>The following text\nis on a new line</simple></transform>
nicoschl
  • 2,346
  • 1
  • 17
  • 17
  • Unfortunately, this doesn't work, I removed recordTerminator setting from mapping.xml, and tried again with adding \n, but it's still the same. And by the way, the content of file with which route works if taken as input, doesn't have newline at the end, that is a reason why I tried to concatenate EOF at the end, and not \n. – hdjur_jcv Feb 19 '18 at 13:34
  • Ok, no problem. Tha is due to my lack of knowledge about Camel – nicoschl Feb 19 '18 at 14:13
  • nicoschl, thanks for the help. I realized that the camel route actually breaks when there are diacritic characters in a reply message. So, it is not about how the complete string is terminated, when got from queue or read from a file, but I have to set something in BeanIO so that it properly parses and unmarshalls utf-8 diacritics. – hdjur_jcv Feb 19 '18 at 14:37
  • I found in Camel something like this: , perhaps I should give it a try before unmarshalling. – hdjur_jcv Feb 19 '18 at 15:14
  • Did the conversion to UTF-8 make any difference? – nicoschl Feb 19 '18 at 22:17
  • Unfortunately not, I'm still struggling to solve the problem, but I'm sure the problem is that fixed positions are messed up when lengths become variable due to utf-8 charset, when diacritic signs are represented with two characters, and then fixedlength unmarshalling fails. – hdjur_jcv Feb 20 '18 at 08:26
  • Could it be that BeanIO doesn't care much about telling it what charset is, if it only counts bytes? In that case, there is no way to unmarshall fixed length utf-8 strings with it, unless I convert it to a charset that has single byte charset representation before, and back to utf-8 afterwards? – hdjur_jcv Feb 20 '18 at 09:14
  • BeanIO doesn't count bytes AFAIK. To make sure that it uses the correct charset, use a `InputStreamReader` and give it the correct charset in the constructor. `Reader in = new InputStreamReader(new FileInputStream("input.txt"), StandardCharsets.UTF_8)); BeanReader beanReader = factory.createReader("streamName", in);` – nicoschl Feb 20 '18 at 09:27
  • In fact, I think that purpose of convertBodyTo is not to tell some class that is supposed to do unmarshalling that the actual string although declared fixedlength, might be variable length, but to do actual conversion. But that requires that I tell somewhere first that the actual source is utf-8, probably in from endpoint. – hdjur_jcv Feb 20 '18 at 09:27
  • OK, my post was only about 40 seconds later, I will investigate your suggestion, thank you. – hdjur_jcv Feb 20 '18 at 09:31
  • After noticing here http://camel.apache.org/schema/spring/camel-spring.xsd that beanioDataFormat has attribute encoding, that allows to define "The charset to use", I added it but this also doesn't help. – hdjur_jcv Feb 20 '18 at 11:24
  • – hdjur_jcv Feb 20 '18 at 11:24
  • I receive: Exhausted after delivery attempt: 1 caught: java.lang.NullPointerException: charset – hdjur_jcv Feb 20 '18 at 11:33
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/165495/discussion-between-nicoschl-and-hdjur-jcv). – nicoschl Feb 20 '18 at 13:23
  • nicoschl, I will join you as soon as I get home, unfortunately, our corporate security policy prohibits usage of that chat, for some unjustified reason, but all guys which could remove that obstacle, already left the office. Looking forward to talk to you. – hdjur_jcv Feb 20 '18 at 16:13
0

I was floating around trying to identify the problem, but in the end, I realized I should have set the charset with the encoding attribute of beanio dataFormat, which I couldn't do because of this defect:

http://camel.465427.n5.nabble.com/Re-Exhausted-after-delivery-attempt-1-caught-java-lang-NullPointerException-charset-tc5817807.html

http://camel.465427.n5.nabble.com/Exhausted-after-delivery-attempt-1-caught-java-lang-NullPointerException-charset-tc5817815.html

https://issues.apache.org/jira/browse/CAMEL-12284

Finally, I was instructed by Claus Ibsen to use such workaround:

    <bean class="org.apache.camel.dataformat.beanio.BeanIODataFormat" 
          id="some_bean_id"> 
        <property name="encoding" value="UTF-8"/> 
        <property name="mapping" value="mapping.xml"/> 
        <property name="streamName" value="some_stream_name"/> 
    </bean>
hdjur_jcv
  • 686
  • 1
  • 12
  • 30