Reading CSVs with Smooks

Question

Using Smooks (1.4) to read CSVs and transform them into a Person POJO.

The CSV consists of comma-separated records on each line, and each record has pipe-delimited fields:

Smith|John|45|male|Johnny|JSmith|JSmizzle,

Smith|Jane|43|female|Janey

etc. Thus each line represents a different person to create. First, a POJO:

public class Person
{
    private String lastName;
    private String firstName;
    private int age;
    private boolean isMale;
    private List<String> aliases;
}

My problem is with the List of aliases. Here are the vital parts from my XML configuration:

<reader class="org.milyn.csv.CSVReader">
    <param name="fields">lastName,fristName,age,gender,aliases</param>
    <param name="separator">&#124;</param>
    <param name="strict">false</param>
</reader>

<core:filterSettings type="SAX"/>

<jb:bean beanId="person" class="net.me.myproject.app.Person" createOnElement="csv-set/csv-record/">
    <jb:value property="lastName" data="csv-set/csv-record/lastName"/>
    <jb:value property="firstName" data="csv-set/csv-record/firstName"/>
    <jb:value property="isMale" data="csv-set/csv-record/gender"/>
    <jb:value property="age" data="csv-set/csv-record/age"/>
    <jb:wiring property="aliases" beanRefId="aliases"/>
</jb:bean>

<jb:bean beanId="aliases" class="java.util.ArrayList" createOnElement="???">
    <jb:wiring beanRefId="alias"/>
</jb:bean>

<jb:bean beanId="alias" class="java.util.String" createOnElement="???">
    ???
</jb:bean>

So where I'm choking is in getting the createOnElement configured correctly for the aliases ArrayList as well as each alias String. Thanks in advance to anybody who can nudge me in the right direction!

Do you have any control over the incoming file format? We had a similar issue when flattening media asset metadata from XML to pipe-delimited text where fields like "genre" or "actor" sometimes had to contain lists. We ended up delimiting elements with such list with another delimiter. Colon worked fine in our case. — Olaf, Jan 30 '12 at 19:36

score 4 · Accepted Answer · answered Jan 30 '12 at 20:04

First of all, your CSVReader's "fields" attribute will be a hodge-podge of all fields contained inside the CSV file, irregardless of which POJO, list or type they map back to. Thus, some fields will be Person properties, and some fields will be aliases that actually belong to the separate aliases bean, which is of type java.util.ArrayList<String>.

Your job is to tell Smooks how to map each field to the respective bean/list/component/type/etc, which means telling it what to do when it encounters each field.

Smooks does not support this sort of "dynamic" field binding, where you can have 0+ CSV fields map back to an ArrayList, that itself will either be empty or populated. You must enumerate each field in the CSVReader, which means having an ArrayList of aliases that has a fixed size.

Ergo, you must decide on a maximum number of aliases that can be associated with each Person, and account for them in the fields list:

<reader class="org.milyn.csv.CSVReader">
    <param name="fields">lastName,fristName,age,gender,alias1,alias2,alias3</param>
    <param name="separator">&#124;</param>
    <param name="strict">false</param>
</reader>

This means that each CSV record must have a credible value for your 3 aliases. I would recommend having an "ignore" value, such as "%%%IGNORE%%%" so that your app logic can no to remove an list items that contain that value (after Smooks has finished performing the transformation that is).

You might also want to check out Smooks's built-in $ignore$ token, which might already do this or something like it.

The last piece before we can tie everything together in a complete code example is to simply accept the sad fact that Smooks either does not (or does not publicly document) any ability to use List<String> in this kind of example. You must convert your POJO to use either to a List<StringBuffer> or a List<StringBuilder> for aliases, so that we may take advntage of a Smooks-JavaBean value element attribute called setterMethod.

Altogether now:

<jb:bean beanId="aliases" class="java.util.ArrayList" createOnElement="csv-set/csv-record">
    <jb:wiring beanRefId="alias1"/>
    <jb:wiring beanRefId="alias2"/>
    <jb:wiring beanRefId="alias3"/>
</jb:bean>

<jb:bean beanId="alias1" class="java.util.StringBuffer" createOnElement="csv-set/csv-record/alias1">
    <jb:value data="csv-set/csv-record/alias1" setterMethod="append" />
</jb:bean>

<jb:bean beanId="alias2" class="java.util.StringBuffer" createOnElement="csv-set/csv-record/alias2">
    <jb:value data="csv-set/csv-record/alias2" setterMethod="append" />
</jb:bean>

<jb:bean beanId="alias3" class="java.util.StringBuffer" createOnElement="csv-set/csv-record/alias3">
    <jb:value data="csv-set/csv-record/alias3" setterMethod="append" />
</jb:bean>

So every time we start parsing a new csv-record, we create both a person bean (as your initial code example shows perfectly) as well as an aliases bean. Then, over the course of parsing this record, we will find Person properties as well as alias1 through alias3. The aliasN fields get stored into the aliases bean, meanwhile the other Person field gets stored into the person bean. Finally, Smooks knows to "wire" the person and aliases beans together to create a Java Person object.

score 0 · Answer 2 · answered Feb 02 '12 at 12:43

Nice reply Adam.

On the List issue you raised at the end there... I haven't tried it, but I'd have though the following should work...

<jb:bean beanId="aliases" class="java.util.ArrayList" createOnElement="csv-set/csv-record">
    <jb:value data="csv-set/csv-record/alias1" decoder="String"/>
    <jb:value data="csv-set/csv-record/alias2" decoder="String"/>
    <jb:value data="csv-set/csv-record/alias3" decoder="String"/>
</jb:bean>

Reading CSVs with Smooks

2 Answers2