4

Here's the problem I have, I need to generate a flat string file with a rather complex (imposed) structure based on field length and start and stop positions. It file will be generated from a .Net application (data stored in SQL Server). It has different headers with different templates. The structure might also change overtime. The same type of file will also have to be parsed back into my system.

What I would preferably like is creating a template that defines the look of the file, for example with the following attribute: Name, Type, Field Length, Start & End Position, Default value.

And be able to generate the file from some kind of view data and then to parse it back from the same template.

I'm pretty sure I'm not the first one to have that kind of trouble, but I cannot find a good library on the Internet. I've looked at StringTemplate but it doesn't seem to be able to create templates based on length and position of data.

Thanks!

Gimly
  • 5,975
  • 3
  • 40
  • 75
  • Have you looked at commercial software like this one: http://www.altova.com/mapforce/flat-file-mapping.html (PS: I'm not affiliated in any way) – Simon Mourier Sep 20 '11 at 08:30

3 Answers3

1

You probably have a good reason to not use standard serialization to XML or JSON. See if T4 Templating Engine or StringTemplate would be of any help.

EDIT: Maybe you should reevaluate your approach and not look for 'templating', as it seem to be for generation only and does not support fixed length. Would it be fair to say that you need serialization to and deserialization from a custom format? If the format is proprietary then you pretty much have to write custom serialization code, that will include all the rules like fixed length etc.

Dmitry
  • 17,078
  • 2
  • 44
  • 70
  • Yes, I do unfortunately have a good reason, exporting data to an old system that can only import and export fixed length flat files. I have already looked at StringTemplate, but it doesn't seem to be able to use fixed-length templates. T4 seems to be the same. – Gimly Aug 13 '11 at 07:55
  • Well, it mis a special format, and it could be done through serialization, but since I have different files with different format (but all fixed length flat files) it would mean I'll have to create a serialization for each format. I guess I don't really have the choice though. – Gimly Aug 16 '11 at 09:35
1

Your problem is very common in the enterprise application world. You can try developing your own library or you can choose a sw that make the entire job. There are many example of that kind of application, generally Enterprise Service Bus (ESB) are usefull for data transport: ESB offers many data handles for all kind of data source, also fixed length file.

Here are same link of open souce sw that you can include in your solution:

Massimo Zerbini
  • 3,125
  • 22
  • 22
1

I don't know about a generic component which deals with this. But I'd write a generic tool which based on the template definition uses reflection to fill properties in an object.

Your 'template' would need to define the structure of the file, as you already described and the full name of the class you want to load the data into (and perhaps the assembly containing the class if that can change).

The basic flow would be:

  • Check header for correctness (optional)
  • Loop though the data lines
    • Create new instance of target class (use Assembly.GetType() andType.GetConstructor()`)
    • Loop through the fields
      • Parse value according to their type
      • Set the value of the property with the same name (using Type.GetProperty() and PropertyInfo.SetValue())
    • Add the object to a result collection.
  • Done

Just make sure your view objects have a default constructor and all the required properties and you should be fine.

Writing the file can be done much the same way using reflection to get the values of your view object.

AVee
  • 3,348
  • 17
  • 17