Read packed decimal and convert to numeric in spring boot

Question

All,

I am using a Spring boot application to store data in DB. I am getting this data from IBM MQ through Kafka topic. I am getting messages in EBCDIC format, so used cobol copybook, JRecord, cb2xml jars to convert to readable format and store in DB.

Now i am getting another file also in the same manner, but after conversion the data looks like this:

10020REFUNDONE
10021REFUNDTWO ·"   ÷/
10022REFUNDTHREE   oú^  " 
10023REFUNDFOUR              ¨jÄ ò≈

Here is how i am converting to readable format from ebcdic:

AbstractLineReader reader = null;
StringBuffer finalBuffer = new StringBuffer();
        try {
            String copybook = "/ds_header.cbl";
            reader = CustomCobolProvider.getInstance().getLineReader(copybook, Convert.FMT_MAINFRAME, new BufferedInputStream(new ByteArrayInputStream(salesData)));
AbstractLine line;
        while ((line = reader.read()) != null) {
            if (null != line.getFieldValue(REC_TYPE)){
                finalBuffer.append(line.getFullLine());
            }
        }
            }

and this is my getLineReader method:

public AbstractLineReader getLineReader(String copybook, int numericType, InputStream fileStream) throws Exception {

        String font = "";
        if (numericType == 1) {
            font = "cp037";
        }

        InputStream stream = CustomCobolProvider.class.getResourceAsStream(copybook);

        if(stream == null ) throw new RuntimeException("Can't Load the Copybook Metadata file from Resource....");
        LayoutDetail copyBook = ((ExternalRecord)this.copybookInt.loadCopyBook(stream, copybook, CopybookLoader.SPLIT_REDEFINE, 0, font, CommonBits.getDefaultCobolTextFormat(), Convert.FMT_MAINFRAME, 0, (AbsSSLogger)null).setFileStructure(Constants.IO_FIXED_LENGTH)).asLayoutDetail();
        AbstractLineReader ret = LineIOProvider.getInstance().getLineReader(copyBook, (LineProvider)null);
        ret.open(fileStream, copyBook);
        return ret;
    }

I am stuck here with the numeric conversion, i got to know it is coming in packed decimal. I have nil knowledge on cobol and mainframe, referred few sites and got to know how to convert from ebcdic to readable format. Please help!

I would like to see how `getLineReader` is called and also how you are `writing` the data, I presume numericType=1. I suspect you are doing **line.getFulline()** which is wrong, but I can not see the code. You need to access individual fields, `for (AbstractFieldValue fieldValue :` **line.getFieldIterator(0)**) would be a better way to go. — Bruce Martin, Mar 23 '22 at 21:31
I'm not fluent with sprint-boot, or Java, but I see CP037 mentioned, which seems to indicate that you do code page conversion on the data. Note that you **must not** convert the bytes that contain the packed decimal data. Packed decimal is binary data, not text. — phunsoft, Mar 25 '22 at 10:47
Hello @BruceMartin, thanks for the reply. Yes numericType=1,and i am calling line.getFullline(). I updated my first line of code, could you please check — Sojan, Mar 28 '22 at 06:33

score 0 · Accepted Answer · answered Mar 28 '22 at 07:59

The problem is getFullLine() method does not do any field translation; you need to access individual fields. You can use the line.getFieldIterator(0) to get a field iterator for the line.

Also unless you are using an ancient version of JRecord, you are better off using the JRecordInterface1 class.

Some thing like the following should work:

    StringBuffer finalBuffer = new StringBuffer();

    try {
        ICobolIOBuilder iob = JRecordInterface1.COBOL .newIOBuilder(copybookName)
                               .setFont("cp037") 
                               .setFileOrganization(Constants.IO_FIXED_LENGTH)
                                   ;  


        AbstractLineReader reader = iob.newReader(dataFile);
        while ((line = reader.read()) != null) {
            String sep = "";
            for (AbstractFieldValue fv : line.getFieldIterator(0)) {
                finalBuffer.append(sep).append(fv);
                sep = "\t";
            }
            finalBuffer.append("\n");
        }

        reader.close();
    } catch (Exception e) {
       // what ever ....
    }

Other points

With MQ data source you do not need to create line-readers. You can create lines directly from a byte array:

        ICobolIOBuilder iob = JRecordInterface1.COBOL .newIOBuilder(copybookName)
                               .setFont("cp037") 
                               .setFileOrganization(Constants.IO_FIXED_LENGTH)
                                   ;  

        AbstractLine line = iob.newLine(byteArrayFromMq);
        for (AbstractFieldValue fv : line.getFieldIterator(0)) {
            // what ever
        }

Read packed decimal and convert to numeric in spring boot

1 Answers1

Other points