How to handle xml encoding in XML GENERATE command?

Question

We are facing an issue where in the xml messages with the special characters like [· (0xB7), Ý (0xDD), ¨(0xA8)] are getting rejected, when they are actually XML valid characters. What we found is that are invalid characters for the default encoding that is assigned to a message, when not specified. We can avoid this, if we specifically add the encoding type to the top of the message. We used the UTF-8 encoding. So by adding "" to the very beginning of the XML message, the characters should then become valid characters. Below is the cobol code:

XML GENERATE RESPONSEDATA FROM ACCT  
    COUNT IN RESPONSEDATALL                  
            WITH ENCODING 1208                      
            WITH XML-DECLARATION                    
END-XML

But when we see the xml generated it consists of characters as shown below:

..Ì_%.ÎÁÊËÑ?>.......Á>Ä?ÀÑ>Å..UTF-8....

The expected output is that the below encoding bit should be added to the header.

<?xml version=“1.0” encoding=“utf-8”?>

So the issue is when this generated XML is parsed back again in another peice of code, it encouters an XML-EXCEPTION and backs out altogether with an exception message.

The question is is there some other peice of code or compile options which need to be added to make this xml valid?

"When we see the xml ...": what it looks like depends on what software you use to look at it, and how you configure that software. This looks to me very like UTF-8 being displayed by something that doesn't understand how to display UTF-8. — Michael Kay, Jun 07 '17 at 15:06
@MichaelKay I can almost guarantee he is looking at it on the mainframe using the stock ISPF viewer, because this happened to me — SaggingRufus, Jun 07 '17 at 16:46
@SaggingRufus yes that is corrent. we were viewing it in ISPF viewer. — kushwah_a, Jun 08 '17 at 05:23

score 2 · Answer 1 · answered Jun 07 '17 at 10:45

This is completely normal.

You are encoding the XML as UTF-8, however you are viewing the XML on the mainframe (which is EBCDIC). When your ISPF viewer open the XML file that you have created, it just interprets the hex values and tried to show you something. Because these values do not line up with the EBCDIC character set, it looks like garbage. If you FTP the file down to your computer, you will see that the XML is in fact the output you are looking for.

When you are parsing the XML, you actually have to do some something similar and specify the encoding that was used during the generate. The code below will parse a UTF-8 XML, and on exception, it will display the all of the XML up until the point of the exception occurred in EBCDIC so it is readable in the mainframe SYSOUT

XML PARSE WS-MY-XML
   ENCODING 1208
   PROCESSING PROCEDURE XXXX-PROCESS-XML
   ON EXCEPTION
      DISPLAY "EXCEPTION OCCURED: "
      DISPLAY FUNCTION DISPLAY-OF (
         FUNCTION NATIONAL-OF (
            XML-TEXT 1208
         )
         1140
      )
END-XML

EDIT: that being said, there still could be a legitimate error in your XML such as an illegal character. If the code I have posted also does work, FTP the XML and run it through an XML validator online to see where the fault is.

The code you posted did help I can see valid xml in spool, got it validated online too. But there is still an exception (XML-CODE = 788818, denoting some kind of invalid xml) right at the end when the entire xml is parsed. I am testing it via a SOAP UI client so the xml which gets thrown in SOAP client is also not readable, just like it was in the ISPF editor. My question here is that we generated the XML using 1208 encode, then parsed back the generated XML using 1208 encode. So we need to do any conversion before sending the final XML out into the CICS container, and eventually in SOAP. — kushwah_a, Jun 08 '17 at 05:23
We even tried to remove the XML PARSE bit and just do XML GENERATE with 1208 encoding. But still the generated XML in SOAP client is not readable(same as ISPF view). So not sure if there is some other piece of code that needs to be added to make this readable/work. — kushwah_a, Jun 08 '17 at 05:30
Ok, now I understand, the SOAP client is also working in EBCDIC. Unfortunately, I do not know much about SOAP, but it is the UTF-8 encoding that is messing you up. Someone with more knowledge of SOAP would have to help you. — SaggingRufus, Jun 08 '17 at 11:23

How to handle xml encoding in XML GENERATE command?

1 Answers1