Sending JSON data from COBOL program to KAFKA rest server

Question

"SQUARE BRACES" is missing from JSON data when I am trying to send from COBOL program to KAFKA

I have started the KAFKA-REST server and tried to format the json data in COBOL program and sent the data using "WEB SEND POST".

NOTE: Media Type used for web send POST is: application/vnd.kafka.json.v1+json

I can see the COBOL hitting my KAFKa server but with the below error at KAFKA server side:

 "POST /topics/test HTTP/1.1" 400 159  (io.confluent.restutils.requests:77)

Data trying to send from COBOL :

{"records":[{"value":{"name":"vikash"}}]}

but the square braces is getting removed at the COBOL end itself and the data goes this format:

{"records": {"value":{"name":"vikash"}} }

Please help!!

Below is the COBOL CODE '''enter image description here

media Type used

Web send POST enter image description here

`EXEC CICS WEB SEND POST                     
  SESSTOKEN(WS-SESSION-TOKEN)            
  FROM(WS-TRY-DATA)                      
  FROMLENGTH(LENGTH OF WS-TRY-DATA)      
  MEDIATYPE(MEDIA-TYPE)                  
  PATH(WS-KAFKA-PATH)                    
  PATHLENGTH(LENGTH OF WS-KAFKA-PATH)    
  RESP(WS-RESP) RESP2(WS-RESP2)          
  NOHANDLE                               
 END-EXEC.

working storage variable

   01 WS-TRY-DATA.                                                
   05 WS-FILLER1                 PIC X(01) VALUE "{".          
   05 WS-FILLER4                 PIC X(01) VALUE '"'.          
   05 WS-RECORD                  PIC X(09) VALUE 'records":'.  
   05 WS-LEFT-BRACE              PIC X(01) VALUE '['.          
   05 WS-FILLER2                 PIC X(01) VALUE "{".          
   05 WS-FILLER6                 PIC X(01) VALUE '"'.          
   05 WS-VALUE-LIT               PIC X(09) VALUE 'value":{"'.  
   05 WS-VALUE                   PIC X(07) VALUE 'name":"'.    
   05 WS-VALUE2                  PIC X(09) VALUE 'vikash"}}'.  
   05 WS-RIGHT-BRACE             PIC X(01) VALUE ']'.          
   05 WS-FILLER3                 PIC X(01) VALUE "}".

media type

    01 MEDIA-TYPE                    PIC X(56)  VALUE 
         'application/vnd.kafka.json.v1+json'.

web send post

Can you share your COBOL code? It's hard to troubleshoot otherwise. — Robin Moffatt, Nov 05 '19 at 09:18
Could be a code page problem what code pages are involved. Can you get a hex dump of the strings? — James Anderson, Nov 06 '19 at 07:22
@RobinMoffatt Thanks for the reply, I have added the cobol code as well. Please have a look. — vikash, Nov 08 '19 at 13:07
@JamesAnderson Hi James, thanks for responding, what code page should be provided , could you please help me. I have added the code for your reference as well. — vikash, Nov 08 '19 at 13:08

score 1 · Answer 1 · answered Nov 19 '19 at 16:01

Square brackets have been known to cause problems between PC and mainframe. You want to make sure that on the mainframe, these are EBCDIC characters x'BA' and x'BB' as the left bracket ('[') and right bracket (']'). Some terminal emulators don't display these correctly or don't assign the correct EBCDIC characters to them when you input with the keyboard.

For example, I replicated your COBOL structure above into a program. When I typed the [ character from my PC into the WS-LEFT-BRACE field, I didn't get the x'BA' character. Instead, it appeared as x'5F'. Likewise, the WS-RIGHT-BRACE showed up as x'A5' rather than x'BB'.

So, the first thing to do is to check the actual hex values in your COBOL structure.

After I got the right values into the field, I then used WEB SEND with the same media type as you. My HTTP client received the correct JSON output including the square brackets. I did explicitly code CHARACTERSET('iso-8859-1'), but you shouldn't have to code that operand since it's the default.

If you do have the correct hex values in your COBOL structure, I'd suggest opening a problem ticket with IBM. CICS should not be stripping the square brackets from the HTTP body.

Leigh's answer applies to the default COBOL codepage EBCDIC 1140. If using EBCDIC 1047, which is another commonly used EBCDIC codepage and the default codepage in USS, the left and right bracket are EBCDIC characters x'AD' and x'BD'. — Nicole Trudeau, Nov 21 '19 at 16:48
Exactly -- I was referring to code page 037 which is the default EBCDIC code page used in the USA for MVS files. 1047 is the default code page for the Unix file system. I made an assumption from the COBOL code that the original poster was using traditional MVS partitioned data sets for his source code. Thanks for pointing out the "it depends" part of this answer. — Leigh Compton, Nov 22 '19 at 17:43
One more thing, generally, JSON is generated as UTF-8, since JSON as a format itself said that should be the encoding. Users are free to convert it to EBCDIC for ease of use/view, which, if the EBCDIC version of the JSON is then transmitted on EBCDIC platform, the above mentioned conversion issue may pop up. However, maybe it's desirable to keep the JSON in UTF-8 and transmit it around. — Nicole Trudeau, Nov 29 '19 at 01:53

Sending JSON data from COBOL program to KAFKA rest server

1 Answers1