0

I recently inherited a testing framework using PYMQI to put a message on IBM Websphere Queue. I fixed most of the code i need by going through this link

https://dsuch.github.io/pymqi/examples.html#how-to-correlate-request-and-response-messages-using-correlationid

The only pending issue is the format of correlation ID. I tried quite a few options but MQ is converting the correlation ID Into byte array (seems IBM MQ default setting) before throwing it on the Q. All my downstream systems are looking for a HEX value and is breaking.

I am stuck on this for a few days now. Is there a way to force the correlation ID in message properties?.

Please let me know

Regards Aravind

@JoshMc thanks a lot for the reply. It is proprietary code so little difficult to paste whole thing however here is the relevant piece. Downstream system is looking at this JMS Correlation ID to increment and put back in the queue. When i am sending this through below code it hits queue gets converted to byte array however downstream system read validation fails. I want this to hit the queue as a HEX value as in it should be the exact value i pass and not byte array.

queue = self._open_write_queue(queue_name)

put_mqmd = pymqi.md()
put_mqmd.Format = CMQC.MQFMT_STRING
CCSID = 1202

 put_opts = pymqi.pmo()
# Set the MsgType to request.
# put_mqmd["MsgType"] = CMQC.MQMT_REQUEST
put_mqmd["MsgId"] = '00002Y0001T1'
put_mqmd["CorrelId"] = '00002Y0001T1'

 put_opts = pymqi.PMO(Options=CMQC.MQPMO_NO_SYNCPOINT +
                                         CMQC.MQPMO_FAIL_IF_QUIESCING + CMQC.MQRO_PASS_CORREL_ID)
            queue.put(message, put_mqmd, put_opts)

when I put a message through to the queue, it is getting posted successfully, but does not get processed by the downstream system:

In the screenshot below the first message in the queue was posted using app JMSToolBox- which has the correlation as expected by the downstream system. The second message was posted using pymqi - which has the correlation-id displayed ID:'hex format'. Also there is JMSDestination displayed as null.

In the screenshot above the first message in the queue was posted using app JMSToolBox- which has the correlationId as expected by the downstream systems. The second message was posted using robot framework/pymqi - which has the correlation-id displayed ID:30303..... Also please note that the JMSDestination is displayed as null for the second message.(For the first message posted manually has the as 'JMSDestination: queue:///QueueName'

Now i have two issues - CorrelationId not coming up as i expect and the JMSDestination:null. I'm not sure how I can set the destination using pymqi library(or is this set at Q-Manager level) Any help is much appreciated.

Screenshot of message sent by JMSToolBox: enter image description here

Screenshot of message sent using robotframework-pymqi: enter image description here

The only difference I find in the two messages now is the JMSDestination:Null in the JMSHeader for the message sent using Pymqi. I also tried changing the PROPCTL setting for the queue- but did not help.

Is there a I can send the java properties using pymqi?

Aravind R
  • 716
  • 1
  • 10
  • 36
  • A correlationID IS a byte value. Most apps display this as the HEX values of those bytes. Can you put since minimal sample that produces the issue you are facing? – JoshMc May 29 '20 at 03:15
  • Please check the update for relevant code pieces. Thanks again for helping me – Aravind R May 29 '20 at 03:32
  • Can you show it with a realistic value being passed? – JoshMc May 29 '20 at 03:58
  • Value like this : 00001Y0001L1 – Aravind R May 29 '20 at 04:11
  • Can you update the code to show how you are specifying `00001Y0001L1` (note Y and L are not valid if this was HEX) – JoshMc May 29 '20 at 06:58
  • The jms message id is the mq message id bytes represented as hex. If in pymqi you are specifying the actual hex characters (ex 1 and A, not the byte 1A), jms will see the hex value of each character. – JoshMc May 29 '20 at 07:39
  • What is the downstream expecting and were does the expected value come from? HEX is 0-9 A-F. Y and L are not HEX, so it still does not make sense. What you show is correct behavior. – JoshMc Jun 01 '20 at 04:26
  • The downstream system expects JMSCorrlationId to be on the sequence number and block length of the message and has to be generated based looking at the latest sequnece number in the database - as mentioned before it has to be in the format, say '00001Y0001L1' . In the code, i have passed the value like this: put_mqmd["MsgType"] = CMQC.MQMT_REQUEST put_mqmd["MsgId"] = '00002Y0001T1' put_mqmd["CorrelId"] = '00002Y0001T1'.ljust(24).encode('utf-8') – Aravind R Jun 01 '20 at 12:31
  • You passed it correctly. What jms shows is just the HEX representation of that value in your character set, "`0`" = "`30`", "`2`" = "`32`", "`Y`" = "`59`", etc, it pads the 48 bytes to the right with the space character = "`20`". The JMS app would need to convert it back to a value in the sending side character set if it is in a different character set. It would be better to set a message property as text and let mq convert the string for you, JMS can then read this property. – JoshMc Jun 01 '20 at 15:13
  • What about the JMSDestination coming up as null. Do i have to set it up in my code? – Aravind R Jun 01 '20 at 15:19
  • What does the down steam expect for `JMSDestination`. – JoshMc Jun 01 '20 at 15:44
  • When i use JMSToolBox to put a message , the destination is coming up as JMSDestination: queue:///QueueName. – Aravind R Jun 01 '20 at 16:13
  • What is the reason for needing this property? `JMSDestination` is only set by a JMS app as part of a RFH2 header, but you might be able to do it via message properties, but I'm not sure. You could get around it by having the receiving app specify `MQGMO_PROPERTIES_FORCE_MQRFH2`, or by change the `PROPCTL` setting for the destination queue to be either `ALL` or `FORCE`. – JoshMc Jun 01 '20 at 17:35
  • 2
    Please add screenshot with your messages from MQ Explorer. Nobody have your "client" , nobody know how it works and nobody know how it parses IDs. – Seyf Jun 01 '20 at 17:36
  • 1
    https://www.ibm.com/support/knowledgecenter/SSEQTP_9.0.5/com.ibm.websphere.base.doc/ae/rjfp0016_.html MQRFH2 jms.Dst (JMSDestination) This field is present in IBM MQ format JMS messages that include the MQRFH2 header. The jms.Dst field contains a serialized representation (a IBM MQ URI) of the send-to JMS destination that was set when the application issued send for the message. IBM MQ does not use the forward routing path, but the send-to destination might be in a remote service integration bus that can use the forward routing path. – Seyf Jun 02 '20 at 07:09
  • Thanks for the response. JMSDestination error was a defect in our system and I got it fixed today. But I still have this issue of correlation: ERROR 1 : The correlation ID ID:303030303359303030315431202020202020202020202020 has a wrong format. The message cannot be processed. – Aravind R Jun 02 '20 at 16:33
  • As shown in the code above I'm passing (say :'00001Y0001T1') the expected correlationId - but it comes in the queue in the format 'ID:30303030...' - which is not the expected format by the downstream system. Can this be taken care by the code? – Aravind R Jun 02 '20 at 16:33
  • It is really just a matter of how the tool is displaying it, my guess is since it is not a proper 24 byte value it displays the one you manually as characters in the local character set, where in the case of pymqi it is getting padded with spaces to 48 characters so it is showing as the value converted to 48 HEX values. – JoshMc Jun 02 '20 at 16:33
  • Since CorrelId is by spec a 24 byte binary value, you should not repurposed it like this. As I mentioned in an earlier comment, why not set a user property to contain the value you want, you can make it a text value, this can be set from pymqi and JMS can read it. – JoshMc Jun 02 '20 at 16:35
  • I think the pymqi is likely doing the padding for you, you would probably need to modify pymqi to pass it on as is to do what you want, but this is likely not to get taken as a upstream pymqi mod as it is not proper MQ spec. – JoshMc Jun 02 '20 at 16:36
  • i have raised a bug for this and the development team is looking at it. – Aravind R Jun 02 '20 at 22:28

1 Answers1

1

As the comments indicate, the correlation id needs to be a byte array padded out to 24 bytes or 48 hexadecimals. So

Python 2 & 3 compatible way

put_mqmd["CorrelId"] = 'Aravind'.ljust(24).encode('utf-8')

Python 3 only way

put_mqmd["CorrelId"] = bytes('Aravind'.ljust(24), 'utf-8')

but the easiest way is to let the underlying client generate it for you by passing in CMQC.MQPMO_NEW_CORREL_ID. I can't tell you where in your code, because you haven't shown that bit, but there are samples in the pymqi documentation.

chughts
  • 4,210
  • 2
  • 14
  • 27
  • I was trying to populate the correlId after migrating to Python3. Your trick worked for me. Thanks a ton. – Saurabh Jun 21 '20 at 14:02