0

I'm experiencing an issue while trying to use pymqi to send messages to IBM MQ. A project i'm working on

  • Reads from DB2
  • Prepares data for WorkerProcess
  • WorkerProcess does some decision based on that data and sens a message to MQ

When the WorkerProcess tries to commit the message for MQ, I get an error:

Stacktrace:
    worker_process.py:
        self.mq.commit()
    mq_service.py:
        self.qmgr.commit()
    pymqi/__init__.py line 1689, in commit
        raise MQMIError (rv[0], rv[1])
pymqi.MQMIError: MQI Error. Comp: 2, Reason 2012: FAILED: MQRC_ENVIRONMENT_ERROR

Code and stack trace has been typed by hand and can contain typos.

Bellow code is a pseudo code of what I'm doing.

Any help and/or advice is greatly appreciated.

main.py:

c_processors = []
for i in range(num_of_proccessors):
    p = WorkerProcess()
    p.start()
    c_processors.append(p)

for p in c_processors:
    p.join()

worker_process.py

class WorkerProcess(Process):
    def __init__(self):
        Process.__init__(self)
        self.mq = MQService()

    def run(self):
        self.mq.send_message('test')
        self.mq.commit()
        self.mq.close_connection()

mq_service.py

class MQService():
    
    def __init__(self):
        self.connect()
        self.pmd = pymqi.MD()
        self.pmd.Format = pymqi.CMQC.MQFMT_STRING
        self.pmo = pymqi.PMO(Options=pymqi.CMQC.MQPMO_FAIL_IF_QUIESCING)
        self.pmo.Options |= pymqi.CMQC.MQGMO_SYNCPOINT

    def connect(self):
        self._connect_to_qmgr(manager,chanel,host,port,user,password) #these arguments are retrieved from config
        self._q = pymqi.Queue(self.qmgr, queue_name) 

    def _connect_to_qmgr(self,manager,chanel,host,port,user,password):
        self.qmgr = pymqi.QueueManager(None)
        _cd = pymqi.CD()
        _cd.ChannelName = channel.encode('utf-8')
        _cd.ConnectionName = f"{host} ({port})".encode('utf-8')
        _cd.ChannelType = pymqi.CMQC.MQCHT_CLNTCONN
        _cd.TransportType = pymqi.CMQC.MQXPT_TCP
        _connect_options = pymqi.CMQC.MQCNO_HANDLE_SHARE_BLOCK

        _qmgr.connect_with_options(manager, cd=_cd, opts=_connect_options, user=user, password=password)

    def send_message(self, message):
        self._q.put(message, self.pmd, self.pmo)

    def commit(self):
        self.qmgr.commit()
    
    def rollback(self):
        self.qmgr.backout()

    def close_connection(self):
        self.qmgr.disconnect()

EDIT: Additional information:

  • I'm running IBM MQ client version 9.1.0.1.

  • There are no errors in AMQERR0*.LOG files.

  • LD_LIBRARY_PATH is set

  • This error showed while refactoring the code.

Below is the code that is working (before refactoring):

Some arguments in function signature are replaced with (args) for the sake of brevity and readability*

main.py:


def connect_to_mq():
    return MQService(args*)  # these arguments are read from Config file


def process_chunk(args*): 
    _mq = connect_to_mq()
    _mq.send_message('test')
    _mq.commit()
    _mq.close_connection()

c_processors = []
for i in range(num_of_proccessors):
    p = Process(target=process_chunk, args=(args*))
    p.start()
    c_processors.append(p)

for p in c_processors:
    p.join()

mq_service.py

class MQService():
    
    def __init__(self, args*):
        self.pmd = pymqi.MD()
        self.pmd.Format = pymqi.CMQC.MQFMT_STRING
        self.pmo = pymqi.PMO(Options=pymqi.CMQC.MQPMO_FAIL_IF_QUIESCING)
        self.pmo.Options |= pymqi.CMQC.MQGMO_SYNCPOINT
        self.connect_to_qmgr(args*)
        self.connect_to_queue(args*)


    def _connect_to_qmgr(self,manager,chanel,host,port,user,password):
        self.qmgr = pymqi.connect(manager, 
                                  chanel, 
                                  "%s (%s)" % (host, port), 
                                  user=user, 
                                  password=password)
   
    def connect_to_queue(q_name):
        self._q = pymqi.Queue(self.qmgr, q_name)

 
    def send_message(self, message):
        self._q.put(message, self.pmd, self.pmo)

    def commit(self):
        self.qmgr.commit()
    
    def rollback(self):
        self.qmgr.backout()

    def close_connection(self):
        self.qmgr.disconnect()
Tawcharowsky
  • 615
  • 4
  • 18
  • Do you have mq client libraries installed? Check for a file `AMQERR01.LOG` and review for errors. – JoshMc May 29 '21 at 01:52
  • Also ensure that your library path includes either the MQ install lib (32bit) or lib64 (64 bit). – JoshMc May 29 '21 at 06:54
  • Thank you for the help @JoshMc; I forgot to mention that this error happened after code refactoring. I've eddited my question and added previous code that worked. /var/mqm/errors/AMQERR01.log is empty. – Tawcharowsky May 31 '21 at 08:56
  • What is the propose of adding `connect_options = pymqi.CMQC.MQCNO_HANDLE_SHARE_BLOCK`? This is documented as "The application and the local-queue-manager agent are part of the same unit of execution." The error description states "The application is linked to the wrong libraries (threaded or nonthreaded)." Is pymqi compiled to support multi threaded? – JoshMc May 31 '21 at 14:02
  • Hello, If I don't use this option, I get MQRC_ALREADY_CONNECTED. I found this solution in pymqi [documentation](https://dsuch.github.io/pymqi/examples.html#how-to-avoid-mqrc-already-connected) – Tawcharowsky Jun 02 '21 at 08:14
  • I asked a 2nd question, can you reply to that as well. – JoshMc Jun 02 '21 at 10:53
  • Hmm I don't know; I haven't compiled it myself. I was using pip to install it to virtualenvironment; pymqi version is 1.12.0 As for `MQCNO_HANDLE_SHARE_BLOCK` [documentation](https://www.ibm.com/docs/api/v1/content/SSFKSJ_9.0.0/com.ibm.mq.javadoc.doc/WMQJavaClasses/com/ibm/mq/constants/CMQC.html#MQCNO_HANDLE_SHARE_BLOCK) says "This option indicates that connection and object handles allocated by one thread of a process can be used by other threads belonging to the same process." – Tawcharowsky Jun 02 '21 at 14:24
  • Are you using a global transaction here? You mentioned DB2. You cannot use a MQCNO_HANDLE_SHARE_BLOCK hConn in a global transaction – Morag Hughson Jul 20 '21 at 06:05
  • Hello Morag, Thanks for the input. No; I'm not using global transactions. Since then; I figured out what the problem is. Library pymqi supports multithreading but it does not support multiprocessing. In the example that works, all connections to MQ are opened in parent process. In the example that doesn't work, each process is trying to connect to MQ; and this seems to be the problem. – Tawcharowsky Jul 21 '21 at 07:23

0 Answers0