2

I have an application which uses MDB, activation specifications and Queue Connection Factories to get/put Messages from WMQ. The application expects a max load of 80 tps. Both Websphere Application Server and WMQ are clustered and each application server connects to seperate host and channel. The application onMessage method is implemented in way so that both session and connection gets closed after message is consumed and response is sent.

As per our configuration, we have WAS version is 8.5, IBM MQ queue manager version 7, max server sessions for act spec set to 40 for each node. max connection count in Connection Factory to 40 for each node and max session in session pool of connection factory to 10. Now on peak load we expect to make max 80 MQ Channel Instance but as per the investigation we can see it goes above 200 which is causing an issue as max instance limit is reached.

Is this happening because max session in session pool of connection factory is set to 10?

Is it possible that even though we are closing the session and connection in onMessage, still one connection can have more than one session. If that is the case, is it wise to set this property to 1? Also can there be some property set at WMQ which could cause this increase in MQ Channel Instances.

Neel
  • 199
  • 3
  • 18
  • WAS version is 8.5. IBM MQ queue manager version 7. MQ Resource Adapter version I am not aware right now but should be what comes default with Was 8.5 – Neel Feb 08 '17 at 00:01

1 Answers1

2

You don't mention specific versions of WAS or MQ, and there could be known problems at a specific version that would change the behavior, but in general it should work as described below.

IBM has a nice Technote "TCP/IP Connection usage between WebSphere Application Server V7 and V8, and WebSphere MQ V7 (and later) explained" which goes into detail on this subject.

You do not mention what you have the SVRCONN channel's SHARECNV set to, as per below this will impact the number of channel instances observed, I'll assume the default of 10 for the calculations.

Note that block quotes below are from the Technote

  • we have set max server sessions for act spec to 40 for each node

The link above states:

Maximum number of conversations = Maximum server sessions + 1

Maximum number of conversations = 40 + 1 = 41

The link also states:

Maximum number of TCP/IP channel instances = Maximum number of conversations / SHARECNV for the channel being used

Maximum number of TCP/IP channel instances = 41 / 10 = 5 (rounded up to nearest connection)

  • max connection count in Connection Factory to 40 for each node
  • max session in session pool of connection factory to 10.

Maximum number of conversations = Connection Pool Maximum Connections + (Connection Pool Maximum Connections * Session Pool Maximum Connections)

Maximum number of conversations = 40 + (40 * 10) = 440

Maximum number of TCP/IP channel instances = Maximum number of conversations / SHARECNV for the channel being used

Maximum number of TCP/IP channel instances = 440 / 10 = 44

If your MQ SVRCONN channel's SHARECNV was set to 10, then you should have no more than 49 channel instances for each channel based on each node connecting to a separate channel.

If you are hitting 200 channel instances I would suspect your SHARECNV is less than 10. If it was 1 the the maximum number of channel instances WAS would try to create would go up to 481 which would be limited by the MAXINST of the the channel to 200.

After an application has finished with a JMS Connection and closed it off, it is moved from the Active Pool to the Free Pool, where it is available for reuse. The Connection Pool property Unused timeout defines how long a JMS Connection will stay in the Free Pool before it is disconnected. This property has the default value of 1800 seconds, which is 30 minutes.

Every JMS Connection that is created from a WebSphere MQ messaging provider Connection Factory has an associated JMS Session Pool, which work in the same way as Connection Pools. The maximum number of JMS Sessions that can be created from a single JMS Connection is determined by the Connection Factory Session Pool property Maximum connections. The default value of this property is 10.

A conversation is started when a JMS Session is first created, and will remain active until the JMS Session is closed because it has remained in the Free Pool for longer than the value of the Session Pool's Unused timeout property.

When your app closes the session and connection in onMessage, the connection is moved to the free pool for reuse and the session is moved to the free pool for reuse, the MQ Channel instance will not be closed until the respective timeout is hit.

If you want to keep your maximum channel count less than 200 then you could tune your Session Pool Maximum Connections) to 1 which combined with your Activation Specifications and a SHARECNV(1) would max out at 121 channel instances.

You can also increase the SHARECNV value of the channel which will result in dividing the channel instances by that number.

It is possible that your connections or sessions are not getting closed properly and you have a "leak".

Community
  • 1
  • 1
JoshMc
  • 10,239
  • 2
  • 19
  • 38
  • Apologies for not mentioning WAS and MQ versions. I have added that in the question now. Thanks for the explanation, this clears my doubts. I will change SHARECNV or session pool max connections and check if the issue gets resolved. One more question, the link you shared also mentions about setting a property SHARECONVALLOWED in connection factory or activation specification. This needs to be present for both act spec and connection factory or only one of them in case I increase the SHARECNV value. – Neel Feb 08 '17 at 09:18
  • Great write-up Josh. Thanks for that. – Nicholas Feb 08 '17 at 17:43
  • @JoshMc I tried these setting in a scaled down version in Test Env which has same version of WAS and MQ with settings max server sessions for act spec set to 10 . max connection count in Connection Factory to 10 and max session in session pool of connection factory to 10. SHARECNV property in channel set to 10 and SHARECONVALLOWED property set to yes in act spec and Queue Conn Factory. But after doing a small load test of firing one request per seconf for 5 mins, the channel instances went upto 54 which wasn't expected as per the calculations. – Neel Feb 09 '17 at 11:24
  • Was there only a single WAS node making connections to that channel? `DIS CHS(CHL_NAME) CONNAME` can help you verify all of those connections are from the same IP. Note also that shared conversations only apply to connections/sessions from the same JVM. – JoshMc Feb 09 '17 at 14:46
  • I would also recommend that you provide the full version information I requested, ex: WAS 8.5.x.x. MQ v 7.x.x.x (you mention only 7 but there is 7.0.x.x, 7.1.x.x, and 7.5.x.x). IBM documents the RA MQ version that is supposed to come with a specific WAS version but I have had issues were the RA did not get updated along with the WAS version, ex: WAS 7.0.0.33 is supposed to include MQ RA 7.0.1.12, but due to some other issue it had stayed with MQ RA 7.0.0.0 version that came with WAS 7.0.0.0. You can check from the queue manager using this command: `DIS CHS(CHL_NAME) RVERSION` – JoshMc Feb 09 '17 at 14:52
  • @JoshMc I have created a new question http://stackoverflow.com/questions/42156365/websphere-mq-high-connection-count-issue. Can you please provide your inputs there – Neel Feb 10 '17 at 09:57