Websphere Liberty 8.5 "com_ibm_ws_batch_partitionNum" is not working

Question

I have configured embeddedJMS in Websphere Liberty 8.5. It is Dispatcher - Executor Setup.

I am trying to configure "com_ibm_ws_batch_partitionNum" attribute in messageSelector but when i configure this attribute the job is not executing.. below is the code I have configured in executor

<jmsActivationSpec id="batchActivationSpec" maxEndpoints="5">
<properties.wasJms destinationRef="batchJobSubmissionQueue"   destinationType="javax.jms.Queue" maxBatchSize="5" maxConcurrency="5" messageSelector="com_ibm_ws_batch_applicationName = 'LibertyBatchPOC' AND com_ibm_ws_batch_partitionNum = 1" remoteServerAddress="localhost:7280:BootstrapBasicMessaging">
    </properties.wasJms>
</jmsActivationSpec>

Complete Executor1 Server.xml as below and Executor2 is also same with different port and com_ibm_ws_batch_partitionNum configured as "2"

  <server description="new server">

    <!-- Enable features -->
    <featureManager>
        <feature>wasJmsClient-2.0</feature>
        <feature>batchManagement-1.0</feature>
        <feature>batch-1.0</feature>
        <feature>javaMail-1.5</feature>
        <feature>batchManagement-1.0</feature>
        <feature>jdbc-4.1</feature>
        <feature>jndi-1.0</feature>
        <feature>localConnector-1.0</feature>
        <feature>jaxb-2.2</feature>
    </featureManager>

    <!-- To access this server from a remote client add a host attribute to 
        the following element, e.g. host="*" -->
    <httpEndpoint httpPort="9082" httpsPort="9445" id="defaultHttpEndpoint"/>

    <!-- Automatically expand WAR files and EAR files -->
    <applicationManager autoExpand="true"/>
<!---DB2 database setup-->
<library id="DB2Lib">
        <fileset dir="${shared.resource.dir}" includes="*.jar"/>
    </library>
    <dataSource id="testDB" jndiName="jdbc/test" transactional="false">
        <jdbcDriver libraryRef="DB2Lib"/>
        <properties.db2.jcc currentSchema="TestSchema" databaseName="DB2C" password="password" portNumber="5002" serverName="aa.test.com" user="TestDB"/>
    </dataSource>
    <!-- Database setup for DERBY -->
    <!-- Database setup for DERBY Network DB -->
    <!-- Batch persistence config. References a databaseStore. -->
    <batchPersistence jobStoreRef="BatchDatabaseStore"/>

    <!-- The database store for the batch tables. -->
    <!-- Note this database store is referenced by the batchPersistence element. -->
    <databaseStore createTables="true" dataSourceRef="batchDB" id="BatchDatabaseStore" schema="JBATCH" tablePrefix=""/>

    <!-- Derby JDBC driver -->
    <!-- Note this library is referenced by the dataSource element -->

    <library id="DerbyLib">
        <fileset dir="${shared.resource.dir}/derby/lib" includes="derbyclient.jar"/>
    </library>
<!-- Data source for the batch tables. -->
<!-- Note this data source is referenced by databaseStore element -->
<dataSource id="batchDB" jndiName="jdbc/batch" type="javax.sql.XADataSource">
<jdbcDriver libraryRef="DerbyLib"/>
<properties.derby.client createDatabase="create" databaseName="${shared.resource.dir}/databases/BATCHDB" password="pass" portNumber="1527" serverName="localhost" user="user"/>

    </dataSource>

    <jdbcDriver id="derbyDriver" javax.sql.ConnectionPoolDataSource="org.apache.derby.jdbc.ClientConnectionPoolDataSource" javax.sql.XADataSource="org.apache.derby.jdbc.ClientXADataSource" libraryRef="DerbyLib"/>

    <!-- Authorization and Security -->
    <keyStore id="defaultKeyStore" password="Liberty"/>

    <basicRegistry id="basic" realm="ibm/api">
        <user name="bob" password="bobpwd"/>
        <user name="jane" password="janepwd"/>
    </basicRegistry>

    <authorization-roles id="com.ibm.ws.batch">
        <security-role name="batchAdmin">
            <user name="bob"/>
        </security-role>
    </authorization-roles>

    <batchJmsExecutor activationSpecRef="batchActivationSpec" queueRef="batchJobSubmissionQueue"/>

    <!-- note that message selector need space around the equal sign -->
    <jmsActivationSpec id="batchActivationSpec" maxEndpoints="5">
        <properties.wasJms destinationRef="batchJobSubmissionQueue" destinationType="javax.jms.Queue" maxBatchSize="5" maxConcurrency="5" messageSelector="com_ibm_ws_batch_applicationName = 'LibertyBatchPOC' AND com_ibm_ws_batch_partitionNum=1" remoteServerAddress="localhost:7280:BootstrapBasicMessaging">
        </properties.wasJms>
    </jmsActivationSpec>

    <jmsQueue id="batchJobSubmissionQueue" jndiName="jms/batch/jobSubmissionQueue">
        <properties.wasJms deliveryMode="Persistent" queueName="batchLibertyQueue" readAhead="AsConnection" timeToLive="0"/>
    </jmsQueue>
<logging maxFileSize="200" maxFiles="10" traceFileName="trace.log" traceSpecification="com.ibm.jbatch.*=all:com.ibm.ws.jbatch.*=all"/>
   <applicationMonitor updateTrigger="mbean"/>
</server>

Dispatcher Server.xml

    <?xml version="1.0" encoding="UTF-8"?>
<server description="BatchDispatcher">

    <!-- BATCH DISPATCHER -->

    <!-- Enabled features -->
    <featureManager>
        <feature>wasJmsClient-2.0</feature>
        <feature>batchManagement-1.0</feature>
    </featureManager>

    <!-- JMS Configuration -->
    <batchJmsDispatcher connectionFactoryRef="batchConnectionFactory"
        queueRef="batchJobSubmissionQueue" />
<batchJmsEvents connectionFactoryRef="batchConnectionFactory" />
    <jmsConnectionFactory id="batchConnectionFactory"
        jndiName="jms/batch/connectionFactory">
        <properties.wasJms remoteServerAddress="localhost:7280:BootstrapBasicMessaging" />
    </jmsConnectionFactory>

    <jmsQueue id="batchJobSubmissionQueue" jndiName="jms/batch/jobSubmissionQueue">
        <properties.wasJms deliveryMode="Persistent"
            queueName="batchLibertyQueue" readAhead="AsConnection" timeToLive="0"></properties.wasJms>
    </jmsQueue>
    <!-- Server HTTP port setup -->
    <httpEndpoint id="defaultHttpEndpoint" httpPort="9081"
        httpsPort="9444" />

    <!-- Database setup for DERBY -->
    <!-- Database setup for DERBY Network DB -->
    <!-- Batch persistence config. References a databaseStore. -->
    <batchPersistence jobStoreRef="BatchDatabaseStore" />

    <!-- The database store for the batch tables. -->
    <!-- Note this database store is referenced by the batchPersistence element. -->
    <databaseStore id="BatchDatabaseStore" createTables="true"
        dataSourceRef="batchDB" schema="JBATCH" tablePrefix="" />

    <!-- Derby JDBC driver -->
    <!-- Note this library is referenced by the dataSource element -->

    <library id="DerbyLib">
        <fileset dir="${shared.resource.dir}/derby/lib" includes="derbyclient.jar" />
    </library>

    <!-- Data source for the batch tables. -->
    <!-- Note this data source is referenced by databaseStore element -->
    <dataSource id="batchDB" jndiName="jdbc/batch" type="javax.sql.XADataSource">
        <jdbcDriver libraryRef="DerbyLib" />
        <properties.derby.client createDatabase="create"
            databaseName="${shared.resource.dir}/databases/BATCHDB" serverName="localhost"
            portNumber="1527" user="user" password="pass" />

    </dataSource>

    <jdbcDriver id="derbyDriver"
        javax.sql.ConnectionPoolDataSource="org.apache.derby.jdbc.ClientConnectionPoolDataSource"
        javax.sql.XADataSource="org.apache.derby.jdbc.ClientXADataSource"
        libraryRef="DerbyLib" />

    <!-- Authorization and Security -->
    <keyStore id="defaultKeyStore" password="Liberty"/>

    <basicRegistry id="basic" realm="ibm/api">
        <user name="bob" password="bobpwd" />
        <!--<user name="jane" password="janepwd" />-->
    </basicRegistry>

    <authorization-roles id="com.ibm.ws.batch">
        <security-role name="batchAdmin">
            <user name="bob" />
        </security-role>
    </authorization-roles>

    <!-- Enable Batch tracing for troubleshooting purposes -->
    <logging maxFileSize="200" maxFiles="10"
        traceSpecification="com.ibm.jbatch.*=all:com.ibm.ws.jbatch.*=all"
        traceFileName="trace.log" />

</server>

Somewhere in your domain you need to have an executor with a selector configured to consume the job dispatch messages. If you only have one executor server configure with the above config snippet, then yes, the job will never execute, it will just sit on the dispatch queue. If your domain does in fact include more executor servers then please share more details of the various executor servers in your domain. — Scott Kurz, Jun 07 '16 at 14:25
I have configured 2 executors and in second executor specified com_ibm_ws_batch_partitionNum=2. — Jay, Jun 07 '16 at 14:40

score 1 · Answer 1 · answered Jun 07 '16 at 15:01

1

If those are your only two executors then the job itself won't run because there is no selector to just get the job. You have an executor to run partition 1 and an executor to run partition 2, but no executor that just runs actual jobs. You could adjust executor 1 to allow the partition num value of 1 or NULL (not set for the job itself) and then the job AND partition 1 would run on executor 1.
Note that you need to be careful with that.. if you tie up the server running actual jobs there can be no room left to run the partitions and it will hang.

answered Jun 07 '16 at 15:01

DFollis

419
2
5

Can you please share some code snippet, how should I add selector to get the job .. – Jay Jun 07 '16 at 15:13
Untested, but something like this: messageSelector="com_ibm_ws_batch_applicationName = 'LibertyBatchPOC' AND (com_ibm_ws_batch_partitionNum=1 OR com_ibm_ws_batch_partitionNum is NULL)" – DFollis Jun 07 '16 at 15:17
In the above code snippet it executes the complete job. and not only the partition. – Jay Jun 07 '16 at 15:24

score 0 · Answer 2 · answered Jun 07 '16 at 15:50

You will need two things.

1) You will need to have an executor which will run the job first and which will queue the partitions to be consumed by other executors

For this, you'll need to have no additional property in a message selector. Alternatively, you can use a property com_ibm_ws_batch_work_type with value Job or Partition is used to select and execute work based on the type.

Please note, to distribute partitions from the job over multiple executors, you will need to configure the <batchJmsDispatcher> element in the executor running the top level Job.

2) Second thing needed would be an executor to run the partition, where you can use your message selector as described in the question(which wi

I am still not able to configure it properly and it is not running as expected.. But will update soon in case of any update.. — Jay, Jun 07 '16 at 18:37

Websphere Liberty 8.5 "com_ibm_ws_batch_partitionNum" is not working

2 Answers2