JMS Local Transactions not working when using a CachingConnectionFactory in Spring Integration

Question

I am using CachingConnectionFactory to cache my JMS Broker Connections for IBM MQ. Messages under the local transaction are not rolled back even if I throw an explicit exception. But when I remove the Caching and use just a plain JMS Connection Factory the message is rolled back in case an Exception is thrown. @Gary Russell specified that we can simply throw an exception to roll back any message in transaction. Gary's Answer

edit: This is how is set up my JMS Broker :

@Bean
public IntegrationFlow primaryInitiationListenerFlow() {
      return IntegrationFlows.from(Jms
                .messageDrivenChannelAdapter(
                        context.getBean("connection" + environment.getProperty("primaryConnection"), ConnectionFactory.class),
                        DefaultMessageListenerContainer.class)
                .autoStartup(false)
                .destination(environment.getProperty("sourceName"))
                .configureListenerContainer(listenerContainerSpec -> listenerContainerSpec
                        .destinationResolver((session, destinationName, pubSubDomain) -> destinationName.toUpperCase().endsWith("TOPIC") ? 
                    session.createTopic(destinationName) : session.createQueue(destinationName))
                        .subscriptionDurable(false))
                .id(environment.getProperty("id") + "PrimaryIn")
                .get())
            .channel("idEnrichmentChannel")
            .get();
}

@Bean
public ConnectionFactory connection301() {
    MQConnectionFactory factory = new MQConnectionFactory();
    try {
        factory.setHostName("xxxxxxx");
        factory.setPort(1416);
        factory.setQueueManager("xxxxxxx");
        factory.setChannel("xxxxxxx");
        factory.setTransportType(WMQConstants.WMQ_CM_CLIENT);
    } catch (JMSException e) {
        e.printStackTrace();
    }
    return factory;
}

Here is my configuration with CachingConnectionFactory :

@Bean
public JmsTemplate template301() {
    CachingConnectionFactory cachedConnection = new CachingConnectionFactory();
    cachedConnection.setTargetConnectionFactory(connection301());
    return new JmsTemplate(cachedConnection); 
}

And this is post removal of CachingConnectionFactory :

@Bean
public JmsTemplate template301() {
    return new JmsTemplate(connection301());
}