2

We use apache ode 1.3.3 inside Servicemix 3.3.2 for implementing our workflows. When the load increases (ie number of flow invocations per unit time ), ode freezes, stops processing new requests and hangs. Usullay, we solve the issue by doing a Servicemix restart after a cleanup of the 'data' folder.

Initially, i thought that this was a thread problem ( lack of threads for ode to use ). After increasing the 'thread pool size' in ode-jbi.properties, this problem got addressed to an extend. But still, under heavy load, ode continues to hang.

Further investigation, with additional logging revealed that under heavy load, ode is unable to get enough connections to the DB (NoManagedConnectionException ) to persist its state. Our ode persistence configuration is INTERNAL (refer to ode-jbi.db.mode=INTERNAL ) where the connection pool is maintained by the container/runtime itself. Efforts to migrate it to a EXTERNAL jndi datasource configuration ( where we can configure the connection pool parameters ) has not yet borne fruit ( Thanks to my inability to comprehend the zillion errors that keep cropping up at each stage :( :) )

So i tried the other available option, 'In-Memory execution' of my processes. This works fine in a 'test setup' and it no longer throws the 'NoManageConnectionException' under simulated load.

But i have a few concerns about moving this to PROD. How different is In-Memory execution different from 'persisted' execution ? How much more memory will this consume ? Will this affect the reliability of my PROD setup by causing 'OutOfMemoryOutages'.

We have about 10 deployed bpel processes. And the load (no. of requests )...well it is usually minimal, but sometimes spikes up ( that's where i need my processes to run really fast but without causing any OutOfMemory PROD outages...hopefully :D :) :P )

Need your thoughts, suggestions and advice on this.

Thanks in advance, Arun

Arun Jolly
  • 342
  • 1
  • 3
  • 12
  • Having the same issues you're mentioning. Thanks for the In-Memory tip, will use that for development. Will update after moving to test/production. – Dormouse Jan 16 '14 at 09:36

1 Answers1

1

A bit late with the answer, though still relevant and hopefully helps someone in the future.

Regarding the external DB config, this link describes how to use Bitronix TM in conjunction with Tomcat. Yes it's specific to Tomcat, but in other application server you'll have other means to specify a managed data source. Just make sure the reference in Apache ODE is set accordingly in the web.xml and ode-axis2.properties files. This should enable you to use an external database.

I have found that in-memory execution of a process under high load may result in a ConcurrentModificationException. This JIRA refers to the problem and has a fixed version set to 1.4.

So I guess the answer, whether in-memory execution is stable or not, is NO, since currently a bug is present that manifests under high load.

Also external DB coupling is possible and I will test tomorrow if the CME is also thrown using an external DB.

EDIT:

Have not seen the ConcurrentModificationException when connected to an external DB (container managed), though did stumble upon the symptoms of this JIRA, which is patchable, although no feedback is provided in the issue and I cannot currently build the source.

EDIT:

In-memory executed BPEL processes are restricted to being synchronous (having one receive action), as explained here, which is actually an answer by an ODE dev on this very same question... :)

In-memory exection of BPEL processes is therefore stable, as long as they are synchronous.

Dormouse
  • 1,617
  • 1
  • 23
  • 33
  • @Dormhouse : So how is the in-memory model holding up ? Did you move this to PROD ? What is analysis after one month ? – Arun Jolly Feb 07 '14 at 08:16
  • We cannot use in-memory execution for our process since it's asynchronous. So we're using an external database. It will be some time before we'll be testing on a PROD-like environment. – Dormouse Feb 07 '14 at 19:54