0

Let me try explaining the situation:

There is a messaging system that we are going to incorporate which could either be a Queue or Topic (JMS terms).

1 ) Producer/Publisher : There is a service A. A produces messages and writes to a Queue/Topic

2 ) Consumer/Subscriber : There is a service B. B asynchronously reads messages from Queue/Topic. B then calls a web service and passes the message to it. The webservice takes significant amount of time to process the message. (This action need not be processed real-time.)

The Message Broker is Tibco

My intention is : Not to miss out processing any message from A. Re-process it at a later point in time in case the processing failed for the first time (perhaps as a batch).

Question:

I was thinking of writing the message to a DB before making a webservice call. If the call succeeds, I would mark the message processed. Otherwise failed. Later, in a cron job, I would process all the requests that had initially failed.

Is writing to a DB a typical way of doing this?

TJ-
  • 14,085
  • 12
  • 59
  • 90

4 Answers4

0

Since you have a fail callback, you can just requeue your Message and have your Consumer/Subscriber pick it up and try again. If it failed because of some problem in the web service and you want to wait X time before trying again then you can do either schedule for the web service to be called at a later date for that specific Message (look into ScheduledExecutorService) or do as you described and use a cron job with some database entries.

If you only want it to try again once per message, then keep an internal counter either with the Message or within a Map<Message, Integer> as a counter for each Message.

Sotirios Delimanolis
  • 274,122
  • 60
  • 696
  • 724
0

Crudely put that is the technique, although there could be out-of-the-box solutions available which you can use. Typical ESB solutions support reliable messaging. Have a look at MuleESB or Apache ActiveMQ as well.

anoopelias
  • 9,240
  • 7
  • 26
  • 39
  • I can not 'commit' in case of exceptions and errors --> I was able to find that in most of the JMS implementations. But what if I want to retry it later, like after an hour or so? – TJ- Feb 12 '13 at 19:21
0

It might be interesting to take advantage of the EMS platform your already have (example 1) instead of building a custom solution (example 2).

But it all depends on the implementation language:

Example 1 - EMS is the "keeper" : If I were to solve such problem with TIBCO BusinessWorks, I would use the "JMS transaction" feature of BW. By encompassing the EMS read and the WS call within the same "group", you ask for them to be both applied, or not at all. If the call failed for some reason, the message would be returned to EMS. Two problems with this solution : You might not have BW, and the first failed operation would block all the rest of the batch process (that may be the desired behavior). FYI, I understand it is possible to use such feature in "pure java", but I never tried it : http://www.javaworld.com/javaworld/jw-02-2002/jw-0315-jms.html

Example 2 - A DB is the "keeper" : If you go with your "DB" method, your queue/topic customer continuously drops insert data in a DB, and all records represent a task to be executed. This feels an awful lot like the simple "mapping engine" problem every integration middleware aims to make easier. You could solve this with anything from a custom java code and multiples threads (DB inserter, WS job handlers, etc.) to an EAI middleware (like BW) or even a BPM engine (TIBCO has many solutions for that) Of course, there are also other vendors... EMS is a JMS standard implementation, as you know.

marteljn
  • 6,446
  • 3
  • 30
  • 43
GhislainCote
  • 1,502
  • 11
  • 18
0

I would recommend using the built in EMS (& JMS) features,as "guaranteed delivery" is what it's built for ;) - no db needed at all...

You need to be aware that the first decision will be:

  • do you need to deliver in order? (then only 1 JMS Session and Client Ack mode should be used)
  • how often and in what reoccuring times do you want to retry? (To not make an infinite loop of a message that couldn't be processed by that web service).

This is independent whatever kind of client you use (TIBCO BW or e.g. Java onMessage() in a MDB).

For "in order" delivery: make shure only 1 JMS Session processes the messages and it uses Client acknolwedge mode. After you process the message sucessfully, you need to acknowledge the message with either calling the JMS API "acknowledge()" method or in TIBCO BW by executing the "commit" activity.

In case of an error you don't execute the acknowledge for the method, so the message will be put back in the Queue for redelivery (you can see how many times it was redelivered in the JMS header).

EMS's Explicit Client Acknolwedge mode also enables you to do the same if order is not important and you need a few client threads to process the message.

For controlling how often the message get's processed use:

  • max redelivery properties of the EMS queue (e.g. you could put the message in the dead letter queue afer x redelivery to not hold up other messages)
  • redelivery delay to put a "pause" in between redelivery. This is useful in case the Web Service needs to recover after a crash and not gets stormed by the same message again and again in high intervall through redelivery.

Hope that helps

Cheers Seb

Seb
  • 684
  • 4
  • 11