3

I need to implement the idempotent consumer pattern in Camel (2.23.0) and was looking at possible implementations. Currently, I am only able to implement either the JdbcMessageIdRepository or JpaMessageIdRepository and was wondering if these implementations are thread-safe when handling concurrent requests? I inspected the source code of these classes in the Camel Github, but could not find any code preventing/handling concurrent access.

Can anybody confirm this? Thx very much!

Kim Zeevaarders
  • 732
  • 1
  • 7
  • 21

1 Answers1

4

Short Answer: Yes (on DB table with primary key)

Long answer: I tried it out today (using the Parallel Controller plugin of JMeter) and noticed that if I don't set a primary key on the CAMEL_MESSAGEPROCESSED table, I get duplicate entries when firing multiple simultaneous HTTP-requests. However, if I add a primary key to this table (as also recommended by the camel docs) the idempotent consumer acts as expected.

Is it safe to say that synchronization is taken care of by the primary key mechanism on the table?

I mean, only one Request-thread succeeds in adding a row to the CAMEL_MESSAGEPROCESSED table and from what I've observed during my tests is that the other Request are marked as duplicate by 

Camel (exchangeProperty(Exchange.DUPLICATE_MESSAGE)=true)
Teocci
  • 7,189
  • 1
  • 50
  • 48
Kim Zeevaarders
  • 732
  • 1
  • 7
  • 21
  • Even with eager=true (default) it looks like on code-level Camel cannot reliably prevent duplicate inserts of exchanges processed at the same time (e.g. in high load scenarios). Two exchanges could see that the key is not yet in the repo and make an insert. To reliably filter out duplicates, like you found out, a unique constraint/PK must be set on the messageId field (in addition to have eager=true). I could not find a recommendation to do this in docs for camel-sql though, not for its idempotent consumer functionality. – SebastianBrandt May 21 '19 at 09:04
  • 2
    @SebastianBrandt take a look at the paragraph "Using the JDBC based idempotent repository" under the following URL: https://camel.apache.org/sql-component.html. Somewhere it says "We recommend to have a unique constraint on the columns processorName and messageId." It doesnt explicitly state that this is required for concurrent access scenario's though :) – Kim Zeevaarders May 21 '19 at 09:49
  • 1
    Thanks, I missed that. – SebastianBrandt May 21 '19 at 09:51
  • years later, they now explicitly state you need a primary key for the idempotent consumer: https://camel.apache.org/components/next/sql-component.html#_using_the_jdbc_based_idempotent_repository: `When working with concurrent consumers it is crucial to create a unique constraint on the column combination of processorName and messageId. This constraint will be prevent multiple consumers adding the same key to the repository and allow only one consumer to handle the message.` – Kim Zeevaarders Dec 16 '22 at 20:53