1

is it possible to store a large amount of messages in bulk? I want to send them sync, persistent, but to get speed very much at one time.

I am using NMS, the .net version of the java-framework. But if you only know how to do this in java, it would even help. Maybe I can find a solution for .net more easier.

I thought of things like transactions. But I only got transactions to work for consumers, not for producers.

Chris
  • 4,325
  • 11
  • 51
  • 70

1 Answers1

2

Conventional wisdom used to suggest that if you wanted maximum throughput when sending in bulk, then you should a SESSION_TRANSACTED acknowledgement mode and batch all of the message sends together with a .commit().

Unfortunately, here's a benchmark showing this not to be the case http://www.jakubkorab.net/2011/09/batching-jms-messages-for-performance-not-so-fast.html and that are you better off just sending them as normal without transactions. If you are already using transactions, then it may make sense to try and batch them.

My advice here also is that unless you are dealing with messages that are extremely time sensitive, the rate at which you produce isn't going to be that big of a deal - you should be more concerned with bandwidth as opposed to speed of message sends. If you don't mind your messages being out of order you can have multiple producers produce these messages to a given destination... or if you need them in order use multiple producers and then a resequencer after they are in the broker.

whaley
  • 16,075
  • 10
  • 57
  • 68
  • Thank ou for your interesing links and your answer. I found out that I can getting higher speed when I use transactions, committing every 100 messages but sending async. Do you know if sending 100 messages async and then commit is the same as sending 100 messages one by one but with sync? – Chris Sep 25 '11 at 14:45
  • I answered under the premise that you had to have sync... if you are using transactions I think the default is async (and probably advised to keep it async as well). – whaley Sep 25 '11 at 16:57
  • I have to be sure that all messages I send will arrive the broker when I call producer.Send() or session.Commit(). When sending async but calling session.Commit() can I be sure that all sent messages arrived and are stored to disk? Or is that something where I can't be sure when sending async, no matter using transactions or not? – Chris Sep 25 '11 at 17:05
  • The commit() constitutes the "synchronous" part, since your client and the broker will ensure any message sent by the client has been received by the broker before that commit() returns successfully. – whaley Sep 26 '11 at 14:11
  • Thank you for your answer. Then I will use that way to send the messages. – Chris Sep 26 '11 at 14:30