RabbitMq Architecture for distributed POS

Question

We are developing a grocery Point of Sale system to replace an existing legacy system. We are evaluating using RabbitMQ for sending product/price changes down to the tills.

A company may have anywhere from 1-50 stores, and each store may have 1-20 tills. Every till in the store will be receiving the same data.

There will be one central backoffice per company.

There will be a Rabbit broker at the backoffice, and at each store.

In my current design, the backoffice broker has a queue setup for each store. The backoffice server software pushes the changes to these queues.

The store broker has a fanout exchange. When a till connects to the store broker, it creates (if one does not yet exist) a durable queue.

I have setup dynamic shovels from the backoffice store queue to the store exchange. Since the till queues are all durable and the messages persistent, this should be reliable shouldn't it?

I hope I have adequately explained what I am trying to achieve. Does this seem like a decent solution? Or is there a better way?

Answer 1

First of all the question is pretty obscured since there are tons of information to consider. Answering it - i'll try to pin point some issues you should take into consideration:

1. In a very large scale of stores (thousands) connected to the back office you will hit a performance issues trying to dynamic shovel for each store - building a virtual queue hierarchy (queue for regions and then stores, and so on) can be a good idea.
2. Remember, such a solution requires you to have rabbit broker installed on all tiers including tills - usually till hardware is very poor and the business functionality is a very resource demanding. Rabbit broker along with erlang engine will consume a nice part of it trying to receive/send data. Consider leaving the tills queues on the store's broker and using only application client on tills to reach the transportation data. This will reduce the software maintenance needed for a lot of tills too.
3. Regarding the dynamic shovel between the back office and the stores: First, imagine that you have number of unprocessed messages shoveled to the store and the store machine crashes (hard disk issues for example) - you gonna loose data, since you've lost them on the store level but they were also shoveled from the back office - so you have a discrepancy now between your store and the back office. Second, obviously your store level machine will be much weaker then the machine on the back office (or a number of machines in a cluster) - your store level possibly be constantly busy handling shoveled messages from the office. For those scenarios you can consider not using shovel but again use application client on store level to pull or rabbit's QueuingBasicConsumer to get pushed to - this way you have only one place of "truth" which is the back office - the consumer can pull (or get messages by pushing in case of the above consumer), apply them, and only then ack them. This can also help you controlling the pace your stores receiving data, since they are in control.
4. You should think of things like - should the messages be persisted or not, should you wait for confirmations (WaitForConfirmOrDie()) or not.
5. What happens if your tills under the stores shouldn't get the same data - pharmacies sections, electronic sections. Which means you probably can't put a fanout exchange on the store.

There are much more to think about, i think we should get more concrete.