0

I'm willing to use ejabberd / mongooseIm in a microservice network. XMPP should be our chat protocol aside from a REST API network. I want to send messages incoming at the xmpp server downstream to worker services. Has anybody done this or could lead me into the right direction?

My first thoughts are using RabbitMQ for sending the new incoming messages to the workers.

Mickaël Rémond
  • 9,035
  • 1
  • 24
  • 44
Alexander Braunreuther
  • 404
  • 5
  • 11
  • What's the interface of the workers? Right now everything looks feasible, but some more details would make it clearer what suggestions you're looking for. – erszcz Feb 24 '16 at 12:13
  • Assuming the API is HTTP/REST you can just write an XMPP client app in a language of your liking which would translate XMPP "commands" you define to respective actions on the workers. If you're looking into extending the server (ejabberd/mongooseim) to do the same, you're stuck with Erlang. – erszcz Feb 24 '16 at 12:31
  • The workers could be accessed via http or amqp for example. The case is a messenger which should connect to the xmpp server. The worker should get all messages the server receives and do some processing on them. – Alexander Braunreuther Feb 24 '16 at 16:08
  • Ok, this all sounds good. What kind of advice are you looking for? – erszcz Feb 24 '16 at 16:25
  • If it's possible to get the data out of ejabberd and if its a good idea - cause this could be done via plugins - i'd like to do it with worker services cause of scaling and having the chance to go for polyglott language wise. – Alexander Braunreuther Feb 24 '16 at 17:07
  • Depending on the approach you prefer you can either scan the message archive (since quite recently the message format there is configurable) or hook into the server with a custom module which will process server events like 'user sent message'. I'll follow up with links, writing from my phone now. Edit: I'm referring to MongooseIM. – erszcz Feb 24 '16 at 17:23

2 Answers2

0

There are basically two choices to giving your workers access to the messages routed by ejabberd / MongooseIM. I'll focus on MongooseIM, since I know it better (DISCLAIMER: I'm in the dev team).

The first is to scan the message archive in an async / polling fashion. The Message Archive Management describes XMPP level protocol for accessing it, but for your use case the important part is message persistence - so just making sure the relevant module (mod_mam) is enabled in server config and the messages will hit the database. The databases supported for MAM are PostgreSQL and Riak, though there was also some work on a Cassandra backend (YMMV). This doesn't require tinkering with the server / in Erlang for as long as there's a DB driver for your language of choice available. Since PR#657 it's possible to store the messages in raw XML or even some custom format if you're willing to write the serialization module.

The second option is to use the server mechanism of hooks and handlers (also available in ejabberd), which can trigger a server action on events like "user sent a message", "user logged in", "user logged out", ... This, however, requires a server side extension written in Erlang. In the simplest case the extension could forward any interesting event (with message content and metadata) via AMQP or just call some external HTTP/REST API - that way the real work is carried out by the workers giving you the freedom with regard to implementation language. This options also doesn't require to enable mod_mam or set up a database for message persistency (which you could still have with a persistent message queue...).

In general, the idea is perfectly feasible.

erszcz
  • 1,630
  • 10
  • 16
  • Thanks for the the very detailed answer, this was pretty much the way I had in mind so it's much clearer for me now how we can build our server structures. – Alexander Braunreuther Feb 24 '16 at 22:26
0

Generally, the most common XMPP extension use to build messaging systems for machines-to-machines, internet of things, microservices, etc is PubSub, as defined in XEP-0060.

This is a module you can enable in ejabberd. It is API based, so you can even customize the behaviour of that module to your application specific.

Pubsub basically allows to decouple senders and receivers and is especially designed for that use case.

Mickaël Rémond
  • 9,035
  • 1
  • 24
  • 44
  • AFAI understand, the question isn't about machine-to-machine communication, but about _extra processing_ of the contents of messages which are exchanged in a regular chat. Thanks for the downvote :p – erszcz Feb 26 '16 at 13:23
  • I understand that it is not about machine to machine. Still, the most common way to build a microservice architecture, to decouple receivers and sender, is to use pubsub. The main goal I building a microservice architecture is decoupling components. – Mickaël Rémond Feb 29 '16 at 12:28