1

This is related to another question specific to payment processing, and that is my example use case, but I was considering trying to integrate node.js and ruby on the same server using beanstalkd. Basically, I want to use node.js as my main web server, but when I need to do some payment processing, I'd like to use something robust and stable like ruby.

I was considering trying to use beanstalkd as a way to have node.js queue up payment processing jobs for ruby to perform in the background. The documentation for beanstalkd is a little slim, so I'm having trouble figuring out if this is a good approach, or exactly how I would go about it. From what I can tell though, it should be fairly straightforward to start a beanstalkd process and then have node.js connect to it to send it jobs, and have a ruby script which can perform the jobs and send back the results.

Community
  • 1
  • 1
Russell Leggett
  • 8,795
  • 3
  • 31
  • 45
  • 1
    I would use redis message queue semantics instead. I think is going to be faster on node.js because there is a very good c extension available. – Alfred Feb 06 '11 at 05:51

2 Answers2

3

Beanstalk is appropriate for this task. Make sure you use the binlog option to make the jobs persistent between beanstalkd restarts.

Your node.js processes will use a tube (called, say 'payments') and put jobs into it, with an appropriate priority.

Your Ruby script can then watch the payments tube and process the jobs.

Make sure you give the jobs an adequate TTL - you want to ensure the payment processing has time to complete before beanstalk assumes the job has failed and re-queues it.

Just curious - how will you provide feedback to the customer that the payment has succeeded? Perhaps the Ruby script will update a record in the database?

dkam
  • 3,876
  • 2
  • 32
  • 24
  • Yeah, I was eventually able to find the beanstalkd protocol docs, so this made sense, but yeah, the sticking point was feedback. I'm pretty sure that the only way to do that part would be to have some kind of return tube that node.js is waiting for or maybe even a return tube specific to the job id. In the end, I'm thinking redis might be a better solution. – Russell Leggett Feb 08 '11 at 13:28
0

So after rooting around enough, I did find the documentation I really needed to evaluate beanstalkd. There is a protocol document in the source not linked to from anything I've been reading or the main page (although it is in a folder called doc) that gives better details about its capabilities and limitations.

It seems very nice as an asynchronous worker queue, which is perfect for node.js, and it would be a good fit to communicate with some Ruby code to do payment processing, but as dkam says, how do I get the response back to node.js to be able to update the client. While I think this makes sense for many tasks, it is not sufficient for mine.

Given alfred's advice, I've investigated redis, and while it isn't exactly what I need right out of the box, I think it will be sufficient. There is already an actor library out there built on top of redis for Ruby, so I think I should be able to make something simple that can talk between node and Ruby with roughly actor style semantics, or at least callback semantics.

Russell Leggett
  • 8,795
  • 3
  • 31
  • 45
  • If you were to use Beanstalk, I think you'd have Node.js create a transaction record, set the state to "Queued" or similar. That record would become the transaction state for the customer - Node.js would poll that record. When your Ruby process grabs the job, update the DB record to "Pending", then "Complete" or "Failed" once complete. – dkam Feb 08 '11 at 23:56
  • I'm not certain of this, but Redis has PubSub that I'm guessing does not use polling. Redis is lightning fast, and would be on the same box as the node and ruby processes. I think this would probably be a cleaner setup than polling the database. Happy to hear if I'm wrong, though. – Russell Leggett Feb 09 '11 at 13:47