Consume Table using EventMachine

Question

I have a model Resend in my Ruby project, it contains content and status columns.

What is the best/fastest way to consume all the records with status 0 using EventMachine ?

I want to create a simple worker that try to find records with status==0 every period ( like every 5 mins)

I'm still new to EventMachine and couldn't find so much examples of how to deal with DB.

So far I made something like the following but not sure if it's the best implementation :

$ROOT_PATH ||= File.expand_path("#{File.dirname __FILE__}")

require "rubygems"
require 'eventmachine'
require 'em-http'

require "#{$ROOT_PATH}/app/models/resend.rb"


EventMachine.run do
  EM.add_periodic_timer(5) do
    Resend.active.each do |msg|
        http = EventMachine::HttpRequest.new($RECEIVER_URL).post :body => {:message => msg.content }
        http.callback { msg.update_status! }
    end
  end
end

Any help would be highly appreciated

`File.expand_path("#{File.dirname __FILE__}")` is an anti-pattern that should be avoided. What you mean is `File.expand_path(File.dirname(__FILE__))` as there's no reason to stringify a string. — tadman, Dec 10 '12 at 19:17

score 0 · Answer 1 · answered Dec 10 '12 at 19:19

0

The problem with this approach is you'll trigger a call for each and every Resend in that scope, and you'll do it every five seconds. At the very least you'll need to flag those records as being processed so you don't duplicate effort. If processing any of these takes more than five seconds you will have overlapping calls.

What you really need is some kind of job queue like beanstalkd or delayed_job.

answered Dec 10 '12 at 19:19

tadman

208,517
23
234
262

queue would work good. Jobs to the queue can be added by cron with Whenever. There is Clockwork gem but I prefer manual inserting - simple script/enqueue.rb - https://gist.github.com/4255780 – Sergii Mostovyi Dec 11 '12 at 03:58
I totally agree about flagging the processed records , I already did that using **update_status!** function, I'm using RabbitMQ already for queuing, but this is for something else – Eki Eqbal Dec 11 '12 at 10:48
You're flagging them *after* they've been processed, but you're not flagging them as *being* processed. – tadman Dec 11 '12 at 15:11

score 0 · Answer 2 · answered Dec 11 '12 at 04:09

0

You need evented version of your DB gem. They usually have prefix em- . If it's a mongoDB there is 'em-mongo' gem. For Mysql the best option I heard is built-in async module in mysql2 gem

answered Dec 11 '12 at 04:09

Sergii Mostovyi

1,361
1
15
19

I'm using mysqlplus and just monkey patched it with something like : class Mysql; alias :query :async_query; end would that do the trick ? – Eki Eqbal Dec 11 '12 at 10:49
Check this article: http://www.igvita.com/2008/09/05/asynchronous-database-access-in-ruby/ . Generally, unless you use Goliath or em-synchrony you need a callback to pass result of query. – Sergii Mostovyi Dec 11 '12 at 19:07

Consume Table using EventMachine

2 Answers2