Delayed_job infinite loop

Question

I am using delayed_job_active_record to schedule quote from my app to be tweeted and shared on FB. The quote created can have 3 status:

Draft where it's just to be saved
Published to publish directly on my app and twitter and facebook
Scheduled to be picked up by delayed_job and published at a later time and date that I define on the form.

Here is my quote.rb model with the current logic:

  # == Schema Information
  #
  # Table name: quotes
  #
  #  id           :integer          not null, primary key
  #  content      :text
  #  author       :string
  #  created_at   :datetime         not null
  #  updated_at   :datetime         not null
  #  published_at :datetime
  #  status       :string
  #  facebook     :boolean
  #  twitter      :boolean
  #  user_id      :integer
  #  error        :text
  #

  class Quote < ActiveRecord::Base

    belongs_to :user
    validates :content, presence: true
    validates :author, presence: true
    # validation using validates_timeliness gem
    # validates_datetime :published_at, :on => :create, :on_or_after => Time.zone.now
    # validates_datetime :published_at, :on => :update, :on_or_after => Time.zone.now
    after_save :schedule

    scope :draft,     ->{ where(status: "Draft") }
    scope :published, ->{ where(status: "Published") }
    scope :scheduled, ->{ where(status: "Scheduled") }

    before_validation :clean_up_status

    def clean_up_status
      self.published_at = case status
                          when "Draft"
                            nil
                          when "Published"
                            Time.zone.now
                          else
                            published_at
                          end
      true
    end

    def schedule
      puts "!!!!!!!!!!!!!!!#{self.status}!!!!!!!!!!!!!!!"
      if self.status == "Scheduled"
        begin
            ScheduleJob.set(wait_until: published_at).perform_later(self)
        rescue Exception => e
            self.update_attributes(status: "Scheduling-error", error: e.message)
        end
      else
        publish
      end
    end

    def publish
      unless self.status == "Draft"
        begin
            if self.facebook
                to_facebook
            end
            if self.twitter
                to_twitter
            end
          self.update_attributes(status: "Published")
        rescue Exception => e
          self.update_attributes(status: "Publishing-error", error: e.message)
        end
      end
    end

    def to_twitter
      client = Twitter::REST::Client.new do |config|
        config.consumer_key        = ENV['TWITTER_KEY']
        config.consumer_secret     = ENV['TWITTER_SECRET']
        config.access_token        = self.user.twitter.oauth_token
        config.access_token_secret = self.user.twitter.secret
      end
      client.update(self.content)
    end

    def to_facebook
      graph = Koala::Facebook::API.new(self.user.facebook.oauth_token)
      graph.put_connections("me", "feed", message: self.content)
    end
  end

I have tried before_create :schedule and before_update :schedule before before_save and none of them seem to work.

My jobs/schedule_job.rb:

class ScheduleJob < ActiveJob::Base
 queue_as :default

 def perform(quote)
   quote.publish
 end
end

With basic config: initializers/delayed_job_config.rb:

Delayed::Worker.destroy_failed_jobs = false
Delayed::Worker.max_attempts = 1

Whenever I schedule a quote delayed_job enter in an infinite loop, eventually publish the quote and keep on runnig in circle.

Here is the log:

rake jobs:work
[Worker(host:Christophes-MacBook-Pro.local pid:8962)] Starting job worker
[Worker(host:Christophes-MacBook-Pro.local pid:8962)] Job ActiveJob::QueueAdapters::DelayedJobAdapter::JobWrapper (id=7) RUNNING
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Published!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!

Any idea what could I be missing? I am using Puma for dev and prod server.

Hmm, is it possible that when your first Publishing-error occurs, a new job is created, but since you have `destroy_failed_jobs` set to false, it's re-running all failed jobs and also creating new jobs? — polskais1, Feb 04 '16 at 16:45
I have added the initializers/config file in an attemp to solve the issue and had no impact whatsoever. In other words I had the issue before. — ChrisEstanol, Feb 04 '16 at 16:48
I see. I guess you could set a `max_run_time` to try to kill the loop early but you shouldn't be getting stuck in a loop to begin with. Does giving it a `max_run_time` stop the loop? If not, that means your app is creating an infinite amount of jobs instead of just trying to run the same job until it succeeds. — polskais1, Feb 04 '16 at 17:00
Nope, giving it a max_run_time doesn't stop the loop. And the jobs error say `stack level too deep`. No idea what it's about... — ChrisEstanol, Feb 04 '16 at 17:29
You are getting the `stack level too deep error` because you are updating an attribute recursively. It's probably caused by `self.update_attributes(status: "Publishing-error", error: e.message)`. Try commenting that update out and see what happens. — polskais1, Feb 04 '16 at 18:01
I commented that update, restarted jobs and server and keep getting the same loopy error. — ChrisEstanol, Feb 04 '16 at 19:59

score 0 · Answer 1 · answered Nov 16 '21 at 00:20

Strange that nobody answered this for you but it appears that the problem is occurring because you are triggering further saves of the same model in your callbacks. Those saves will then call the callbacks again, which will again trigger saves and so on.

You can really only use things like update_column etc... in callbacks which won't trigger callbacks, or add some more complicated logic into the model that checks if you should run a certain callback or not.

Delayed_job infinite loop

1 Answers1