0

I have been trying to see if this is possible and so far have found nothing so I will try and ask specifically

Is it possible to have a sidekiq worker which can recive a method as for example a lambda method and pass on arguments to it?

Example case: I need to make some heavy computation on my server and my options are to either make a specific sidekiq worker for the job which will only be done 1 time ever and will end up cloddering my code base, or make a worker which could lets say accept something like:

lot_of_work.each do |args|
  Workers::Tmp::LetsGo.perform_async(args) { |a| a.lets_go }
end

I've tried looking through old stackoverflow posts and documentation for sidekiq. I've tried the above method which I hoped worked as a normal method but it does not.

I would have liked it to execute the method which was pass to the worker such that I do not need to make workers for 1 time cases and dont have to use single thread computation.

Casper Akuma H. Iversen
  • 23
  • 5
  • 1
    I'm not sure how that would work since it has to be handed off to an entirely different process. – tadman Oct 27 '22 at 15:23
  • @tadman Hmmm that is excatly my problem :/ maybe I could write the code as a string and the eval the string and then call the method. That could work I will try that ^^ – Casper Akuma H. Iversen Oct 27 '22 at 15:34
  • 1
    I might be a little out of date in my thinking here, but to the best of my knowledge Sidekiq and other delayed job type systems work well if you can provide arguments that are *trivially serialized*. A `Proc` is not one of those things. You may need to re-think how you're approaching this so you can pass in some plain arguments to a method that can then do the equivalent of that lambda on the other side. – tadman Oct 27 '22 at 15:36
  • @tadman actually my method works now! it was very simple: def perform(method_name, method) eval(method) send(method_name) end And then it will simply just be to write a script to run through what I want ^^ I know that generalizing things is what sidekiq does very well and is also what I mainly use it for, but sometimes when you work with a large database you need to do 1 time changes which does not need its own sidekiq worker and will only cludder up after that which is why I wanted this ^^ – Casper Akuma H. Iversen Oct 27 '22 at 15:41

1 Answers1

0

I found a solution to this problem, there are probably better ones but this worked for me. Make a worker like this:

module Workers
  module Default
    class TesterWorker
      include Sidekiq::Worker
      sidekiq_options queue: :default, retry: false

      def perform(method_name, method)
       eval(method)
       send(method_name)
      end
    end
  end
end

After this you simply just have to write your code as a string like this:

methode_name = 'tester'
spec = "def test; puts 1; end"
Workers::Default::TesterWorker.perform_async(methode_name, spec)

And this will execute the for example the puts 1 action on the sidekiq ^^

Casper Akuma H. Iversen
  • 23
  • 5
  • 1
    The one glaring problem with this appoach will be what happens when the method doesn't work as expected and you have to troubleshoot it. Instead of an actual source location you just have an eval'ed string. I would take tadmans advice. – max Oct 27 '22 at 18:04
  • And if you're concerned with clutter just delete the worker after you're done with the job. Thats after all what VCS is for. – max Oct 27 '22 at 18:07
  • 1
    Not thread-safe. Other threads might be executing similar jobs, all metaprogramming the same Class. Will they clash? Who knows, it's a race! – Mike Perham Oct 27 '22 at 20:22
  • Definitely not a safe solution at all. All I wanted to figure out is if this was technically possible, not necessarily if it is safe practice ^^ – Casper Akuma H. Iversen Oct 28 '22 at 05:44