0

I have a Rails application that is importing data from various third parties. The jobs are taking a long time and I am looking into how I can use threads to speed this up. I know nothing about Java so apologies if this makes no sense.

Shepmaster
  • 388,571
  • 95
  • 1,107
  • 1,366
BC00
  • 1,589
  • 3
  • 29
  • 47

2 Answers2

1

No, JRuby is an alternate Ruby interpreter, so you cannot "switch" to it in the middle of running MRI (the standard Ruby interpreter, written in C).

You can create threads in MRI, but many people use a background job queue to handle this type of problem. If you really wanted to, you could also write a second application in JRuby that your first application made remote calls to.

Shepmaster
  • 388,571
  • 95
  • 1,107
  • 1,366
0

Yes, you can, as long as you intend to run it in a separate process like a rake task for example,

in Gemfile you'll have to handle both options where gem sets are different, for example:

platform :ruby do
  gem "pg"
  gem "ruby-oci8", '>= 2.1.4'
end

platform :jruby do
  gem "activerecord-jdbcpostgresql-adapter"
end

and you need to decide what to do with Gemfile.lock. One option is to keep 2 Gemfile.lock on git, one for JRuby gems and one for MRI Ruby gems - Gemfile.lock.mri and Gemfile.lock.jruby and then symlink accordingly when running bundle.

The other question is if you should, the above approach is a bit finicky and I'm generally using it for building gems or for playing with porting an app to JRuby.

Spawning multiple processes and background job queues is generally how things are done in MRI Ruby, it seems an overkill to introduce a whole new ruby for purpose of a single import task.

That said, I've had situations (on massive data imports done through ruby) where proper multithreading would have saved me a LOT of hassle, but in the end I always fell back to partitioning the problem and running it in separate processes (every odd id in one rake task, every even id in another, for example - bear in mind you might need to rebuild tables in database if you do this or possibly face very slippery regression issues, do a benchmark definitely).

bbozo
  • 7,075
  • 3
  • 30
  • 56
  • This answer is correct if you are OK with running a separate process. The original question wanted a way to "use JRuby just for that script/class". This answer implies that you will be using JRuby for all of the code. I also assumed that this needed to be a long-term solution. If you just needed a one-shot import, then this answer would be spot on, and you could avoid checking into git as a bonus. – Shepmaster Jan 04 '14 at 15:15
  • @Shepmaster, yes, the question isn't clear, I interpreted "running imports from various third parties" to mean something in the lines of cron-triggered rake jobs so separate processes by definition (as opposed to files being uploaded and processed through the web service) – bbozo Jan 04 '14 at 15:43
  • Hey guys- this is very helpful thank you. You were correct in your assumpiton @bbozo. Currently its cron triggered rake jobs and various processes. Just looking for a way to speed up this process. – BC00 Jan 06 '14 at 12:57
  • @BC00, in that case, my answer is what you want to follow and the one you accepted is plain wrong, get me an upvote :) – bbozo Jan 06 '14 at 13:22