Threading in rails console with active record doesn't find Model in the database

Question

I'm using Rails 3 and Ruby 1.9.

I'm running 2 methods in various rails tests (and in the console). The methods are called index_cases and index_new_cases and the method bodies are shown below. The contents of the index_new_cases method probably aren't relevant (I'm indexing ModelCase information using the Sunspot gem), but I leave it there for completeness.

I have 3 case_numbers. Each case_number matches a ModelCase in the database (i.e. there are 3 ModelCase records in the db).

When I use those 3 case_numbers to run tests on the index_cases method, the index_new_cases method does NOT retrieve any cases using the ModelCase.where… method. However, if i remove the "threading" calls in the index_cases method, the index_new_cases function now retrieves all 3 cases and indexes them properly.

Can anyone explain to me why my threads can't find the database records? Is my threading implementation wrong? Thanks!

  def index_cases(case_numbers)
    threads = []
    case_numbers.each_slice(500) do |slice_of_case_numbers|
      threads << Thread.new(slice_of_case_numbers) do |a_slice|
        index_new_cases(a_slice)
      end
    end
    threads.each {|thr| thr.join}
  end

  def index_new_cases(case_numbers)
    cs = ModelCase.where(case_number: case_numbers).includes(:child_tables)
    puts cs.size # prints 0 with threading and 3 without threading
    Sunspot.index(cs)
    Sunspot.commit
  end

This method (without threading) works properly to find and index my database records

  def index_cases(case_numbers)
    #threads = []
    case_numbers.each_slice(500) do |slice_of_case_numbers|
      #threads << Thread.new(slice_of_case_numbers) do |a_slice|
        index_new_cases(slice_of_case_numbers)
     #end
    end
    #threads.each {|thr| thr.join}
  end

Can you post the log info for the query and compare it with the non-threaded version? — zetetic, Jan 21 '11 at 07:10
Last time i tried to do similar things, i found out, active record is not thread-safe or capable, I have problem connecting to database. while i moved to a activerecord pool or multi process solution, seems to fix. btw i was on 1.9 activerecrod3 as well. — c2h2, Jan 29 '11 at 16:47
I was going nuts until I saw the answer below. It did it for me; check my comment for details =) You might want to accept @MGPalmer's answer — Abdo, Jun 04 '13 at 15:34

Martin T. · Accepted Answer · 2011-08-23T16:00:29.530

6

I had a very similar problem, though only in tests.

The problem lies in the transactions that are used in test cases (when transactional fixtures are on) - changes to data are not visible to other connections as long as the transaction is not committed.

See http://api.rubyonrails.org/classes/ActiveRecord/Transactions/ClassMethods.html

As a consequence changes to the database are not seen outside your connection until the operation is complete.

Threads should have a new connection to the database. And since when using transactional fixtures, the whole test run is wrapped in a transaction, nothing inside it will be visible to threads, except what they create on their own.

You can turn off the transactions, luckily also for single tests: http://ar.rubyonrails.org/classes/Fixtures.html

edited Aug 23 '11 at 16:00

answered Aug 23 '11 at 15:51

Martin T.

3,132
1
30
31

Thanks! The link above was timing out, http://api.rubyonrails.org/classes/ActiveRecord/Fixtures.html did it for me. Check the answer to http://stackoverflow.com/questions/3774742/activerecord-rollback-does-not-work-in-rails-test ; class FooTest < ActiveSupport::TestCase; self.use_transactional_fixtures = true; end is how you turn off transactional fixtures =) – Abdo Jun 04 '13 at 15:24
A thousand times yes ... my entry with `id: 2` was killing me! – RichieAHB Nov 01 '16 at 15:11

Threading in rails console with active record doesn't find Model in the database

1 Answers1

Linked