rails 4 rspec sunspot solr tests fail intermittently

Question

Lets set the stage for this question.

Here is our model:

class Deal < ActiveRecord::Base
  belongs_to :retailer
  has_many :content_locations, as: :locatable
  has_many :stores, through: :content_locations

  searchable do
    text :title
    text :store_name
    text :store_sort_name
    text :description
  end

  scope :by_keyword, ->(keyword, limit, offset) {
    search {
      fulltext keyword
      paginate offset: offset, per_page: limit
    }.results
  }
end

Here is the section of our rspec test that hits SOLR:

  describe 'searchable' do
    before do
      DatabaseCleaner.clean
    end
    target = 'foo'

    let!(:deal_with_title) { create :deal, title: target }
    let!(:deal_with_description) { create :deal, description: target }
    let!(:deal_with_store_name) { create :deal, store_name: target }
    let!(:deal_with_store_sort_name) { create :deal, store_sort_name: target }
    let!(:deal_with_nothing) {
      create :deal, title: 'bar', description: 'bar', store_name: 'bar', store_sort_name: 'bar'
    }

    it 'includes matches' do
      sleep 1
      results = Deal.by_keyword(target, 25, 0)

      expect(results).to include(deal_with_title)
      expect(results).to include(deal_with_description)
      expect(results).to include(deal_with_store_name)
      expect(results).to include(deal_with_store_sort_name)

      expect(results).to_not include(deal_with_nothing)
    end
  end

What we are seeing is that our test fails randomly. I can execute this test over and over successfully for 1-3xs but the 4th will fail, or it will fail 4xs and the 5th will fail. The pattern is completely random. As you see we tried adding a sleep to this setup and that doesn't do anything but slow it down.

Any tips would be greatly appreciated here. This has been driving us nuts!

I suppose u tried also sleeping more than 1 sec. Would try with a ridiciously long time. Saving to Solr is async and it just smells for the async problem. I once checked the 'tire' testsuite (elasticsearch gem) to see how they deal with it. And they had none of the tests checking real data. Just checking if the queries were generated correctly. Makes kindof sence. Why checking that solr works correctly? And if you also can rely on the solr gem, just check that deal_by_keyword was called with right params by mocking it — dre-hh, Jun 16 '14 at 22:36
I am considering that approach too, but wanted to make sure the interaction with SOLR is working. — Chris Hough, Jun 16 '14 at 23:35

dre-hh · Accepted Answer · 2014-12-18T09:18:07.943

1

In the Sunspot Model Spec they're allways calling

 Sunspot.commit

after data creation and before any check

Update:

You can also call model.index!, which will immediately sync the particular model into solr index. In one specific case i also had to call model.reload after saving and before indexing, because of some tag relationships attached to the model (however this was acts_as_taggable specific)

  #simple
  Factory.create(:model).tap { |m| m.index! }


  #advanced
  model = Factory.build :model_with_relationships
  model.save
  model.reload #optional
  model.index!

edited Dec 18 '14 at 09:18

answered Jun 16 '14 at 22:46

dre-hh

7,840
2
33
44

hmmm... I like where this is going but we are having the issue with feature specs? thoughts? – Chris Hough Dec 18 '14 at 01:16
updated the answer. We are using `model.index!` in the feature(integration) specs now. However i think this is only because of these tag relationships we have on the models. They are only loaded properly after `model.reload`. Having `Sunspot.commit` only lead to flanky passing specs – dre-hh Dec 18 '14 at 09:17
We did end up going with approach similar to what you posted so I tossed you credit, however, we have a new issue. Would you have insights on this http://stackoverflow.com/questions/27554145/ruby-2-1-rails-4-sunspot-solr-tests-fail-in-suite-but-pass-individually by chance? – Chris Hough Dec 18 '14 at 19:25

score 1 · Answer 2 · answered Jun 18 '14 at 17:52

1

We decided to go a completely different direction and honestly I would not recommend hitting SOLR in real time for anyone reading this post. It is a pain.

Our final solution was to with the sunspot_matchers gem instead.

answered Jun 18 '14 at 17:52

Chris Hough

3,389
3
41
80

rails 4 rspec sunspot solr tests fail intermittently

2 Answers2