How can I integrate parallel_test and thinking sphinx in cucumber tests?

Question

I have Rails 4 app with cucumber features. In some tests i have thinking sphinx data such as:

@javascript @sphinx
Feature: Edit a service
  Scenario: Editing
    Given I exist as an "individual"
    And I have few services as individual user
    And Sphinx indexes all models
    ...
    Then I should see I18n translation for key "views.messages.notices.add.updated"

Where @sphinx hook contains this logic:

Before('@sphinx') do
  self.use_transactional_fixtures = false
  ThinkingSphinx::Test.start
  sleep 2 # Give some time for sphinx to start.
end

After('@sphinx') do
  ThinkingSphinx::Test.stop
  self.use_transactional_fixtures = true
end

And Sphinx indexes all models contains this:

Given(/^Sphinx indexes all models$/) do
  ThinkingSphinx::Test.index
  # Wait for Sphinx to finish loading in the new index files.
  sleep 0.25 until index_finished?
end

Here is database.yml:

development:
  adapter: postgresql
  database: my-project_development
  host: localhost
  encoding: UTF8
  pool: 5

test: &test
  adapter: postgresql
  database: my-project_test<%= ENV['TEST_ENV_NUMBER'] %>
  username: my-project
  password: my-project
  host: localhost
  encoding: UTF8
  pool: 15

When I run my specs via rake parallel:features (via parallel_tests) specs with thinking sphinx ALWAYS fail:

Can't connect to MySQL server on '127.0.0.1' (61) (ThinkingSphinx::ConnectionError)
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/thinking-sphinx-3.1.0/lib/thinking_sphinx/connection.rb:90:in `rescue in query'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/thinking-sphinx-3.1.0/lib/thinking_sphinx/connection.rb:93:in `query'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/thinking-sphinx-3.1.0/lib/thinking_sphinx/connection.rb:75:in `query_all'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/thinking-sphinx-3.1.0/lib/thinking_sphinx/search/batch_inquirer.rb:17:in `block in results'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/thinking-sphinx-3.1.0/lib/thinking_sphinx/connection.rb:37:in `block in take'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/innertube-1.1.0/lib/innertube.rb:138:in `take'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/thinking-sphinx-3.1.0/lib/thinking_sphinx/connection.rb:35:in `take'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/thinking-sphinx-3.1.0/lib/thinking_sphinx/search/batch_inquirer.rb:16:in `results'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/thinking-sphinx-3.1.0/lib/thinking_sphinx/middlewares/inquirer.rb:9:in `block in call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/activesupport-4.0.3/lib/active_support/notifications.rb:159:in `block in instrument'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/activesupport-4.0.3/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/activesupport-4.0.3/lib/active_support/notifications.rb:159:in `instrument'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/thinking-sphinx-3.1.0/lib/thinking_sphinx/search/context.rb:22:in `log'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/thinking-sphinx-3.1.0/lib/thinking_sphinx/middlewares/inquirer.rb:8:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/thinking-sphinx-3.1.0/lib/thinking_sphinx/middlewares/geographer.rb:11:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/thinking-sphinx-3.1.0/lib/thinking_sphinx/middlewares/sphinxql.rb:14:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/middleware-0.1.0/lib/middleware/runner.rb:31:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/middleware-0.1.0/lib/middleware/builder.rb:102:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/thinking-sphinx-3.1.0/lib/thinking_sphinx/search.rb:67:in `populate'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/thinking-sphinx-3.1.0/lib/thinking_sphinx/search.rb:113:in `method_missing'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/squeel-1.1.1/lib/squeel/adapters/active_record/relation_extensions.rb:323:in `hash'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/squeel-1.1.1/lib/squeel/adapters/active_record/relation_extensions.rb:323:in `-'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/squeel-1.1.1/lib/squeel/adapters/active_record/relation_extensions.rb:323:in `merge_resolving_duplicate_squeel_equalities'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/squeel-1.1.1/lib/squeel/adapters/active_record/relation_extensions.rb:45:in `merge'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/activerecord-4.0.3/lib/active_record/relation.rb:555:in `with_default_scope'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/activerecord-4.0.3/lib/active_record/relation.rb:582:in `exec_queries'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/activerecord-4.0.3/lib/active_record/relation.rb:471:in `load'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/activerecord-4.0.3/lib/active_record/relation.rb:220:in `to_a'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/activerecord-4.0.3/lib/active_record/relation.rb:565:in `blank?'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/activesupport-4.0.3/lib/active_support/core_ext/object/blank.rb:20:in `present?'
      /Users/serj/Projects/my-project/app/controllers/services_controller.rb:90:in `find_services'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/activesupport-4.0.3/lib/active_support/callbacks.rb:537:in `_run__1953055605195034856__process_action__callbacks'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/activesupport-4.0.3/lib/active_support/callbacks.rb:80:in `run_callbacks'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/abstract_controller/callbacks.rb:17:in `process_action'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/action_controller/metal/rescue.rb:29:in `process_action'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/action_controller/metal/instrumentation.rb:31:in `block in process_action'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/activesupport-4.0.3/lib/active_support/notifications.rb:159:in `block in instrument'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/activesupport-4.0.3/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/activesupport-4.0.3/lib/active_support/notifications.rb:159:in `instrument'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/action_controller/metal/params_wrapper.rb:245:in `process_action'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/activerecord-4.0.3/lib/active_record/railties/controller_runtime.rb:18:in `process_action'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/abstract_controller/base.rb:136:in `process'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/abstract_controller/rendering.rb:44:in `process'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/action_controller/metal.rb:195:in `dispatch'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/action_controller/metal/rack_delegation.rb:13:in `dispatch'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/action_controller/metal.rb:231:in `block in action'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/action_dispatch/routing/route_set.rb:80:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/action_dispatch/routing/route_set.rb:80:in `dispatch'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/action_dispatch/routing/route_set.rb:48:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/action_dispatch/journey/router.rb:71:in `block in call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/action_dispatch/journey/router.rb:59:in `each'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/action_dispatch/journey/router.rb:59:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/action_dispatch/routing/route_set.rb:680:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/rack-pjax-0.7.0/lib/rack/pjax.rb:12:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/request_store-1.0.5/lib/request_store/middleware.rb:9:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/simple_captcha2-0.2.1/lib/simple_captcha/middleware.rb:20:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/warden-1.2.3/lib/warden/manager.rb:35:in `block in call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/warden-1.2.3/lib/warden/manager.rb:34:in `catch'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/warden-1.2.3/lib/warden/manager.rb:34:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/rack-1.5.2/lib/rack/etag.rb:23:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/rack-1.5.2/lib/rack/conditionalget.rb:25:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/rack-1.5.2/lib/rack/head.rb:11:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/remotipart-1.2.1/lib/remotipart/middleware.rb:27:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/action_dispatch/middleware/params_parser.rb:27:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/action_dispatch/middleware/flash.rb:241:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/rack-1.5.2/lib/rack/session/abstract/id.rb:225:in `context'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/rack-1.5.2/lib/rack/session/abstract/id.rb:220:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/action_dispatch/middleware/cookies.rb:486:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/activerecord-4.0.3/lib/active_record/query_cache.rb:36:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/activerecord-4.0.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:626:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/activesupport-4.0.3/lib/active_support/callbacks.rb:373:in `_run__4201420912784056511__call__callbacks'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/activesupport-4.0.3/lib/active_support/callbacks.rb:80:in `run_callbacks'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/action_dispatch/middleware/remote_ip.rb:76:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/cucumber-rails-1.4.0/lib/cucumber/rails/action_controller.rb:10:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/railties-4.0.3/lib/rails/rack/logger.rb:38:in `call_app'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/railties-4.0.3/lib/rails/rack/logger.rb:20:in `block in call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/activesupport-4.0.3/lib/active_support/tagged_logging.rb:67:in `block in tagged'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/activesupport-4.0.3/lib/active_support/tagged_logging.rb:25:in `tagged'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/activesupport-4.0.3/lib/active_support/tagged_logging.rb:67:in `tagged'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/railties-4.0.3/lib/rails/rack/logger.rb:20:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/action_dispatch/middleware/request_id.rb:21:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/rack-1.5.2/lib/rack/methodoverride.rb:21:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/rack-1.5.2/lib/rack/runtime.rb:17:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/activesupport-4.0.3/lib/active_support/cache/strategy/local_cache.rb:83:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/actionpack-4.0.3/lib/action_dispatch/middleware/static.rb:64:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/rack-1.5.2/lib/rack/sendfile.rb:112:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/railties-4.0.3/lib/rails/engine.rb:511:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/railties-4.0.3/lib/rails/application.rb:97:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/rack-1.5.2/lib/rack/builder.rb:138:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/rack-1.5.2/lib/rack/urlmap.rb:65:in `block in call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/rack-1.5.2/lib/rack/urlmap.rb:50:in `each'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/rack-1.5.2/lib/rack/urlmap.rb:50:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/capybara-2.2.1/lib/capybara/server.rb:19:in `call'
      /Users/serj/.rvm/gems/ruby-2.0.0-p353@my-project/gems/rack-1.5.2/lib/rack/handler/webrick.rb:60:in `service'
      /Users/serj/.rvm/rubies/ruby-2.0.0-p353/lib/ruby/2.0.0/webrick/httpserver.rb:138:in `service'
      /Users/serj/.rvm/rubies/ruby-2.0.0-p353/lib/ruby/2.0.0/webrick/httpserver.rb:94:in `run'
      /Users/serj/.rvm/rubies/ruby-2.0.0-p353/lib/ruby/2.0.0/webrick/server.rb:295:in `block in start_thread'

But if I run these failed tests in non parallel mode (I mean cucumber path/to/my/test.rb), it works (and yes, parallel_tests works well for the other tests). How can I solve this trouble?

I see you use `postgres` in your `database.yml` how come you get a `MySQL server` error? — xlembouras, Apr 28 '14 at 12:33
@xlembouras I did it because documentation states that we need to do it (http://pat.github.io/thinking-sphinx/installing_sphinx/mac.html): `Make sure you do have MySQL installed so the SphinxQL/mysql41 protocol behaves correctly.` — ExiRe, Apr 28 '14 at 12:40
Which db do you actually use? can you update the question with your full `database.yml` ? — xlembouras, Apr 28 '14 at 12:51
The error is indicating that you can't connect to Sphinx (as it operates with the MySQL protocol, hence the mysql2 gem is used). The error message has been made to be more clear in TS v3.1.1. — pat, Apr 28 '14 at 14:05
@xlembouras @pat I use PosgreSQL as my main db. I updated `database.yml` — ExiRe, Apr 29 '14 at 06:10

score 2 · Accepted Answer · answered Apr 28 '14 at 14:11

You'll want to configure config/thinking_sphinx.yml to include that environment variable TEST_ENV_NUMBER in the paths for Sphinx files - the settings listed in the deployment page of the docs should be what's required, plus the port number (the mysql41 setting) as well. Perhaps something like this?

test:
  mysql41: <%= ENV['TEST_ENV_NUMBER'].to_i + 9307 %>
  pid_file: /path/to/app/tmp/searchd.<%= ENV['TEST_ENV_NUMBER' %>.pid
  indices_location: /path/to/app/db/sphinx/<%= ENV['TEST_ENV_NUMBER' %>
  configuration_file: /path/to/app/config/test.<%= ENV['TEST_ENV_NUMBER' %>.sphinx.conf
  binlog_path: /path/to/app/db/sphinx/<%= ENV['TEST_ENV_NUMBER' %>/binlog

I've not used this approach (or parallel_test) myself, so this is certainly guesswork, but it's worth a shot.

Thank you very much for your answer! It helped me to solve the problem! I updated the wiki for the `parallel_test` with working solution, which could be just copied and pasted: https://github.com/grosser/parallel_tests/wiki#with-sphinx----by-ujh — ExiRe, Apr 29 '14 at 08:25

How can I integrate parallel_test and thinking sphinx in cucumber tests?

1 Answers1