2

Ruby-2.0.0p247 ActiveRecord-4.0.1 Cucumber 1.3.10 Aruba-0.5.3 SimpleCove-0.8.2

We use Cucumber with Aruba in a NON-RAILS project that nonetheless uses ActiveRecord. Our cucumber features exercise the code both in-process and out-of-process. The out-of-process code is executed using the same loader sequences as in production through a startup stub in bin:

#!/usr/bin/env ruby
require 'bundler/setup'
Bundler.require

require 'pathname'
my_dir = Pathname.new(
  File.join( File.dirname(
    __FILE__ ), '../', 'lib/' ) ).realpath.to_s + '/'

require my_dir +  File.basename( __FILE__ )

HllThForexRssFetch::Main.new( ARGV ).execute
#EOF

Our features/support/env.rb file contains this:

$ cat features/support/env.rb
# Must load and start simplecov before any application code
require 'simplecov'
SimpleCov.start do
  add_filter "/features/"
  add_filter "/libexec"
  add_filter "/lib/hll_active_record/"
  add_filter "/test/"
  add_filter "/tmp/"
end
SimpleCov.command_name( "Cucumber Features" )

# Do not use cucumber/rails in standalone projects
#require 'cucumber/rails'

. . .

When our step definitions call the external bin/file through aruba's run command the step definitions work properly and the tests complete as expected but the code coverage is not merged with the rest of the run. What I am seeking are instructions for how to set up simplecov to report the code coverage of the out-of-process tests together with the portions that are run in-process by cucumber directly.

How does one do this?

James B. Byrne
  • 1,048
  • 12
  • 27

2 Answers2

5

I have an environment similar to yours, this is how I got it working:

Assuming a directory tree like:

project
|- bin
|  |- binary
|- lib
|  |- ...
|- spec
|  |- ...
|- features
|  |- support
|  |  |- env.rb
|  |- ...

Fist check this issue https://github.com/colszowka/simplecov/issues/234

It describes that the binary should start simplecov. It's hackish but I added this header to my binary (project/bin/binary):

if ENV['COVERAGE']
  require 'simplecov'

  # As described in the issue, every process must have an unique name:
  SimpleCov.command_name "binary #{Process.pid}"

  # When running with aruba simplecov was using /tmp/aruba as the root folder. 
  # This is to force using the project folder
  SimpleCov.root(File.join(File.expand_path(File.dirname(__FILE__)), '..'))

  SimpleCov.start do
    filters.clear

    # Because simplecov filters everything outside of the SimpleCov.root
    # This should be added, cf. 
    # https://github.com/colszowka/simplecov#default-root-filter-and-coverage-for-things-outside-of-it
    add_filter do |src|
      !(src.filename =~ /^#{SimpleCov.root}/) unless src.filename =~ /project/
    end

    # Ignoring test folders and tmp for Aruba
    add_filter '/spec/'
    add_filter '/test/'
    add_filter '/features/'
    add_filter '/tmp/'
  end
end

Then in the calling of binary inside cucumber the COVERAGE environment variable should be set. In the feature/support/env.rb at the before clause:

require 'simplecov'
SimpleCov.command_name 'Cucumber'

Before do
  # This is using the aruba helper, 
  # cf. https://github.com/cucumber/aruba/blob/master/lib/aruba/api.rb
  set_env('COVERAGE', 'true')
  # This could also be accomplished with the "I set the environment variables to:" step
end

If in your environment you have two Frameworks (like RSpec and Cucumber in this example) don't forget to https://github.com/colszowka/simplecov#merging-results

Aitherios
  • 96
  • 2
0

Using the above, setting the command_name based on the PID of the aruba process causes SimpleCov to accumulate a very large result set and pollutes the results with old runs.

I had better luck setting the command name as follows:

# As described in the issue, every process must have an unique name:
SimpleCov.command_name ARGV.join(' ')

Which causes only the latest run with the same arguments to get included in the results.

ddoherty
  • 275
  • 1
  • 3
  • 14