Ruby code coverage without all the requires and method defintions?

Question

When using a super simple simplecov setup like

require 'simplecov'
SimpleCov.start

I get loads of "coverage" for files and code that were not really "executed" (in any sense that I would care about when looking at code coverage for a specific test) at all with the individual test I was running.

For example all the require, module, class, def, attr_accessor etc. are marked in green for all files that were loaded on the coverage report generated. I do not care about those, and would be happy if those files would report 0% coverage if no "actual code" was executed in them.

end, rescue and comments for example are considered not relevant and not marked red or green in any files. I would like similar behavior for the methods listed above.

Is there a way to get a code coverage that really only includes (and measures) the actual code lines being executed that I care about?

Update after first answer: Marking all other code as "not relevant" via e.g. # :nocov: is unfortunately not an option as this would affect thousands of files for each individual test run.

How are [`Kernel#require`](https://ruby-doc.org/core/Kernel.html#method-i-require) (and others) not executed?! It’s plain old good ruby method declared on `Kernel`. — Aleksei Matiushkin, Sep 12 '18 at 11:52
I edited the question to specify that this means code I care about - which does not include all the `require`, `module` etc. in files that were not loaded. Sorry for being imprecise before. — janpio, Sep 12 '18 at 12:14
How do you suppose any tool would distinguish `Foo#bar` from `Kernel#require`; _that was indeed loaded_? — Aleksei Matiushkin, Sep 12 '18 at 12:16
No idea, which is why I am asking here. I only have the usecase, not the solution yet ;) Maybe some configuration to exclude and mark the methods listed above as irrelevant (similar to `end` and `rescue`) perhaps? — janpio, Sep 12 '18 at 12:18
Neither `end` nor `rescue` are _methods_. `require` is indeed _a method_. Declared on `Kernel`. That is _included_ into every ruby object. It might _fail_. — Aleksei Matiushkin, Sep 12 '18 at 12:29
I get that now. I still don't care about these methods in this code coverage result though as it doesn't help me to achieve my goal which is to be able to look at the _for me_ relevant code coverage of specific tests. — janpio, Sep 12 '18 at 12:37

score 4 · Answer 1 · answered Sep 27 '18 at 09:34

SimpleCov is simply showing you what Ruby's Coverage library collects and as the comments on your question have said, those things are executed, but I too find this frustrating - I don't want a class that's loaded, but not used, to be at 40% coverage.

My solution is to remove coverage that only occurs as part of the loading process. We're using Rails, so we start by loading every file in the application (this is easy in rails, there's a eager_load config); take a snapshot of coverage at that time; run the test suite; then finally deduct the snapshot from the final result before simplecov outputs. Any line covered exactly once in the load snapshot and exactly once in the final result is removed.

Below is a custom SimpleCov formatter I cobbled together a while back that does this job. It's a bit of a hack since it messes with the internal result object of the gem, so be aware that it might not be stable for new versions, but it's working for us with the current simplecov (0.16.1). Also note that since it uses Coverage.peek_result this requires Ruby 2.3 or later.

Use it by setting SimpleCovWithoutLoadingFormatter as your formatter for simplecov and in your test suite setup call SimpleCovWithoutLoadingFormatter.take_load_snapshot immediately after loading every file your application.

require 'simplecov'

class SimpleCovWithoutLoadingFormatter
  def self.take_load_snapshot
    @coverage_load_lines_mask = Coverage.peek_result
  end

  def self.coverage_load_lines_mask
    @coverage_load_lines_mask
  end

  def format(result)
    if self.class.coverage_load_lines_mask&.any?
      result_merge_count = result.command_name.split(',').count

      result.files.each do |source_file|
        _file, load_mask = self.class.coverage_load_lines_mask.detect { |file, _load_mask| file == source_file.filename }
        next unless load_mask

        source_file.coverage.each_with_index do |count, i|
          source_file.coverage[i] = nil if count&.positive? && count&.==(load_mask[i] * result_merge_count)
        end
      end
    end

    SimpleCov::Formatter::HTMLFormatter.new.format(result)
  end
end

score 0 · Answer 2 · answered Sep 12 '18 at 12:41

0

The very README of simplecov says:

You can exclude code from the coverage report by wrapping it in # :nocov:.
^{— https://github.com/colszowka/simplecov#ignoringskipping-code}

E. g. to skip require:

# :nocov:
require 'foo'
# :nocov:

answered Sep 12 '18 at 12:41

Aleksei Matiushkin

119,336
10
100
160

Thanks, while this might work for a small project, I unfortunately can't edit a few thousand files for each individual spec being run. (I added that limitation to my initial question for future readers) – janpio Sep 12 '18 at 12:51
4

@janpio it appears you could patch [SimpleCov::LinesClassifier](https://www.rubydoc.info/gems/simplecov/SimpleCov/LinesClassifier) if you really wanted to – engineersmnky Sep 12 '18 at 14:57

Ruby code coverage without all the requires and method defintions?

2 Answers2