Using Ruby and Minitest, how do I run the same testcase with different data, controlled only by a list

Question

I have Ruby 2.0 code that operates on phone numbers, which I want to test using MiniTest. I have a function which takes a phone number arguments and tests it (including asserts). Each time I call that function, I want it to be a new test case. Something like this:

listOfPhoneNumbersForTesting.each { |phone|   testphone phone }  

What I DO NOT want is this:

class test2125551212 < MiniTest::Unit::TestCase
    def t2125551212 
        testphone "2125551212"
    end
end

... repeated 10, 20, or 100 times to test each phone number ...

Obviously, I could put the loop code within a MiniTest::Unit::TestCase, but that results in just one testcase no matter how many phone numbers I test, and I don't like that. (Also, if one of the asserts fail, then no more phone numbers are tested, and I don't want that!) Also the second form looks like a violation of DRY to me, since the class name, function name, and argument all contain the phone number.

Somehow I feel that I should be able to have one class called TestPhone, and create it with the phone number argument, and get that into the MiniTest framework. but I would be willing to use setup(), Fixtures, metaprogramming, or just about anything else, if it would work.

listOfPhoneNumbersForTesting.each { |phone|   TestPhone.new phone }

Where TestPhone is a TestCase subclass, and ends up calling testphone to do the work.

Basically, what I want is this: 1. One list of phone numbers, and if I add a number to the list, I get one more TestCase in reporting output. 2. If the tests associated with one phone number fail, the rest are still tested. 3. All phone numbers get the same testing, which includes several assertions.

Thanks very much!

Answer 1

You could dynamically define the methods.

In the following example, 6 tests are dynamically created (2 tests for each of the 3 values being tested). This means that if anything fails, the other tests still run.

require "minitest/autorun"
class MyTests < MiniTest::Unit::TestCase
    ['0', '1111111', '2222222'].each do |phone_number|
        define_method("test_#{phone_number}_has_7_characters") do
            assert_equal(7, phone_number.length)
        end

        define_method("test_#{phone_number}_starts_with_1") do
            assert_equal('1', phone_number[0])
        end
    end
end

The apply test case gives the following results:

# Running tests:

F..F.F

Finished tests in 0.044004s, 136.3512 tests/s, 136.3512 assertions/s.

  1) Failure:
test_0_starts_with_1(MyTests) [stuff.rb:13]:
Expected: "1"
  Actual: "0"

  2) Failure:
test_0_has_7_characters(MyTests) [stuff.rb:9]:
Expected: 7
  Actual: 1

  3) Failure:
test_2222222_starts_with_1(MyTests) [stuff.rb:13]:
Expected: "1"
  Actual: "2"

6 tests, 6 assertions, 3 failures, 0 errors, 0 skips

Applying the same concept to your tests, I think you want:

class MyTests < MiniTest::Unit::TestCase
    listOfPhoneNumbersForTesting.each do |phone|
        define_method("test_#{phone}") do
            TestPhone.new phone
        end
    end
end

A similar approach can be taken when using the spec-style tests:

require 'minitest/spec'
require 'minitest/autorun'

describe "my tests" do
  ['0', '1111111', '2222222'].each do |phone_number|
    it "#{phone_number} has 7 characters" do
      assert_equal(7, phone_number.length)
    end

    it "#{phone_number} starts with 1" do
      assert_equal('1', phone_number[0])
    end
  end
end

IMPORTANT: One thing to note is that you need to make sure that the name of the test methods created are unique for each test case.

For example, if you do not put the phone number in the method name, you end up overwriting your previously defined methods. Ultimately this means that only the last phone number gets tested.

This is because MiniTest generates the test methods on the fly and will overwrite already generated test methods, ultimately only using the last .each variable.