What should output for Heroku CI TAP integration look like exactly?

Question

Heroku CI supports TAP (Test Anything Protocol) output, to give an enhanced UI when its detected.

However, I've been unable to get Heroku CI to show this enhanced UI. My guess is that the TAP output I'm producing isn't quite right, however I can't see what's wrong with it.

If you've got TAP output working with Heroku CI specifically and it is definitely showing the enhanced UI, please could you supply an example below?

In addition, can you confirm if its OK to print out any extra information before and/or after the TAP output, or does the output surrounding the TAP report need to be completely empty?

Eliot Sykes · Answer 1 · 2018-07-09T11:00:07.463

At time of writing, on Heroku, TAP output lines that have leading whitespace are not valid. Heroku CI's default test command on the ruby-buildpack auto-indents all test output, so the TAP output was invalid due to these auto-indents.

To workaround the auto-indent and produce valid TAP output, specify a custom test script in app.json. Using a custom test script bypasses the ruby-buildpack's auto-indent.

Here's an example app.json with a custom test script bin/rspec:

{
  "environments": {
    "test": {
      "addons": ["heroku-postgresql:in-dyno"],
      "buildpacks": [
        { "url": "heroku/ruby" },
        { "url": "https://github.com/heroku/heroku-buildpack-google-chrome" }
      ],
      "env": {
        "DISABLE_SPRING": "true"
      },
      "scripts": {
        "test": "bin/rspec"
      }
    }
  }
}

Here's an example of valid TAP (version 12) output that Heroku CI will process to show the enhanced UI (you may need to refresh the page to see the enhanced UI if you've been watching a build in progress):

# Randomized with seed 12345

1..2
ok 1 - PayHelper#js_host returns production host as default
not ok 2 - PayHelper#js_url returns production v9 URL as default
#
#  PayHelper#js_url returns production v9 URL as default
#  Failure/Error: expect(helper.js_url).to eq 'https://example.com/v9/'
#  
#    expected: value != "https://example.com/v9/"
#         got: "https://example.com/v9.0/"
#  
#    (compared using ==)
#  # ./spec/helpers/pay_helper_spec.rb:15:in `block (3 levels) in <top (required)>'
#  # -e:1:in `<main>'
1..2

# Randomized with seed 12345

Note, the above output is TAP specification version 12, though I understand Heroku also supports TAP specification version 13.

What should output for Heroku CI TAP integration look like exactly?

1 Answers1