1

I am running tests and collecting coverage with Vitest configured to use the c8 coverage library. This seems to be working because when the tests finish, I get a coverage report in the terminal and a directory named coverage is created with all sorts of files I can look at in a browser:

$ ls -la coverage/
total 712
drwxr-xr-x  14 shawn  staff     448 Apr 10 11:08 .
drwxr-xr-x  24 shawn  staff     768 Apr 10 11:23 ..
-rw-r--r--   1 shawn  staff    5394 Apr 10 11:08 base.css
-rw-r--r--   1 shawn  staff    2655 Apr 10 11:08 block-navigation.js
-rw-r--r--   1 shawn  staff  108109 Apr 10 11:08 clover.xml
-rw-r--r--   1 shawn  staff  190921 Apr 10 11:08 coverage-final.json
drwxr-xr-x   4 shawn  staff     128 Apr 10 11:08 examples
-rw-r--r--   1 shawn  staff     445 Apr 10 11:08 favicon.png
-rw-r--r--   1 shawn  staff    5802 Apr 10 11:08 index.html
-rw-r--r--   1 shawn  staff     676 Apr 10 11:08 prettify.css
-rw-r--r--   1 shawn  staff   17590 Apr 10 11:08 prettify.js
-rw-r--r--   1 shawn  staff     138 Apr 10 11:08 sort-arrow-sprite.png
-rw-r--r--   1 shawn  staff    6181 Apr 10 11:08 sorter.js
drwxr-xr-x  11 shawn  staff     352 Apr 10 11:08 src

My next step is to send this coverage information along to Coveralls to get a coverage badge for my README. For this part, I'm using the GitHub Action provided by Coveralls for this use-case: coverallsapp/github-action@v2

This doesn't work, however: it seems the action is not finding the coverage report since it says "Nothing to report".

Here are the relevant parts of the deployment workflow:

[…]
  test:
    […]
    steps:
      […]
      - name: Test
        run: npm test
        env: […]

      - name: Report Coveralls
        uses: coverallsapp/github-action@v2

The npm test part creates the coverage directory (it basically just runs vitest run --coverage).

So why is Coveralls not finding the coverage report? Should I be configuring c8 to produce a different format of output? Do I need extra configuration when calling coverallsapp/github-action@v2? Should I be using istanbul instead of c8, or perhaps something else? Should I be using CodeDov instead of Coveralls, or perhaps something else?

Shawn
  • 10,931
  • 18
  • 81
  • 126

2 Answers2

1

Just wanted to chime in and add a couple of things:

  1. The first is that, given the Istanbul's alternative reporters, you could also have chosen Cobertura (XML) as a format. That said, LCOV is a better choice, if possible, because it's been around longer and has been supported longer by Coveralls.

  2. The second is about the Coveralls Github Action (v2.0) and the fact that it is based on Coveralls' Universal Coverage Reporter (which you already surmised), which is Coveralls' primary official integration.

Per (2), the Coverage Reporter is where Coveralls is putting all its future efforts in terms of their "official" integrations (There are over 30 community-created language integrations), and, while they continue adding support for new coverage report formats and CI services, they have also provided relatively simple developer instructions, with checklists, to add support for new coverage report formats (here, example) and new CI services (here, example).

In addition, Coveralls will compensate you for contributing those with one month of free service, each.

Could be worth it if you're integrating many repos in various languages and want to standardize all those projects around a single Coveralls integration.

James
  • 66
  • 3
0

Nothing in your question tells the action wouldn't work perfectly fine.

Only the coverage data is not found. You need to provide the correct location to the action otherwise it can't work.

Find this explained in the Coveralls Github Action read-me.

You have to use at least one coverage format that is compatible with the coverage reporter used by that Coveralls Github Action.

Showing the HTML coverage directory and insisting in your question text is not enough to actually make it properly configured.

As you haven't asked anything more specifically, this is the answer.

Encouraging you to do even more wild guesses next to those you outline already in your question also does not look useful to me.

My suggestion instead is to focus on the clover.xml file and tell the Coveralls action where to find it. Technically, you can use any other supported format, too.

Correction: The Coveralls Github Action does not yet support clover.xml format.

hakre
  • 193,403
  • 52
  • 435
  • 836
  • Thanks. I created a `vite.config.ts` file with the following contents to configure c8 to use lcov (despite not finding very much at all about lcov in the docs) and now the GitHub Action is finding the report (aka problem solved): `import { defineConfig } from 'vitest/config'; export default defineConfig({ test: { coverage: { reporter: ['lcov'] } } })` – Shawn Apr 11 '23 at 16:02
  • @Shawn: Good to read you could achieve some traction. Isn't the default example the `lcov.info` file? (is that the same as you have?). I'm referring to this setting: https://github.com/coverallsapp/github-action/blob/master/README.md?plain=1#L24 – hakre Apr 11 '23 at 16:22
  • Yes, I saw that, but the coverage report didn't include a `lcov.info` file before creating the vite.config.ts file as described above. My problem was that I wasn't finding documentation that explains how to choose `lcov` as a report format in the context of running `c8` via `vitest`. So instead, I used the introspection capabilities provided by Typescript to find out what values are valid in the `test.coverage` key of the object sent as first parameter to the `defineConfig` function from `vitest/config`. That's how I discovered I could use `reporter:['lcov']`… – Shawn Apr 11 '23 at 17:59
  • This is really an interesting find of yours. The only appearance of `lcov` is on this document, and they don't list it for the formats: https://vitest.dev/config/#coverage-reporter - only within an example code to demonstrate multiple reporters. Correction: They reference the list of reporters, it is at https://istanbul.js.org/docs/advanced/alternative-reporters/ . – hakre Apr 11 '23 at 18:29
  • That looks like Istanbul, not c8… vitest allows using Istanbul but uses c8 by default if I understand correctly – Shawn Apr 12 '23 at 02:26
  • 1
    Yes, but they link only Istanbul there. I think the reason is that c8 also uses the libraries form Istanbul for the coverage report formats. – hakre Apr 12 '23 at 04:33