4

It seems that coverage report with coveralls is not possible for VSCode extension built with TypeScript.

Currently, I am adding test cases to our project https://github.com/PicGo/vs-picgo/pull/42, I have found several ways to report coverages, but none of them work for me.

Using custom TestRunner

The official documentation mentions little about custom test runners, but I found a post here. It works when I use F5 to launch an Extension Test, but does not work when I run npm run test in the console (Got no coverage output at all).

I have also tried to understand the custom runner (source code) in the blog post, but I found I have nothing to do because I do not know why it works.

Using nyc

nyc with mocha is very powerful, but we cannot take advantage of it. When I run nyc ./node_modules/vscode/bin/test, I will got 0% coverage:

20190525000930

I have searched the issue page of nyc, lots of the same 0% coverage problems about TS projects exist, but none of them are the same with our environment. The main difference is that they are using mocha for testing, not like VSCode's ./node_modules/vscode/bin/test script, it will create a new process to run the test js files. I don't know how to deal with this.

I searched all the issues (mocha, tyc, istanbul, vscode, etc...), and there are few (I did not find any ) vscode TypeScripts are using coverage report for me to copy from. So my question is: how do I get the coverage report for my VSCode TS extension?

upupming
  • 1,437
  • 1
  • 12
  • 15

3 Answers3

3

I have struggled with this myself for some time until I got it working properly. There were three main challenges in getting it working:

  1. Proper setup of the nyc instance
  2. Preventing race conditions on startup
  3. Capturing nyc output and displaying it in the debug console

You can find my working test runner here. I'm also sharing additional insights on my blog.

frenya
  • 239
  • 2
  • 11
  • 1
    Please don't post the same thing multiple times! Like [here](https://stackoverflow.com/a/65804621/10871073), [here](https://stackoverflow.com/a/65804630/10871073) and [here](https://stackoverflow.com/a/65804635/10871073). – Adrian Mole Jan 20 '21 at 07:50
  • This answer is great. I could solve my problem, I posted [here](https://stackoverflow.com/q/70398369/10138996). – pascalre Mar 09 '22 at 21:58
2

I got everything working with Mocha, NYC, and VSCode!

You can see my solution to this in https://github.com/jedwards1211/vscode-extension-skeleton.

Basically, I use Babel to transpile my .ts code with @babel/preset-typescript and babel-plugin-istanbul before running the tests. This allows me to skip the convoluted extra steps of instrumenting the tsc output and using remap-istanbul.

Then in the test runner, I use the (not really documented) NYC API to write the coverage to disk after tests finish.

Finally, in my package scripts, I run nyc report after the test command finishes.

UPDATE: you need to delete the .nyc_output folder before each test run too.

src/test/index.js

import NYC from 'nyc'

export async function run(): Promise<void> {
  const nyc = new NYC()
  await nyc.createTempDirectory()
  // Create the mocha test
  const mocha = new Mocha({
    ui: 'tdd',
  })
  mocha.useColors(true)

  const testsRoot = path.resolve(__dirname, '..')

  const files: Array<string> = await new Promise((resolve, reject) =>
    glob(
      '**/**.test.js',
      {
        cwd: testsRoot,
      },
      (err, files) => {
        if (err) reject(err)
        else resolve(files)
      }
    )
  )

  // Add files to the test suite
  files.forEach(f => mocha.addFile(path.resolve(testsRoot, f)))

  const failures: number = await new Promise(resolve => mocha.run(resolve))
  await nyc.writeCoverageFile()

  if (failures > 0) {
    throw new Error(`${failures} tests failed.`)
  }
}
Andy
  • 7,885
  • 5
  • 55
  • 61
0

Add custom test runner

See this post for more information, you can just copy the test runner code to your project's test/index.ts file.

Demo azure pipeline configurations

variables:
  system.debug: true
jobs:
  - job: Windows
    pool:
      name: Hosted VS2017
      demands: npm
    steps:
      - task: NodeTool@0
        displayName: 'Use Node 12.3.1'
        inputs:
          versionSpec: 12.3.1
      - task: Npm@1
        displayName: 'Install dependencies'
        inputs:
          verbose: false

      - task: Npm@1
        displayName: 'Compile sources and run tests'
        inputs:
          command: custom
          verbose: false
          customCommand: 'test'

      # https://stackoverflow.com/questions/45602358/lcov-info-has-absolute-path-for-sf
      - script: 'sed -i -- 's/..\\..\\//g' coverage/lcov.info && npm run coveralls'
        displayName: 'Publish code coverage'  
        env:
          COVERALLS_SERVICE_NAME: $(COVERALLS_SERVICE_NAME)
          COVERALLS_REPO_TOKEN: $(COVERALLS_REPO_TOKEN)

      - script: 'npm install -g vsce && vsce package'
        displayName: 'Build artifact'
      - task: CopyFiles@2
        inputs:
          contents: '*.vsix'
          TargetFolder: '$(Build.ArtifactStagingDirectory)'
      - task: PublishBuildArtifacts@1
        inputs:
          pathtoPublish: '$(Build.ArtifactStagingDirectory)'
          artifactName: vs-picgo-dev-build
trigger:
  branches:
    include:
    - '*'  # must quote since "*" is a YAML reserved character; we want a string
pr:
- dev*

Note that you have to use sed to delete the ..\..\ prefix of SF paths in lcov.info:

Before:

SF:..\..\src\vs-picgo\index.ts

After:

SF:src\vs-picgo\index.ts

Demo project: https://github.com/PicGo/vs-picgo

upupming
  • 1,437
  • 1
  • 12
  • 15