11

I am trying to generate code coverage reports with IstanbulJS for my code written in TypeScript and tested with Cypress. But things get reported out of place:

IstanbulJS's HTML report with hit counter in the wrong lines

I created a git repository MCVE specifically for this question, so you can reproduce exactly my situation:

git clone https://github.com/papb/cy-ts-istanbul-question
cd cy-ts-istanbul-question
npm install
npm test
# And then open the file `coverage/index.ts.html` to see the image above.

How to fix that?

Details

I have some code written in TypeScript which I transpile and bundle to a single JavaScript (ES6) file with rollup, rollup-plugin-typescript2 and rollup-plugin-istanbul. This works perfectly, my source code in TypeScript becomes a file ready to be included with a <script> tag into the browser and used.

Secondly, I use cypress to run tests on an HTML page which includes the transpiled JS code mentioned above. This also works perfectly, cypress is able to test my functions originally written in TypeScript.

Now, I want to set up coverage reports for those tests. On Cypress FAQ we can find the question Is there code coverage? to which the answer is currently no (regarding built-in functionality) but is in discussion as a welcome thing to be done in the future, and in fact it can be done.

The thing is: the guy who did it above was not using TypeScript. I am. So I have a little extra step to do, and this is where I'm currently stuck. Intuitively, I think this is just a matter of configuring IstanbulJS to follow the source maps correctly, but I couldn't find any documentation on how to do it. Every guide about TypeScript + IstanbulJS that I can find assumes that I am using Mocha, but I'm not - I am using Cypress with a transpiled source coming from TypeScript.

Note: I am aware that in general the usual "code coverage" approach to cypress testing doesn't make much sense, but in my exact situation I think it does, I've thought about it already, please don't make this frame challenge to the question.

EDIT: To be clear, using rollup here is not a hard requirement. If you have a solution that uses something else, it is totally acceptable as well. The important thing is, as title says, Cypress + TypeScript + IstanbulJS.

Pedro A
  • 3,989
  • 3
  • 32
  • 56
  • 3
    Have you really need to use `Cypress`? For a similar thing i used [Intern.js](https://theintern.io/) with some transpilation ES6 to ES5 with `Rollup` and code coverage worked like a charm – Troopers Mar 01 '19 at 08:48
  • @Troopers Hello, thanks for the tip, I will take a look later. I wanted cypress to make interface testing, clicking on buttons, snapshot testing, etc. Exactly what cypress shines at. Can Intern.js do those as well? – Pedro A Mar 01 '19 at 09:11
  • [Functionnal testing](https://theintern.io/docs.html#Intern/3/docs/docs%2Ffunctional-testing.md/writing-a-functional-test) is supported. For snapshot i'm not sure – Troopers Mar 01 '19 at 09:37
  • @Troopers Thanks. I will consider using it, but for now I am still looking for a solution in Cypress. Also, you said you used transpilation ES6 -> ES5, but how about TypeScript? Would it work? – Pedro A Mar 01 '19 at 13:21
  • Intern.js is writed in TypeScript and tested with Intern. So i think this works – Troopers Mar 01 '19 at 13:27
  • Vue-cli can generate a config exactly like this. You should create a sample project and look at what they did – Sean Kelly Mar 07 '19 at 19:30
  • @SeanKelly - Are you sure it makes code coverage reports? Looks like they only use cypress + typescript, but not instanbuljs or nyc – Pedro A Mar 07 '19 at 21:26

2 Answers2

8

I used webpack + babel-loader + @babel/preset-typescript + babel-plugin-istanbul

basically the strategy is:

  • instrument your app code so that coverage is generated on window.__coverage__
  • after cypress spec runs, use cy.writeFile to save the report to .nyc_output
  • and generate the report with cy.exec('nyc report --reporter=html')

you should then be able the view the html coverage report in the coverage/ directory

Here's the changes I made to your project, switching to webpack with fully working code coverage:

https://github.com/Bkucera/cypress-code-coverage/commit/40f88aa27778dc55ad3fae56af66724f73b6496d enter image description here

I have put together other working examples here. It has examples of setting up code coverage on top of a newly ejected create-react-app(uses webpack) and a vanilla typescript + webpack project:

create-react-app-ejected:

cypress with code coverage set up on a newly-ejected create-react-app uses:

  • @babel/preset-typescript
  • @cypress/webpack-preprocessor
  • babel-loader
  • babel-plugin-istanbul

vanilla-typescript-webpack:

code coverage on a vanilla typescript & webpack project uses:

  • @babel/preset-typescript
  • @cypress/webpack-preprocessor
  • babel-loader
  • babel-plugin-istanbul

enter image description here

In both of these I also instrument the cypress code so that you could possibly merge the coverage reports, but I don't do that currently

kuceb
  • 16,573
  • 7
  • 42
  • 56
  • 2
    Perfect!! Thank you very much. Bounty well deserved. I have also opened a PR on your repo with a minor fix. In the near future I will be updating my own repo with your changes, adding them manually, in parts, so I fully understand what is going on. For now I only had the chance to run your code and see that it really works! Thank you very much. – Pedro A Mar 11 '19 at 00:25
0

I spent some time trying to figure this out. It seems to be an issue with how rollup generates compatible code. I was able to get the branch up to 75% by changing your tsconfig target

{
  "compilerOptions": {
    "target": "es5",
    "module": "es2015",
    "declaration": true,
    "declarationMap": true,
    "sourceMap": true,
    "outDir": "./dist",
    "strict": true,
    "esModuleInterop": true
  }
}

It covers all the code, but there must be a branch generated by rollup that's getting passed. But I don't really think Cypress is intended for functional testing like this

Sean Kelly
  • 901
  • 7
  • 27
  • Hello, first of all, thank you very much for spending your time :) I am on my phone now, can't give a detailed look now, but will do later or perhaps tomorrow. By the way, if you think rollup is messing up, feel free to suggest a solution with something else. Rollup is not a hard requirement for this question, only TypeScript, IstanbulJS and Cypress are. – Pedro A Mar 08 '19 at 00:00
  • Thanks for the attempt once again, but this didn't really solve my issue. The coverage changed, but everything is still off in the coverage report: [see here](https://i.imgur.com/Y2YcsGE.png) the output I've got with your modifications. – Pedro A Mar 10 '19 at 14:54