Angular 2 + Karma + karma-jspm + karma-coverage + typescript report phantomJS failure

Question

I've been fighting the last few days with getting coverage reports working and I've hit an issue I haven't been able to resolve. The stack trace I'm receiving, with a couple of lines of surrounding debug code, is

15 07 2016 14:41:53.413:DEBUG [middleware:source-files]: Requesting /jspm_packages/npm/process@0.11.5/browser.js /
15 07 2016 14:41:53.413:DEBUG [middleware:source-files]: Fetching /jspm_packages/npm/process@0.11.5/browser.js
15 07 2016 14:41:53.415:DEBUG [proxy]: proxying request - /jspm_packages/npm/process@0.11.5/browser.js to localhost:9876
15 07 2016 14:41:53.417:DEBUG [middleware:source-files]: Requesting /base/jspm_packages/npm/process@0.11.5/browser.js /
15 07 2016 14:41:53.417:DEBUG [middleware:source-files]: Fetching /home/administrator/assist-2.0/client/jspm_packages/npm/process@0.11.5/browser.js
15 07 2016 14:41:53.419:DEBUG [web-server]: serving: /home/administrator/assist-2.0/client/jspm_packages/npm/process@0.11.5/browser.js
PhantomJS 2.1.1 (Linux 0.0.0) ERROR
  Error: (SystemJS) /home/administrator/assist-2.0/client/jspm_packages/system.src.js:3047:137
        /home/administrator/assist-2.0/client/jspm_packages/system.src.js:3756:33
        /home/administrator/assist-2.0/client/jspm_packages/system.src.js:4251:37
        /home/administrator/assist-2.0/client/jspm_packages/system.src.js:1508:27
        /home/administrator/assist-2.0/client/jspm_packages/system.src.js:2738:28
        register@/home/administrator/assist-2.0/client/jspm_packages/system.src.js:2998:23
        eval code
        eval@[native code]
        __exec@/home/administrator/assist-2.0/client/jspm_packages/system.src.js:1544:18
        execute@/home/administrator/assist-2.0/client/jspm_packages/system.src.js:3723:20
        linkDynamicModule@/home/administrator/assist-2.0/client/jspm_packages/system.src.js:3281:36
        link@/home/administrator/assist-2.0/client/jspm_packages/system.src.js:3124:28
        execute@/home/administrator/assist-2.0/client/jspm_packages/system.src.js:3491:17
        doDynamicExecute@/home/administrator/assist-2.0/client/jspm_packages/system.src.js:774:32
        link@/home/administrator/assist-2.0/client/jspm_packages/system.src.js:972:36
        doLink@/home/administrator/assist-2.0/client/jspm_packages/system.src.js:631:11
        updateLinkSetOnLoad@/home/administrator/assist-2.0/client/jspm_packages/system.src.js:677:24
        /home/administrator/assist-2.0/client/jspm_packages/system.src.js:493:30
        invoke@/home/administrator/assist-2.0/client/node_modules/zone.js/dist/zone.js:323:34
        runGuarded@/home/administrator/assist-2.0/client/node_modules/zone.js/dist/zone.js:230:54
        /home/administrator/assist-2.0/client/node_modules/zone.js/dist/zone.js:206:40
        Evaluating http://localhost:9876/dist/src/components/widgets/nested-table/nested-table.component.js
        Error loading http://localhost:9876/dist/tests/unit/components/nested-table.spec.js
PhantomJS 2.1.1 (Linux 0.0.0): Executed 0 of 0 ERROR (1.095 secs / 0 secs)
15 07 2016 14:41:53.436:DEBUG [karma]: Run complete, exiting.
15 07 2016 14:41:53.437:DEBUG [launcher]: Disconnecting all browsers
15 07 2016 14:41:53.441:DEBUG [launcher]: Process PhantomJS2 exited with code 0
15 07 2016 14:41:53.442:DEBUG [temp-dir]: Cleaning temp dir /tmp/karma-88547336
15 07 2016 14:41:53.449:DEBUG [launcher]: Finished all browsers

karma.conf.js

module.exports = function(config) {
    config.set({
        frameworks: ['jspm', 'jasmine'],

        basePath: '.',

        files: [
            'node_modules/zone.js/dist/zone.js',
            'node_modules/zone.js/dist/jasmine-patch.js',
            'node_modules/reflect-metadata/Reflect.js',
            'node_modules/es6-shim/es6-shim.js',
            'jspm_packages/system-polyfills.js'
        ],

        jspm: {
            loadFiles: [
                'dist/tests/**/*.js',
            ],

            serveFiles: [
                'dist/src/**/*.js',
                // 'src/**/*.ts'
            ]
        },

        proxies: {
            // '/src/': '/base/src/',
            '/dist/src/': '/base/dist/src/',
            '/dist/tests/': '/base/dist/tests/',
            '/jspm_packages/': '/base/jspm_packages/',
        },

        port: 9876,
        logLevel: config.LOG_INFO,
        colors: true,
        autoWatch: true,
        browsers: [
            // 'PhantomJS',
            'PhantomJS2',

        ],

        plugins: [
            'karma-jasmine',
            'karma-jspm',
            'karma-phantomjs-launcher',
            'karma-phantomjs2-launcher',
            'karma-junit-reporter',
            'karma-coverage',
            'karma-sourcemap-loader',
            // 'karma-typescript-preprocessor'
        ],

        reporters: [
            'coverage',
            'junit',
            'dots',
        ],

        junitReporter: {
            outputDir: '.',
            outputFile: "./reports/client-test-results.xml",
            useBrowserName: false
        },

        preprocessors: {
            'dist/src/**/!(*.spec).js!(.map)': [
                'sourcemap',
                // 'typescript',
                'coverage'
            ],
            // 'src/**/*.ts': [
            //     'sourcemap',
            //     'typescript',
            //     'coverage',
            // ],
        },

        coverageReporter: {
            dir: 'reports',
            subdir: 'coverage',
            includeAllSources: true,
            reporters: [
                {
                    type: 'json',
                    file: 'coverage.json'
                },
                // {
                //     type: 'cobertura',
                //     file: 'coverage.xml'
                // },
                // {
                //     type: 'html',
                //     subdir: 'coverage/html'
                // }
            ],
            instrumenterOptions: {
                istanbul: {
                    noCompact: true
                }
            }
        },

        // typescriptPreprocessor: {
        //     options: {
        //         inlineSourceMap: true,
        //         inlineSources: true,
        //         "target": "es5",
        //         "module": "system",
        //         "sourceMap": true,
        //         "emitDecoratorMetadata": true,
        //         "experimentalDecorators": true,
        //         "removeComments": false,
        //         "noImplicitAny": false,
        //     },
        //     transformPath: function(path) {
        //         return path.replace(/\.ts$/, '.js');
        //     }
        // },

        singleRun: true
    })
};

nested-table.spec.ts

import {NestedTableComponent} from '../../../src/components/widgets/nested-table/nested-table.component';
import {beforeEach, describe, expect, it} from '@angular/core/testing';
import {DEBUG_VIEW_TEST_DATA, DEBUG_VIEW_TEST_HEADERS} from '../../../src/db/mockdata';
import { ImmutableMatchers } from '../helpers/jasmine-immutable-matchers';

describe('Nested Table Component', () => {
    let ntable, data, headers;
    beforeEach(function() {
        jasmine.addMatchers(ImmutableMatchers);
        ntable = new NestedTableComponent();
        data = DEBUG_VIEW_TEST_DATA[0].data;
        headers = DEBUG_VIEW_TEST_HEADERS[0].headers;
    });

    it('should return an array of keys', () => {
        expect(ntable.keys(data)).toEqualImmutable(data.keySeq());
    });

    it('should calculate the widths of columns', () => {
        let expected = {
            "Element Property": 4,
            "Key": 2,
            "Property Value": 2,
            "Last Refresh": 2,
            "Element Definition": 2
        }
        expect(ntable.getColWidths(headers)).toEqual(expected);
    });
});

When I change the preprocessors to

preprocessors: {
    'dist/src/!(*.spec).js!(.map)': [
        'sourcemap',
        // 'typescript',
        'coverage'
    ],
    // 'src/**/*.ts': [
    //     'sourcemap',
    //     'typescript',
    //     'coverage',
    // ],
},

I don't receive the stack trace, and it gives me coverage, but obviously not on my other components and such. This appears to be a problem with SystemJS module loading, but I can not figure out what's going on, or if that's even the actual problem.

The tests pass fine when I remove coverage from the reporter list.

I also attempted to use karma-typescript-preprocessor (refer to the commented out bits of code in karma.conf.js), and I was getting a coverage report, but the sourcemap files would not map correctly, which I also could not figure out.

Has anyone experienced anything similar and found a way to overcome the issue?

Answer 1

I wish I can answer your question directly, but I too struggled to get a project running that has code coverage. I finally broke down and refactored karma-jspm to explicitly includes code coverage.

To see a sample, I created a seed project incorporating all the libraries in your question. The key component is that SystemJS is utilized for development, production, unit tests, e2e tests, and code coverage.

See angular2-jspm-typescript-seed

This project is using @uiuxengineering/karma-jspm instead of the original karma-jspm to support SystemJS code coverage and angular2.

The Karma Config looks like this:

    // Karma configuration
    // Generated on Wed Jul 15 2015 09:44:02 GMT+0200 (Romance Daylight Time)
    'use strict';

    var argv = require('yargs').argv;

    module.exports = function(config) {
      config.set({

        // base path that will be used to resolve all patterns (eg. files, exclude)
        basePath: './src/client',

        // frameworks to use
        // available frameworks: https://npmjs.org/browse/keyword/karma-adapter
        frameworks: ['jspm', 'jasmine'],

        // list of files / patterns to load in the browser
        jspm: {
          stripExtension: false,

          /**
           * main config
           */
          config: 'jspm.config.js',

          /**
           * dev jspm config.
           */
          dev: null,

          /**
           * node jspm config
           */
          node: null,

          /**
           * browser jspm config.
           */
          browser: null,

          packages: 'jspm_packages',

          /**
           * Adapters do any preloading with systemJs before tests start,
           * and impletent the karma.start method.
           *
           * 'angular2' is the only option for now.
           * If not defined, a default adapter is used.
           *
           * @param path to adapter or 'angular2'
           */
          adapter: 'angular2',


          /**
           * Files loaded by system js before app is loaded.
           * They will load in same order provided.
           *
           * Default files are set for 'angular2' adapter.
           * This property will override defaults if set.
           */
          // preloadBySystemJS: [
          //   'zone.js/dist/zone.js',
          //   '@angular/core/testing',
          //   '@angular/platform-browser-dynamic/testing',
          //   'zone.js/dist/jasmine-patch.js',
          //   'zone.js/dist/async-test.js',
          //   'zone.js/dist/fake-async-test.js'
          // ],

          /**
           * Files may be provided in a object to
           * configure specific serve options.
           *
           * Files will be put in an object similar to:
           *
           * {
           *    pattern: 'someFile.js,
           *    included: false,  // configurable
           *    served: true,     // configurable
           *    nocache: false,   // configurable
           *    watched: true     // configurable
           * };
           *
           * All options configured if provided in an object.
           *
           */
          loadFiles: [
            'app/**/*.spec.ts',
            'testing/**/*.ts'
          ],

          /**
           * Files will be put in an object EXACTLY with the options:
           *
           * {
           *    pattern: 'someFile.js,
           *    included: false,
           *    served: true,
           *    nocache: false,
           *    watched: true
           * };
           */
          serveFiles: [
            'app/**/*!(*.spec|*.e2e-spec).ts',
            'app/**/*.html',
            'app/**/*.scss',
            'assets/**/*.json'
          ]
        },

        // must go along with above, suppress annoying 404 warnings.
        proxies: {
          '/app/': '/base/app/',
          '/assets/': '/base/assets/',
          '/jspm_packages/': '/base/jspm_packages/',
          '/scss/': '/base/scss/',
          '/testing/': '/base/testing/'
        },

        // list of files to exclude
        exclude: [],


        // preprocess matching files before serving them to the browser
        // available preprocessors: https://npmjs.org/browse/keyword/karma-preprocessor
        preprocessors: {
          'app/**/!(*.spec).ts': ['jspm']
        },

        // test results reporter to use
        // possible values: 'dots', 'progress'
        // available reporters: https://npmjs.org/browse/keyword/karma-reporter
        // reporters: ['junit', 'coverage', 'spec'],
        reporters: ['mocha', 'jspm'],

        coverageReporter: {

          // map coverage to source typescript or es6 files.
          remap: true,

          dir: process.cwd() + '/test-reports/unit',

          subdir: function(directory) {
            return directory.replace(/\s/g, '_');
          },

          reporters: [

            // will generate html report
            {type: 'html'},

            // will generate json report file and this report is loaded to
            // make sure failed coverage cause gulp to exit non-zero
            {type: 'json', file: 'coverage-final.json'},

            // will generate Icov report file and this report is published to coveralls
            {type: 'lcov'},

            // it does not generate any file but it will print coverage to console
            // a summary of the coverage
            // {type: 'text-summary'},

            // it does not generate any file but it will print coverage to console
            // a detail report of every file
            {type: 'text'}
          ]
        },

        // web server port
        port: 9876,


        // enable / disable colors in the output (reporters and logs)
        colors: true,


        // level of logging
        // possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
        logLevel: config.LOG_DEBUG,

        loggers: [{
          "type": "file",
          "filename": "./log_file.log",
          "maxLogSize": 20480,
          "backups": 3,
          "category": "absolute-logger"
        }],

        // enable / disable watching file and executing tests whenever any file changes
        autoWatch: true,


        // start these browsers
        // available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
        browsers: [
          'Chrome'
        ],

        plugins: [
          require('@uiuxengineering/karma-jspm'),
          'karma-jasmine',
          'karma-chrome-launcher',
          'karma-mocha-reporter',
          'karma-ie-launcher',
          'karma-phantomjs-launcher'
        ],

        customLaunchers: {
          Chrome_travis_ci: {
            base: 'Chrome',
            flags: ['--no-sandbox']
          }
        },

        // Continuous Integration mode
        // if true, Karma captures browsers, runs the tests and exits
        singleRun: false,

        // Passing command line arguments to tests
        client: {
          files: argv.files
        }
      });

      if (process.env.APPVEYOR) {
        config.browsers = ['IE'];
        config.singleRun = true;
        config.browserNoActivityTimeout = 90000; // Note: default value (10000) is not enough
      }

      if (process.env.TRAVIS || process.env.CIRCLECI) {
        config.browsers = ['Chrome_travis_ci'];
        config.singleRun = true;
      }
    };

The Karma config is designed to run in WebStorm as well with the gulp task "test".

Answer 2

I'm also struggling with a working setup for karma, systemjs and typescript. I found a working setup but it's a bit ugly and depends on remaping the coverage report through gulp, because the karma plugin for remaping doesn't work correctly inside karma but only in gulp. I used the systemjs plugin instead of the jspm plugin, as it seams to work better.

// Karma configuration
// Generated on Sat Aug 13 2016 18:33:27 GMT+0200 (CEST)

module.exports = function (config) {
config.set({

    // base path that will be used to resolve all patterns (eg. files, exclude)
    basePath: './',


    // frameworks to use
    // available frameworks: https://npmjs.org/browse/keyword/karma-adapter
    frameworks: ['systemjs', 'jasmine'],


    // list of files / patterns to load in the browser
    files: [

        'node_modules/core-js/client/shim.min.js',
        'node_modules/reflect-metadata/Reflect.js',
        'node_modules/zone.js/dist/zone.js',

//      {pattern: 'node_modules/systemjs/dist/system.src.js', included: false, watched: false},
//      {pattern: 'jspm_packages/**/*.js', included: false, watched: false},

        {pattern: 'jspm_packages/npm/@angular/**/*.js', included: false, watched: false},
        {pattern: 'jspm_packages/npm/rxjs@5.0.0-beta.10/**/*.js', included: false, watched: false},
        'jspm_packages/npm/lodash@4.13.1/lodash.js',
        'jspm_packages/npm/angular2-swiper@0.4.0/dist/ks-swiper.js',
        'jspm_packages/npm/swiper@3.3.1/dist/js/swiper.js',
        'jspm_packages/npm/jquery@2.2.4/dist/jquery.js',
        'jspm_packages/npm/foundation-sites@6.2.3/dist/foundation.js',
        'jspm_packages/npm/symbol-observable@1.0.1/index.js',
        {pattern: 'jspm_packages/npm/symbol-observable@1.0.1/lib/*.js', included: false, watched: false},
        {pattern: 'jspm_packages/npm/rxjs@5.0.0-beta.6/**/*.js', included: false, watched: false},


        {pattern: 'jspm_packages/npm/crypto-js@3.1.6/*.js', included: false, watched: false},
        {pattern: 'jspm_packages/github/**/*.js', included: false, watched: false},
        {pattern: 'jspm_packages/npm/*.js', included: false, watched: false},
        {pattern: 'dev/**/*.js', included: true, watched: true},
        {pattern: 'test/*.spec.js', included: true, watched: true},
        {pattern: 'templates/*.html', included: true, watched: true}
    ],


    // list of files to exclude
    exclude: [],

    proxies: {
//      '/jspm_packages/': '/base/jspm_packages/'
//      '/templates/': '/base/templates/'
    },

    systemjs: {
        configFile: 'dev/systemjs.config.js',
//      serveFiles: ['test/**/*.spec.js'],
        config:     {
            paths: {
//          "github:*": "./jspm_packages/github/*",
//          "npm:*": "./jspm_packages/npm/*"
            },
            map:   {
                'systemjs':          'node_modules/systemjs/dist/system.js',
                'system-polyfills':  'node_modules/systemjs/dist/system-polyfills.js',
                'es6-module-loader': 'node_modules/es6-module-loader/dist/es6-module-loader.js'
            },
            meta:  {
                'dev/app/*': {format: 'register'}
            }
        }
    },

    // preprocess matching files before serving them to the browser
    // available preprocessors: https://npmjs.org/browse/keyword/karma-preprocessor
    preprocessors: {
        'dev/app/**/*.js': ['coverage']
    },


    // test results reporter to use
    // possible values: 'dots', 'progress'
    // available reporters: https://npmjs.org/browse/keyword/karma-reporter
//        reporters: ['spec', 'coverage', 'karma-remap-istanbul'],
    reporters: ['spec', 'coverage'],

    coverageReporter: {
        reporters: [{
            type:   'json',
            subdir: '.'
        }, {
            type: 'text-summary'
        }/*, {
            type: 'html'
        }*/]
    },

    /*
    remapIstanbulReporter: {
        src:     'coverage/coverage-final.json',
        reports: {
            html: 'coverage'
        },
        dest:    './coverage-source'
    }, */

    // web server port
    port: 9876,


    // enable / disable colors in the output (reporters and logs)
    colors: true,


    // level of logging
    // possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
    logLevel: config.LOG_WARN,


    // enable / disable watching file and executing tests whenever any file changes
    autoWatch: true,


    // start these browsers
    // available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
    browsers: ['PhantomJS', 'Chrome'],


    // Continuous Integration mode
    // if true, Karma captures browsers, runs the tests and exits
    singleRun: false,

    // Concurrency level
    // how many browser should be started simultaneous
    concurrency: Infinity
})
};

This worked for me. Additionaly I have this task in my gulpfile

gulp.task('coverage', ['test'], function () {
return gulp.src('coverage/**/coverage-final.json')
    .pipe(remapIstanbul({
        reports: {
            'json': 'reports/coverage/coverage.json',
            'html': 'reports/coverage/html'
        }
    }))
    .pipe(gulp.dest('reports/coverage/'));
});

This setup still has some issues. For example some empty or not "executed" files may break it, because SystemJS complains about registering and not executing. Test are only included through *.spec.js because using **/*.spec.js doesn't load any tests.

I already filled a bug report for karma-remap-istanbul plugin. https://github.com/marcules/karma-remap-istanbul/issues/21 Maybe there is a chance to get is work inside karma.

I Hope that may help to improve your setup and maybe we can even work out a better solution with everything working.

BTW: I tried to use a setup similar to your's and got the following error, I already have seen a lot of time with jspm and systemjs extension:

Module http://localhost:9876/src/common/modules/product/product.js interpreted as global module format, but called System.register.