72

When running my unit tests, from time to time, even if they pass, at the end of all the tests running, I will get the following error.

On my Jenkins CI build running PhantomJS:

.PhantomJS 2.1.1 (Linux 0.0.0) ERROR
  {
    "message": "An error was thrown in afterAll\nReferenceError: Can't find variable: $ thrown",
    "str": "An error was thrown in afterAll\nReferenceError: Can't find variable: $ thrown"
  }

Or on Chrome:

Chrome 67.0.3396 (Windows 7 0.0.0) ERROR
  {
    "message": "An error was thrown in afterAll\n[object ErrorEvent] thrown",
    "str": "An error was thrown in afterAll\n[object ErrorEvent] thrown"
  }

I also have really unreliable tests, without changing anything some times they would succeed and other times the same tests would fail, so I knew something weird was going on.

Oisin
  • 2,082
  • 2
  • 12
  • 8

16 Answers16

113

My issue was that I had a race condition in my tests due to a very stupid way of setting up my tests, but I wanted to document it here anyways because I struggled to find the answer to my issue on the internet.

What I had somehow done was to declare two beforeEach functions to setup my test, and one of the two was asynchronous, so I had a race condition where sometimes they ran out of order and failed.

Here is how my test looked:

beforeEach(async(() => {
    TestBed.configureTestingModule({
      declarations: [ HomeComponent ]
    })
    .compileComponents();
  }));

  beforeEach(() => {
    fixture = TestBed.createComponent(HomeComponent);
    component = fixture.componentInstance;
    fixture.detectChanges();
  });

So to resolve this I put all the setup into one, synchronous beforeEach.

  beforeEach(() => {
    TestBed.configureTestingModule({
      declarations: [HomeComponent]
    }).compileComponents();
    fixture = TestBed.createComponent(HomeComponent);
    component = fixture.componentInstance;
    fixture.detectChanges();
  });

I wasted too much time trying to figure this out, so I'm putting it here to save someone else.

Oisin
  • 2,082
  • 2
  • 12
  • 8
  • 17
    Don't you now have a race condition that the compilation of components won't necessarily have finished by the time you try to create one? The "very stupid way" is [what the CLI generates](https://angular.io/guide/testing#cli-generated-tests) and is referred to as [*"A typical approach"*](https://angular.io/guide/testing#compilecomponents-is-async). – jonrsharpe Jul 26 '18 at 16:12
  • I'm not saying if it's right or wrong (though I lean towards wrong), but the docs say what you had was correct because `compileComponents` returns a promise. [https://angular.io/guide/testing#calling-compilecomponents](https://angular.io/guide/testing#calling-compilecomponents) – Todd Sjolander Mar 12 '19 at 18:16
  • 3
    To improve this answer, please refer to [Calling compileComponents()](https://angular.io/guide/testing#calling-compilecomponents) section of the official Angular testing guide. Tl;dr: if you're using `ng test` (which I hope many people do), calling `compileComponents()` is unnecessary since `ng` CLI already compiles components for us. Thus, no need to call this method, so no need to pass `async()` to `beforeEach()`. Thus, we're getting synchronous calls throughout whole test suite without any race conditions. – Mladen May 20 '19 at 10:52
  • 1
    The point of calling `compileComponents()` is to make tests runnable without CLI which is very common on CI environments. – Mozgor Jun 04 '19 at 12:59
60

I had a similar problem, it seems that since Angular v6 & Karma v3 this vague afterAll error has started to appear (https://github.com/jasmine/jasmine/issues/1523). For myself this error doesn't fail the test but fails the suite.

After looking at many answers to this problem, it seems that the cause is nearly always different which makes it hard to find help online. One can hope that a patch update will be added at some point to bubble up a better error.

My Error

[INFO] HeadlessChrome 71.0.3542 (Linux 0.0.0) DialogComponent #apply should save. FAILED
[INFO]  Uncaught TypeError: Cannot read property 'nativeElement' of undefined thrown
[INFO]       [31m✗ [39m[31mshould save.[39m
[INFO]  Uncaught TypeError: Cannot read property 'nativeElement' of undefined thrown
[INFO] 
[INFO] HeadlessChrome 71.0.3542 (Linux 0.0.0) DialogComponent #apply should save. FAILED
[INFO]  Uncaught TypeError: Cannot read property 'nativeElement' of undefined thrown
[INFO] HeadlessChrome 71.0.3542 (Linux 0.0.0) DialogComponent #apply should save. FAILED
[INFO]  Uncaught TypeError: Cannot read property 'nativeElement' of undefined thrown
[INFO] HeadlessChrome 71.0.3542 (Linux 0.0.0) ERROR
[INFO]   {
[INFO]     "message": "An error was thrown in afterAll\nUncaught TypeError: Cannot read property 'nativeElement' of undefined thrown\nUncaught TypeError: Cannot read property 'nativeElement' of undefined thrown",
[INFO]     "str": "An error was thrown in afterAll\nUncaught TypeError: Cannot read property 'nativeElement' of undefined thrown\nUncaught TypeError: Cannot read property 'nativeElement' of undefined thrown"
[INFO]   }

Finding the problematic test

I got this afterAll error message and had no idea what was causing it, or what test triggered it. The first thing I did was install the karma-spec-reporter: npm install karma-spec-reporter --save-dev

Add this to the plugins array in the karma.conf.js file: require('karma-spec-reporter') This gives you the spec reporter, and add spec into the reporters array: reporters: ['spec'],

Next time you run the test you will see the afterAll error in the console after the problematic test.

My Issue / Solution

I found that the test was calling htmlElement.click(). I changed this to: htmlElement.dispatchEvent(new Event('click)) And voila the tests started to pass.

Summary

As a general rule I avoid using .click() on HTMLElement's now. Also when users interact with the UI it's done via events, so it mimics the users actions more correctly which is always a good thing when testing.

David Glass
  • 2,334
  • 1
  • 25
  • 35
RonanCodes
  • 1,016
  • 9
  • 14
  • 7
    The plugin karma-spec-reporter is very handy, it outputted the name of the spec file before the tests are run and it was far easier to see which tests were causing this issue. –  Dec 05 '19 at 09:38
  • 3
    Did you mean to include `require('karma-spec-reporter')` to the end of the sentence "Add this to the plugins array in the karma.conf.js file:" in your answer? Without that "require" piece (meaning, you're trying to just add the 'spec' to the reporters array), you'll receive an error. – Gregg L Oct 07 '20 at 15:27
47

Another thing which has helped me as well and solved a few of my problems was to add the statement to destroy the fixture after each test

  afterEach(() => {
    fixture.destroy();
  });
aquilesb
  • 2,182
  • 1
  • 19
  • 19
  • 1
    This works for me too. Was having this issue on github actions – IdiakosE Sunday Jul 01 '20 at 13:05
  • Thank you very much! I had a similar error: An error was thrown in afterAll Failed: Cannot destructure property 'profile' of 'undefined' as it is undefined. - and this was the solution! Cheers! – eyesfree Nov 24 '20 at 11:16
  • God this finall worked! thanx. I kept getting afterAll Failed: SyntaxErrors , randomly from different components at different times – Glare Storm Jun 10 '21 at 15:49
  • This worked for me too! But I don't understand why... perhaps some dangling variable left behind in memory? I would have thought karma would completely tear down the tests each time? – ProxyTech Sep 01 '21 at 23:12
  • This worked for me. The problem was a hanging dialog. – otembajelle Jul 09 '22 at 09:58
  • Thank God, thank you so much, it seems to have solved my problem ! – marojbor May 09 '23 at 09:12
31

We were facing similar issues, both with the same intermittent error and the intermittent failing tests. This came up when, after upgrading to Angular 6 we also upgraded to Jasmine 3 wherein running tests in random order is now the default.

As noted by @marsraits below, that such a change was the source of these errors means that our tests were improperly sharing dependencies, and thus the true source of the error is in our own code. That being said, rather than rewriting numerous legacy tests, by turning random off for the test runner we were able to quickly resolve the issue.

We did this by adding this setting in the karma.conf.js:

  config.set({
    client: {
      jasmine: {
        random: false
      }
    }
  })
dibbledeedoo
  • 1,479
  • 1
  • 11
  • 9
  • 54
    Usually when tests sometimes fail and sometimes not, it indicates that there are dependencies between tests which should be isolated. Setting "random: false" just hides the problem. Correct solution would be to track down the root cause and remove dependencies. – martsraits Jan 16 '19 at 12:00
  • 4
    While I totally agree with @martsraits above, this answer can save the day when you have a large flaky test suite and your CI/CD build fails intermittently. This gives you time to address the root cause. – ynovytskyy May 02 '19 at 11:14
  • Same issue in my case, `object ErrorEvent` without stacktrace, randomly failing in different test cases in each run, I can't debug it, and only happends on PhantomJS, not in Chrome or ChromeHeadless – Matias Fernandez Martinez Oct 23 '19 at 11:36
  • 1
    The issue I have is that there is no definitive traceback provided that leads to the source of the tests that are failing. It's nearly impossible to debug. –  Dec 05 '19 at 09:34
  • All good points. I've updated my answer to better reflect the feedback. – dibbledeedoo Oct 18 '20 at 14:00
28

When this error happens, check the browser opened by karma and check its console for errors.

Typically there will be a stack trace there which will help you fix the issue. This also applies for other errors thrown by karma which are not informative.

Rui Marques
  • 8,567
  • 3
  • 60
  • 91
4

I'm extremely grateful to Oisin, as his answer pointed me into the right direction. Please consider this answer as a complement to his.

There are, however, two aspects which, IMHO, need clarification:

  1. We're not dealing with a race condition.
    A race condition would mean the two instances of beforeEach run in parallel and we cannot determine which one finishes first. In reality, the non-async beforeEach runs first and the async one runs second.
    Every single time.
    When you have two beforeEachs and one of them is async (including the ones using the shiny waitForAsync wrapper provided by @angular/core/testing), the async instance's execution gets pushed at the end of the execution queue.

  2. I also find Oisin's proposed solution:

[...] put all the setup into one, synchronous beforeEach.

... too limiting. It doesn't have to be synchronous. It can be asynchronous without a problem.

The important bit is that, TestBed.createComponent() should run after TestBed.configureTestingModule() has resolved.
That's all there is to it.

To make it crystal clear, this random example:

import { TestBed, waitForAsync } from '@angular/core/testing';
// more imports...

describe('SomeComponent', () => {
  beforeEach(waitForAsync(() => {
    TestBed.configureTestingModule({
      declarations: [SomeComponent],
      imports: [SharedModule, RouterTestingModule, HttpClientTestingModule],
      providers: [{
        provide: SomeService, useValue: {
          someObservableMethod$: () => of()
        } as Partial<SomeService>
      }]
    })
      .overrideModule(MatIconModule, MatIconModuleMock)
      .compileComponents();
  }));

  beforeEach(() => {
    fixture = TestBed.createComponent(SomeComponent);
    component = fixture.componentInstance;
    fixture.detectChanges();
  });

  /* tests here */
});

... should be turned into:

import { TestBed, waitForAsync } from '@angular/core/testing';
// more imports...

describe('SomeComponent', () => {
  beforeEach(waitForAsync(() => {
    TestBed.configureTestingModule({
      declarations: [SomeComponent],
      imports: [SharedModule, RouterTestingModule, HttpClientTestingModule],
      providers: [{
        provide: SomeService, useValue: {
          someObservableMethod$: () => of()
        } as Partial<SomeService>
      }]
    })
      .overrideModule(MatIconModule, MatIconModuleMock)
      .compileComponents();

    fixture = TestBed.createComponent(SomeComponent);
    component = fixture.componentInstance;
    fixture.detectChanges();
  }));

  /* tests here */
});

The code in the second (synchronous) beforeEach was appended to the first (asynchronous) beforeEach. That's it.

tao
  • 82,996
  • 16
  • 114
  • 150
4

I've just had faced this problem. My problem was related to HttpClient. I was mocking a httpClient to return error 503 and I was not creating the error function on the subscribe:

  httpClient.get('/api').subscribe(() => {
      fail('Should not return success');
    }, (data) => {
      expect(data.status).toEqual(503); // without adding this callback here, you will get the error of this question
    });
    const request = mock.expectOne('/api');

    request.flush({data: 'test'}, { status: 503, statusText: 'Internal Error'});
3

My specific issue with this error was caused by not mocking sub-components of the component I was testing. In this case I had a homepage component with two sub components, which required declarations for the sub components, which I failed to mock.

As a result the sub components had real dependencies which would intermittently cause the tests to fail in this non-obvious manner (it looks like different tests are randomly failing, but it is not the case).

Mocking as follows works pretty well in this case:

@Component({
    selector: 'app-exercise',
    template: '<p>Mock Exercise Component</p>'
})
class MockExerciseComponent {
}

@Component({
    selector: 'app-user',
    template: '<p>Mock User Component</p>'
})
class MockUserComponent {
}

describe('HomepageComponent', () => {
    let component: HomepageComponent;
    let fixture: ComponentFixture<HomepageComponent>;

    beforeEach(async(() => {
        TestBed.configureTestingModule({
            // note you need to mock sub components!
            declarations: [HomepageComponent, MockExerciseComponent, MockUserComponent],
Richard
  • 4,079
  • 1
  • 14
  • 17
  • 5
    It's better to use `schemas: [ NO_ERRORS_SCHEMA ]` in your TestBed configuration than to mock child components. It will then ignore any child components that it cannot resolve. This abstracts your component away from its children so that you can test in isolation. – jkyoutsey Dec 21 '18 at 20:05
  • Thanks @jkyoutsey - will come in handy – Richard Jan 08 '19 at 02:14
  • 7
    Actually, I no longer endorse using NO_ERRORS_SCHEMA. It hides too many errors. Here are my current recommendations https://medium.com/@fivedicephoto/why-you-shouldnt-use-no-errors-schema-in-angular-unit-tests-cdd478c30782 – jkyoutsey Jan 08 '19 at 15:10
2

I had similar issue as well. In my angular 10 project. Out of 10 times, 8 times it would fail with afterall error. Turns out, in spec.ts files, the imports should have HttpClientTestingModule instead of HttpClientModule. Try this. It might solve the random error in Afterall issue.

1

In my case, the problem seems to be the lack of "catch".

Instead of just :

this.service.getUsers().subscribe( r => { doSomething... } );

I have to do :

this.service.getUsers().subscribe( r => { doSomething... }, err => { doSomething... }  );
Henry Ecker
  • 34,399
  • 18
  • 41
  • 57
Rubens
  • 91
  • 3
1

In my case: the problem seems to be the lack of "catch". Instead of just :

this.service.method().subscribe( r => { doSomething... } );

I have to do :

this.service.method().subscribe( r => { doSomething... }, err => { doSomething... } );
Suraj Rao
  • 29,388
  • 11
  • 94
  • 103
Rubens
  • 91
  • 3
1

In my case, with the following system: Angular 14 Linux

Upgrading puppeteer from version 2 to 18 solved the issue.

Eli
  • 1,670
  • 1
  • 20
  • 23
0

In my case

I am using karma-parallel and when I updated the executors number it worked (don't know why)

 parallelOptions: {
      executors: 4, // Earlier it was 5 I have updated it to 4 and it worked
      shardStrategy: 'round-robin',
      ......
 }
Bhavin
  • 970
  • 1
  • 13
  • 20
0

One possible problem is that one of your components runs a third-party script that makes a CORS request that fails. (My problem was solved by https://github.com/karma-runner/karma/issues/1268#issuecomment-70422477) This can make tests appear to work on Chrome and fail intermittently on your CI.

When in doubt, follow the advice of Rui (https://stackoverflow.com/a/56662721/3370010) and open up your console, even if all the tests are passing.

waternova
  • 1,472
  • 3
  • 16
  • 22
0

In my case, using jasmine, the problem was that I throw an error in tested function. So this thrown error appears in afterAll.

this.userService.getUsers().subscribe(
  (data: User[]) => {
    if (data?.length === 0) {
      ...
    } else if (data?.length !== 0) {
      ...
    } else {
      throw new Error('Not valid response'); // This cause the error in afterAll if tested
    }
  }
);
Experimenter
  • 2,084
  • 1
  • 19
  • 26
0

I had this issue and this issue helped me to solve it. In my case it was subscribe in it without anyway test handling asynchronous code. More here

So finding that handing subscription around, and wrapping it with fakeAsync helped me to solve the issue.