Diagnosing duplicate spec reporting with Karma, Mocha, and React 16.5

Question

I have a project using React for the view layer. To test it, I am using mocha, karma, karma-webpack, etc. For some reason, in React 16+, karma is reporting the afterEach as having run three times for two specs. This happens only in React 16+ and only when process.env.NODE_ENV is development and not production.

In previous explorations of this issue, the cause(s) of spec failures could cascade and pollute subsequent specs. To help identify the underlying cause, this is the simplest example I can find.

I have tried to trace the behavior, but have been stumped by the complexity within and around karma and sockets. Consider the example below, available for the moment at https://github.com/jneander/react-mocha.

Example.js

import React, {Component} from 'react'

export default class Example extends Component {
  render() {
    try {
      return (
        <div>{this.props.foo.bar}</div>
      )
    } catch(e) {
      console.log(e) // for logging purposes
      throw e
    }
  }
}

Example.spec.js

import {expect} from 'chai'
import React from 'react'
import ReactDOM from 'react-dom'

class ExampleWrapper extends React.Component {
  constructor(props) {
    super(props)

    this.state = {
      error: false
    }
  }

  componentDidCatch(error) {
    console.log('there was a problem')
    this.setState({
      error: true
    })
  }

  render() {
    console.log('rendering!')
    if (this.state.error) {
      console.log('- rendering the error version')
      return <div>An error occurred during render</div>
    }

    console.log('- rendering the real version')
    return (
      <Example {...this.props} />
    )
  }
}

import Example from './Example'

describe('Example', () => {
  let $container

  beforeEach(() => {
    console.log('beforeEach')
    $container = document.createElement('div')
    document.body.appendChild($container)
  })

  afterEach(() => {
    console.log('afterEach')
    ReactDOM.unmountComponentAtNode($container)
    $container.remove()
  })

  async function mount(props) {
    return new Promise((resolve, reject) => {
      const done = () => {
        console.log('done rendering')
        resolve()
      }
      ReactDOM.render(<ExampleWrapper {...props} />, $container, done)
    })
  }

  it('fails this spec', async () => {
    console.log('start test 1')
    await mount({})
    expect(true).to.be.true
  })

  it('also fails, but because of the first spec', async () => {
    console.log('start test 2')
    await mount({foo: {}})
    expect(true).to.be.true
  })
})

The spec output is below:

LOG LOG: 'beforeEach'
LOG LOG: 'start test 1'
LOG LOG: 'rendering!'
LOG LOG: '- rendering the real version'

  Example
    ✗ fails this spec
  Error: Uncaught TypeError: Cannot read property 'bar' of undefined (src/Example.spec.js:35380)
      at Object.invokeGuardedCallbackDev (src/Example.spec.js:16547:16)
      at invokeGuardedCallback (src/Example.spec.js:16600:31)
      at replayUnitOfWork (src/Example.spec.js:31930:5)
      at renderRoot (src/Example.spec.js:32733:11)
      at performWorkOnRoot (src/Example.spec.js:33572:7)
      at performWork (src/Example.spec.js:33480:7)
      at performSyncWork (src/Example.spec.js:33452:3)
      at requestWork (src/Example.spec.js:33340:5)
      at scheduleWork (src/Example.spec.js:33134:5)

ERROR LOG: 'The above error occurred in the <Example> component:
    in Example (created by ExampleWrapper)
    in ExampleWrapper

React will try to recreate this component tree from scratch using the error boundary you provided, ExampleWrapper.'
LOG LOG: 'there was a problem'
LOG LOG: 'done rendering'
LOG LOG: 'rendering!'
LOG LOG: '- rendering the error version'
LOG LOG: 'afterEach'
LOG LOG: 'beforeEach'
LOG LOG: 'start test 2'
LOG LOG: 'rendering!'
LOG LOG: '- rendering the real version'
LOG LOG: 'done rendering'
    ✓ also fails, but because of the first spec
    ✓ also fails, but because of the first spec
LOG LOG: 'afterEach'
LOG LOG: 'afterEach'

Chrome 69.0.3497 (Mac OS X 10.13.6): Executed 3 of 2 (1 FAILED) (0.014 secs / NaN secs)
TOTAL: 1 FAILED, 2 SUCCESS


1) fails this spec
     Example
     Error: Uncaught TypeError: Cannot read property 'bar' of undefined (src/Example.spec.js:35380)
    at Object.invokeGuardedCallbackDev (src/Example.spec.js:16547:16)
    at invokeGuardedCallback (src/Example.spec.js:16600:31)
    at replayUnitOfWork (src/Example.spec.js:31930:5)
    at renderRoot (src/Example.spec.js:32733:11)
    at performWorkOnRoot (src/Example.spec.js:33572:7)
    at performWork (src/Example.spec.js:33480:7)
    at performSyncWork (src/Example.spec.js:33452:3)
    at requestWork (src/Example.spec.js:33340:5)
    at scheduleWork (src/Example.spec.js:33134:5)

What is causing the duplicate reports?

Why does this happen in React 16+ and not in React 15?

How can I resolve this?

Answer 1

There may be race conditions because a promise is resolved with ref function. That component ref has been received doesn't mean that initial rendering has been completed.

As the reference states,

If you need a reference to the root ReactComponent instance, the preferred solution is to attach a callback ref to the root element.

A proper way to resolve a promise is to use render callback parameter,

If the optional callback is provided, it will be executed after the component is rendered or updated.

It should be:

async function mount(props) {
  return new Promise(resolve => {
    ReactDOM.render(<Example {...props} />, $container, resolve)
  })
}

The problem doesn't occur in second test, it occurs in first test regardless of whether there is second test and is not specific to React 16.5. It is specific to how React development mode works.

Here's a simplified demo that excludes Mocha factor. Expected errors are console.warn output but two Error: Cannot read property 'bar' of undefined errors are console.error which are being output by React itself. ReactDOM.render runs component render function twice and outputs an error from first test asynchronously.

The same demo with production build of React shows a single Error: Cannot read property 'bar' of undefined error synchronously as it would be expected. Failed render doesn't make ReactDOM render to reject, an error can be caught by error boundary component if needed:

class EB extends Component {
  componentDidCatch(err) {
    this.props.onCatch(err);
  }

  render() {
    return this.props.children;
  }
}

async function mount(props) {
  return new Promise((resolve, reject) => {
    ReactDOM.render(<EB onCatch={reject}><Example {...props} /></EB>, $container, resolve)
  })
}

It's a good practice to not rely on React DOM renderer in unit tests. Enzyme serves this purpose and allows to test components synchronously in isolation, shallow wrapper in particular.

Answer 2

It seems that React 16+ surfaces an uncaught error during render, even when using componentDidCatch in a wrapper. This means that Mocha will fail the test with the uncaught error, then continue with the next test, after which the second render of the component will complete and resolve the promise, running an assertion. This runs within the test currently in progress, causing the double success seen in this example.

An issue has been opened with the React repo on Github.