Why are Karma tests failing after moving to Webpack 5?

Question

I have a set of karma tests which were passing with webpack v4.46.0 but they fail because of spyOn after moving to webpack v5.0.0.

A lot of tests spy on a module as follows:

import Module from '...';

...

beforeEach(() => {
    spyOn(Module, 'getSomething').and.returnValue(true);
});

This was working fine, but moving to Webpack v5.0.0 (with karma-webpack v5.0.0) causes the following error:

Error: <spyOn> : getSomething has already been spied upon

Has some behavior changed, either in how import works or in how spyOn works, in the new version? I'm at a loss for why this would be breaking.