Spring boot @SpyBean causing test suite to error probably due to issue with context not being reset

Question

I have an issue with a spring integration test.

The behavior:

When I run the test below in isolation, it is in success.
However, when all tests are run, many of them including the one below are in error.
When I ignore the test below and run all test, all are in success.

I haven't included the error stacktrace because it is highly related to our business logic and I suspect the error is related to my usage of spring boot test @SpyBean.

Here is the test:

@RunWith(SpringRunner.class)
@ActiveProfiles(profiles = "test")
@SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT)

...

@Autowired
private TestRestTemplate restTemplate;

@Autowired
private DataKeyStore dataKeyStore;

@SpyBean
private TokenTools tokenTools;

...

@Test
public void myTest() throws Exception {

   doReturn("someGeneratedToken")
        .doReturn("someGeneratedToken")
        .doCallRealMethod()
        .when(tokenTools)
        .createToken(any(TokenProfile.class), anyString(), anyString());
  ...

Please note that DataKeyStore is a dependency of TokenTools.

As I said above, I suspect tests are stepping on each other and my @SpyBean somehow leaks on other test classes...

My question is how can I make sure this test does not step on the other tests? I have tried the @DirtiesContext annotation to no avail...

Also what puzzles me is that the @SpyBean is already reset (as per the documentation/javadoc).

Can anyone please help?

edit: Using my IDE to debug the tests indicates that TokenTools is instantiated only twice for all tests: once at the initialization of tests and a second time for creating the @SpyBean for the test above. The remainder of tests run after the test above use the second instance i.e. the @SpyBean instance...

It turned out some of the `@SpyBean`'s dependencies changed global state upon instantiation causing a issues in later tests. No worries with `@DirtiesContext` therefore, but rather with our application design... — balteo, Mar 24 '17 at 12:51

score 4 · Answer 1 · answered Jul 31 '19 at 14:24

I recently ran into the same issue. Make sure to set the right classMode for your @DirtiesContext annotation.

By default, @DirtiesContext will reset the @SpyBean after the complete test class. You probably want to reset it before or after each test method.

So add @DirtiesContext(classMode = DirtiesContext.ClassMode.BEFORE_EACH_TEST_METHOD) or @DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_EACH_TEST_METHOD) to your test class.

score 1 · Answer 2 · answered Mar 23 '17 at 14:51

I can confirm that @DirtiesContext didn't work for us as well. We had problems initialize DB (using Liquibase) for new context after old context was closed (by @DirtiesContext annotation).

We ended up naming Spring test context differently for tests that are faking some bens:

E.g.:

@RunWith(SpringRunner.class)
@SpringBootTest
@ContextConfiguration(classes = SpringBootApp.class, name = "mainContext")
public class TestThatDoesntFakeBeans(){
}

@RunWith(SpringRunner.class)
@SpringBootTest
@ContextConfiguration(classes = SpringBootApp.class, name = "contextWithFakeBean")
public class TestThatFakeBeans(){
    @SpyBean
    //...
}

This way there is separate Spring context created for each name. Contexts with same name are reused by tests. But of course you need to make sure that tests with same context name doesn't affect each other.

score 0 · Answer 3 · answered Jun 12 '22 at 02:20

@SpyBean seems to not be reset after each test which leads to unusual behavior. I would suggest using Mockito @Spy instead and check if the problem still persists.

import org.mockito.Spy

....

@RunWith(SpringRunner.class)
@ActiveProfiles(profiles = "test")
@SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT)

...

@Autowired
private TestRestTemplate restTemplate;

@Autowired
private DataKeyStore dataKeyStore;

@Spy
private TokenTools tokenTools;

...

@Test
public void myTest() throws Exception {

   doReturn("someGeneratedToken")
        .doReturn("someGeneratedToken")
        .doCallRealMethod()
        .when(tokenTools)
        .createToken(any(TokenProfile.class), anyString(), anyString());
  ...

Spring boot @SpyBean causing test suite to error probably due to issue with context not being reset

3 Answers3