Better performance to consolidate Spring configuration classes?

Question

I have a basic test setup with TestA dependent on ConfigA and TestB dependent on ConfigB.

@Configuration
public class ConfigA {

    // define A beans

}

@Configuration
public class ConfigB {

    // define B beans

}

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes = { ConfigA.class })
public class TestA {

    @Test
    public void testA() {
        // test with A beans
    }

}

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes = { ConfigB.class })
public class TestB {

    @Test
    public void testB() {
        // test with B beans
    }

}

I want to run both TestA and TestB using test suite AllTests.

@RunWith(Suite.class)
@SuiteClasses({ TestA.class, TestB.class })
public class AllTests {

}

As it stands, running AllTests will force Spring to load both ConfigA and ConfigB at runtime.

Would it be better for performance to consolidate ConfigA and ConfigB into ConfigC and have both tests use ConfigC instead?

@Configuration
public class ConfigC {

    // define A and B beans

}

score 2 · Accepted Answer · edited Jun 20 '20 at 09:12

It should be more or less the same, performance-wise - Spring test caches the contexts either way so if any of your other tests use either ContextA or ContextB they will reused from cache instead of being recreated for each test. You would have a bit more overhead with two separate contexts (A and B) than with the consolidated one but it shouldn't be that noticeable.

Here's a quote from the official documentation:

The Spring TestContext Framework provides consistent loading of Spring ApplicationContexts and WebApplicationContexts as well as caching of those contexts. Support for the caching of loaded contexts is important, because startup time can become an issue — not because of the overhead of Spring itself, but because the objects instantiated by the Spring container take time to instantiate. For example, a project with 50 to 100 Hibernate mapping files might take 10 to 20 seconds to load the mapping files, and incurring that cost before running every test in every test fixture leads to slower overall test runs that reduce developer productivity.

By default, once loaded, the configured ApplicationContext is reused for each test. Thus the setup cost is incurred only once per test suite, and subsequent test execution is much faster. In this context, the term test suite means all tests run in the same JVM — for example, all tests run from an Ant, Maven, or Gradle build for a given project or module.

Here are some more notable quotes from the official documentation concerning context caching:

The Spring TestContext framework stores application contexts in a static cache. This means that the context is literally stored in a static variable. In other words, if tests execute in separate processes the static cache will be cleared between each test execution, and this will effectively disable the caching mechanism.

To benefit from the caching mechanism, all tests must run within the same process or test suite. This can be achieved by executing all tests as a group within an IDE. Similarly, when executing tests with a build framework such as Ant, Maven, or Gradle it is important to make sure that the build framework does not fork between tests. For example, if the forkMode for the Maven Surefire plug-in is set to always or pertest, the TestContext framework will not be able to cache application contexts between test classes and the build process will run significantly slower as a result.

Since Spring Framework 4.3, the size of the context cache is bounded with a default maximum size of 32. Whenever the maximum size is reached, a least recently used (LRU) eviction policy is used to evict and close stale contexts. The maximum size can be configured from the command line or a build script by setting a JVM system property named spring.test.context.cache.maxSize. As an alternative, the same property can be set programmatically via the SpringProperties API.

Is there any documentation to support this? The reason for my original question is I have many test classes (~200) with many configs (~100). Running the maven build takes considerable time. — Zack, Aug 10 '16 at 19:34
Ah, that changes things a lot - context cache is limited to 32 I think, but let me double-check — Miloš Milivojević, Aug 10 '16 at 20:29
Yes, as of Spring 4.3, the default context cache size is 32. I've updated my answer with relevant documentation links. You'll see that the first quote hints that Spring's own overhead is relatively small, meaning that the biggest overhead comes from the beans themselves. With that in mind, if your configurations have some common beans, it would definitely make sense to merge the configurations. Also, pay attention to surefire settings. — Miloš Milivojević, Aug 10 '16 at 20:45
If I understand this correctly, in order to run my testing more efficiently during build, I should increase the context cache in the JVM params. It seems like I'm hitting the cache cap under the default settings and reloading the context files rather frequently during the test phase of my build. — Zack, Aug 10 '16 at 21:01
So the only hit is when I load the context, once it is loaded it doesn't matter if the objects all came from 10 configs or 1, they're all in the same cache. — Zack, Aug 10 '16 at 21:03
Yeah, you're definitely hitting the cap if you're using Spring 4.3.x. As far as performance hits are concerned, yes, the only hit is the initial context load. But it's also worth noting that if you have beans that are present in several configurations, those beans will be instantiated several times - once per context - and will also take up more memory — Miloš Milivojević, Aug 10 '16 at 21:10
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/120737/discussion-between-zack-teater-and-milos-milivojevic). — Zack, Aug 11 '16 at 19:30

Better performance to consolidate Spring configuration classes?

1 Answers1