SpringBootTest - how to assert context not loads and change properties at test level?

Question

I rely on @SpringBootTest heavily when testing application configuration. App properties can be complex, having default values and non-trivial validations. For example:

prop:
  ports: 1205,2303,4039
  fqdn: ${server.host}:${ports[0]}/${path}

@Configuration
SomeConfigClass{
   @Value{"${ports}:{#collections.emptyList}"}
   List<Integer> ports;

   ...
}

When testing such apps, I bring up a full or partial application context without mocks, as there is complexity in the context and validation itself - mocks don't capture this. Unfortunately, there are two limitations I keep finding with this pattern:

1. How can we test that bad configurations fail to load?

   Imagine testing that the port in invalid because it is not on the restricted range of 500 - 1500.

   @SpringBootTest(
           classes = {SomeConfigClass.class},
           properties = "port=9000"
   )
   public class BadConfigTest{
   
       @Test(expected = ApplicationContextFailedException.class)
       public void WHEN_port_9000_THEN_appcontext_fails_to_load() {}
   
   }
   

   Since the test framework loads after the application context, there appears to be no way to test that an app context fails to load. For now I actually write the tests, manually confirm they fail, and then annotation with @Ignored so they are not lost.

2. How to change properties at the test method, rather than class, level?

   @SpringBootTest is a class annotation, meaning application properties are bound at the test-class level. This results in needing a test class for many sets of properties and bloats the test suite. For example, I'll end up with test classes like:

   ConfigPropertiesAllValidTest
   ConfigPropertiesNoneSetTest
   ConfigPropertiesSomeValidSomeNotTest
   

   Where each of these only has one or two test cases. Preferably, there'd be a single ConfigPropertiesTest class with different props for each test. Is this possible?

Again - I want to avoid mocks as they don't capture the non-trivial context autoconfiguration performed by Spring at runtime.

Answer 1

We ended up using the ApplicationContextRunner described in this document:

https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-developing-auto-configuration.html#boot-features-test-autoconfig

Answer 2

You can use the @ActiveProfiles annotation along with the @SpringBootTest annotation to load properties for different profiles. This is class level, so will only help for case 1 of your question.

@SpringBootTest(classes = {SomeConfigClass.class})
@ActiveProfiles("badconfigtest")
public class BadConfigTest{
    ...
}

Then have an application-badconfigtest.properties with your bad config.

I don't think you'll find a way of changing properties between test methods in the same class. You can use @DirtiesContext to reload the application context, but I've not seen a way to use different property files. I guess you could inject the values into the config classes that have already loaded the properties.