Difference between webEnvironment = RANDOM_PORT and webEnvironment = MOCK

Question

I wrote spring boot integration test and it is working. Here is the test config:

@RunWith(SpringRunner.class)
@SpringBootTest(webEnvironment = RANDOM_PORT)
@AutoConfigureMockMvc
@Transactional
public class SomeTest {
   @Autowired
   private MockMvc mvc;

   @Test
   public void insertEventTest(){
      ...testing something...
   }

}

I understand that when setting webEnvironment = RANDOM_PORT spring will initialize an embedded web server and run this test against that web server. I take a look at logs when running this test and saw that embedded TomcatWebServer was started. It takes about 6 seconds to initialize Tomcat but between those two parts of the logs few other beans were initialized so I am pretty sure that initializing Tomcat was not 6 seconds but less than 6 seconds. One part of the logs:

2019-10-13 16:03:20.065  INFO 8596 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 0 (http)
2019-10-13 16:03:20.098  INFO 8596 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2019-10-13 16:03:20.098  INFO 8596 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.14]
2019-10-13 16:03:20.108  INFO 8596 --- [           main] o.a.catalina.core.AprLifecycleListener   : The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: [/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib]
2019-10-13 16:03:20.228  INFO 8596 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext

...some more logs and then finally

  2019-10-13 16:03:26.366  INFO 8596 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 38335 (http) with context path ''

I run test 3 times and it takes 12 ,11.4 and 12 seconds for test to complete. After that, I tried to set @SpringBootTest(webEnvironment = MOCK) . I noticed that this time Tomcat was not initialized(web server was mocked by spring). Execution times were 11.3, 11 and 10.8 seconds. In both cases, all tests were green. My thoughts were that I will improve performance of my tests with mocked web server but what I got is 1 second. If we have in mind that my application context is cached between test classes, I basically got nothing. So my question is, in which cases test will pass with @SpringBootTest(webEnvironment = RANDOM_PORT) and fail with @SpringBootTest(webEnvironment = MOCK) or vice versa and when I should use RANDOM_PORT and when MOCK ?

Answer 1

Using @SpringBootTest(webEnvironment = WebEnvironment.MOCK) loads a web application context and provides a mock web environment. It doesn’t load a real http server, just mocks the entire web server behavior.

WebEnvironment.MOCK gives you some advantages like ease of use or isolation of other factors but it might not be a good integration test practice.

Integration tests should be as similar as possible to the production environment. Considering this, using @SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT) would be a better choice. This approach is closer to test the real application. You can see whether the whole system is going to work as expected.

When you use @SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT) you test with a real http server. In this case, you need to use a TestRestTemplate. This is helpful when you want to test some surrounding behavior related to the web layer.

TestRestTemplate is a convenience alternative to Spring’s RestTemplate that is useful in integration tests. ... if you use the @SpringBootTest annotation with WebEnvironment.RANDOM_PORT or WebEnvironment.DEFINED_PORT, you can inject a fully configured TestRestTemplate... https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-testing.html#boot-features-rest-templates-test-utility

Answer 2

There are some cases in which MOCK is incapable of catching failures while RANDOM_PORT, as it starts a real web server, is.

The example is in the docs:

For example, Spring Boot’s error handling is based on the “error page” support provided by the Servlet container. This means that, whilst you can test your MVC layer throws and handles exceptions as expected, you cannot directly test that a specific custom error page is rendered. If you need to test these lower-level concerns, you can start a fully running server as described in the next section.

Answer 3

1. WebEnvironment.DEFINED_PORT - DEFINED_PORT in application-fixed.properties for the embedded web server. You can use RestTemplate or TestRestTemplate to test application. Creates a (reactive) web application context.

                application-fixed.properties 
                server.port=7777
            
            @RunWith(SpringRunner.class)
            @SpringBootTest(classes = Application.class,
              webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
            @ActiveProfiles("fixed")
            public class ApplicationTest {
                private final static int EXPECTED_PORT = 7777;
                ....
            }

2. WebEnvironment.RANDOM_PORT - Don't want to specify any port for testing the springboot. You can use RestTemplate or TestRestTemplate to test application. Creates a web application context (reactive or servlet based).

                application-random.properties 
                server.port=0 - ServletWebServerApplicationContext.getWebServer().getPort() to get embedded server port. 

3. WebEnvironment.MOCK - @SpringBootTest without parameters, or with WebEnvironment= WebEnvironment.MOCK, doesn't load a real HTTP server. You can not use RestTemplate or TestRestTemplate to test application. Have to use @MockMVC.

Creates a WebApplicationContext with a mock servlet environment if servlet APIs are on the classpath.