15

I have an AWS lambda RequestHandler class which is invoked directly by AWS. Eventually I need to get it working with Spring Boot because I need it to be able to retrieve data from Spring Cloud configuration server.

The problem is that the code works if I run it locally from my own dev environment but fails to inject config values when deployed on AWS.

@Configuration
@EnableAutoConfiguration
@ComponentScan("my.package")
public class MyClass implements com.amazonaws.services.lambda.runtime.RequestHandler<I, O> {
   public O handleRequest(I input, Context context) {
        ApplicationContext applicationContext = new SpringApplicationBuilder()
                .main(getClass())
                .showBanner(false)
                .web(false)
                .sources(getClass())
                .addCommandLineProperties(false)
                .build()
                .run();

        log.info(applicationContext.getBean(SomeConfigClass.class).foo);
        // prints cloud-injected value when running from local dev env
        //
        // prints "${path.to.value}" literal when running from AWS 
        //    even though Spring Boot starts successfully without errors
   }
}

@Configuration
public class SomeConfigClass {
   @Value("${path.to.value}")
   public String foo;
}

src/main/resources/bootstrap.yml:
spring:
  application:
    name: my_service
cloud:
  config:
    uri: http://my.server
    failFast: true
    profile: localdev

What have I tried:

  • using regular Spring MVC, but this doesn't have integration with @Value injection/Spring cloud.
  • using @PropertySource - but found out it doesn't support .yml files
  • verified to ensure the config server is serving requests to any IP address (there's no IP address filtering)
  • running curl to ensure the value is brought back
  • verified to ensure that .jar actually contains bootstrap.yml at jar root
  • verified to ensure that .jar actually contains Spring Boot classes. FWIW I'm using Maven shade plugin which packages the project into a fat .jar with all dependencies.

Note: AWS Lambda does not support environment variables and therefore I can not set anything like spring.application.name (neither as environment variable nor as -D parameter). Nor I can control the underlying classes which actually launch MyClass - this is completely transparent to the end user. I just package the jar and provide the entry point (class name), rest is taken care of.

Is there anything I could have missed? Any way I could debug this better?

Community
  • 1
  • 1
mindas
  • 26,463
  • 15
  • 97
  • 154
  • is your config server available from 'outside'? I mean, are you sure its not hidden behind ip protection or something similar? – hi_my_name_is Mar 18 '16 at 11:11
  • @freakman good point, but no - just checked, the config request is served from any host. – mindas Mar 18 '16 at 11:23
  • I am assuming you have looked at this: http://cloud.spring.io/spring-cloud-aws/spring-cloud-aws.html#_spring_boot_auto_configuration and it is not working for you? I don't really know much about AWS Lambda but saw this post: https://github.com/cagataygurturk/aws-lambda-java-boilerplate. Maybe you already checked these but wanted to confirm. – Rob Baily Mar 21 '16 at 14:05
  • @RobBaily thanks, aws-java-lambda-boilerplate is using classic Spring .xml app context approach (see `AbstractHandler.java:58`) which works, but as I mentioned in my post, it doesn't have integration with with `@Value` injection/Spring cloud (whereas Spring Boot does). Re: documentation - yes, I'm using `spring-cloud-config` artifactId in my Maven cfg, unfortunately there's no mention of lambda in the documentation so not much I can pick up from there. – mindas Mar 21 '16 at 14:33
  • @mindas Have you tried setting a log level locally for Spring Cloud classes where you can see what it is doing? If so then you may be able to set the same logging levels for your Lambda deployment to see what is different. Not sure how much logging is available but you could also clone it and add your own log messages. – Rob Baily Mar 21 '16 at 21:35
  • Also maybe you could create a different request handler which just does the HTTP connection to your cloud config server and return that information to confirm you do not have any connectivity issues. I think I am going to experiment on my own a little to see what I can see. I am curious if Spring Boot is a good fit for this architecture as it seems like the overhead in loading Spring dependencies for every request may make it too slow to be really usable for anything that requires relatively fast responses. – Rob Baily Mar 21 '16 at 21:51
  • @RobBaily I have enabled debug logging for everything but unfortunately not much is available. Also I have done some debugging myself (while running on local instance) but the rabbit hole is a bit too deep... I'm not too much worried about the response time, as long as the request is served within max interval (5 minutes). – mindas Mar 22 '16 at 08:53
  • @mindas Is Spring Cloud the only reason you need Spring Boot? It looks like we could make a thin Spring Cloud client that could be used if that is all you need it for. I'm going to play with Lambda here a bit and try some things out and see if I can see what is going on. – Rob Baily Mar 22 '16 at 19:48
  • Here is the author of aws-lambda-java-boilerplate. I am currently working on a full jax-rs implementation project for aws lambda (not documented version: https://github.com/lambadaframework/lambadaframework) and for injecting configuration values i'll use a properties file in a s3 bucket that'll populate system properties on instance creation. Do you think is it a good method? I think using Spring boot just for injecting properties is a overkill for lambda. – Cagatay Gurturk Mar 30 '16 at 10:48
  • @ÇağatayGürtürk thanks for reaching out! I think either option is good. Unfortunately I had to use Spring Boot in order for Spring Cloud to work, as we use Spring cloud for configuration. I could, of course, take the plunge and write a parser for JSON config and use "classic" Spring (and indeed this was the plan B), but given current circumstances I was able to avoid it. The world is better without yet another JSON parser :-) So why reinvent the wheel, especially since lambdas are kept warm by AWS? And thanks for `aws-lambda-java-boilerplate` - it gave me a good head start! – mindas Mar 30 '16 at 15:23
  • I released the very first version of https://github.com/lambadaframework/lambadaframework you can take a look. – Cagatay Gurturk Mar 31 '16 at 17:57

1 Answers1

24

After a bit of debugging I have determined that the issue is with using the Maven Shade plugin. Spring Boot looks in its autoconfigure jar for a META-INF/spring.factories jar see here for some information on this. In order to package a Spring Boot jar correctly you need to use the Spring Boot Maven Plugin and set it up to run during the maven repackage phase. The reason it works in your local IDE is because you are not running the Shade packaged jar. They do some special magic in their plugin to get things in the right spot that the Shade plugin is unaware of.

I was able to create some sample code that initially was not injecting values but works now that I used the correct plugin. See this GitHub repo to check out what I have done.

I did not connect it with Spring Cloud but now that the rest of the Spring Boot injection is working I think it should be straightforward.

As I mentioned in the comments you may want to consider just a simple REST call to get the cloud configuration and inject it yourself to save on the overhead of loading a Spring application with every request.

UPDATE: For Spring Boot 1.4.x you must provide this configuration in the Spring Boot plugin:

            <configuration>
                <layout>MODULE</layout>
            </configuration>

If you do not then by default the new behavior of the plugin is to put all of your jars under BOOT-INF as the intent is for the jar to be executable and have the bootstrap process load it. I found this out while addressing adding a warning for the situation that was encountered here. See https://github.com/spring-projects/spring-boot/issues/5465 for reference.

Rob Baily
  • 2,770
  • 17
  • 26
  • `Spring Boot Maven Plugin` was the missing element. I had to tweak the settings a bit but the whole combination worked. Interesting fact, the bootstrap took 10 seconds (!) according to logs - this is definitely a factor which discourages using Spring Boot in AWS environment. Nevertheless, thanks for your hard work, much appreciated! – mindas Mar 23 '16 at 08:52
  • @mindas Yes I think Spring Boot may make more sense in one of their container environments rather than Lambda. I am also going to see about putting in a request for some type of warning when this happens as it was definitely not obvious and could happen in other situation. – Rob Baily Mar 23 '16 at 13:30
  • 1
    The good news is that Amazon is keeping the Lambda instance alive for some (undefined?) time before discarding it, so the next request can be served using the same instance. This means that code should check if application context has already been initialized and skip it if so. That said, for frequent requests the problem of overhead is irrelevant. – mindas Mar 23 '16 at 14:07
  • 1
    I got this warning for 1.5.7.RELEASE: `Module layout is deprecated. Please use a custom LayoutFactory instead.` – luboskrnac Oct 13 '17 at 15:02
  • I don't much about this but it looks like it is reference here https://docs.spring.io/spring-boot/docs/1.5.x-SNAPSHOT/reference/htmlsingle/#build-tool-plugins-gradle-configuration-custom-repackager You may want to post a separate question if you are having trouble with a new Spring Boot version. – Rob Baily Oct 13 '17 at 19:01
  • did anyone resolve this issue with spring boot 2+? – Hardik Uchdadiya Jun 19 '22 at 12:40