7

I have a mobile app where the back-end is currently running as a NodeJS Cloud Function, but I'm nowhere near as comfortable with NodeJS as I am with Java. So, I've re-written the API in Java - however, when it comes to deploying that as a Cloud Function or on Cloud Run the cold-start performance is obviously not very good. I'm seeing roundabouts 15 second cold-start time when I add in the dependencies that I need, which is not going to work. I do have a "warmup" endpoint that I call immediately when a user logs into the mobile app to kick off the initialization of the API back-end, which does help a little.

I've been playing around with GraalVM and generating a native image for a while as well, and while I can get your basic hello-world app and some slightly more elaborate examples working, my app has some dependencies like gRPC and Cloud Firestore, among others, and I have not been successful in generating a native image for that with Micronaut, Quarkus, or Spring Boot.

I considered running on a managed instance group with a minimum of 1 so there's always at least one instance up and running, ready to serve requests, but I would then need a Cloud Loadbalancer in front and I've read some horror stories where the Cloud Loadbalancer wound up costing folks a lot more than they had expected.

Is there a way to front a managed instance group using Cloud Endpoints? I see where you can do it with a single VM instance, but not across a group which leads me to believe that in that case I would need a Cloud Loadbalancer to do what I need?

Cost-effectiveness is important, because my app is super new and is not generating any revenue at all yet, and since it's just me funding it using personal money, my infrastructure budget is not super high :)

TL;DR/ Looking for tips on what the cheapest way would be to host a Java-based API app on a framework like Micronaut, Quarkus, or Spring Boot on GCP while maintaining good performance and elasticity.

Any insight would be greatly appreciated.

cloudwalker
  • 2,346
  • 1
  • 31
  • 69

2 Answers2

3

I wrote an article on Java framework cold start on Cloud Run (the results are outdated because after this article release and discussions with Googlers, the team has updated the Cloud Run platform and the way to manage Java containers. Now they start quickly now!)

Anyway, your question seems relevant at the beginning, but finally not really. I will explain why.

Firstly, the cold start is a temporary issue. Your first request is slow, and the dozens, hundreds after are very fast. Does it really a problem?

If so, the min instance feature (only available on CLoud Run for Anthos for now) is coming in the managed version. Like this you never really scale to 0, an instance is kept warm and start instantly (but, as counterpart, it won't be free).

Secondly, if you look for maintainability, I recommend you the framework that you know. You will be more efficient to improve your code, fix your issue and to save your time (and time is money) much more that infrastructure consideration!

All the Java framework are relatively close when optimized (Naive Spring Boot on Cloud Run start in 20s, in 2s after packaging optimizations!). Of course, native compilation (with GraalVM) is the fastest, but it's not really stable for now with several side effect (and I won't recommend it for production).

Personal opinion: I'm a big fan of Spring Boot and its ecosystem. But Micronaut and its AOT compilation, in addition of annotation compliant with Spring Boot idioms, is absolutely awesome. Quarkus is more recent, and I haven't real opinion on it (never used in production/real project)

guillaume blaquiere
  • 66,369
  • 2
  • 47
  • 76
  • I actually have your article bookmarked and have referenced it heavily - it was super helpful! Ideally I would love to use a native image but, like you mentioned, I've found the native image compilation to not be super stable at the moment. I've tried Spring Boot, Micronaut and Quarkus without native image compilation and with cold-start times of about 15s or so it's just that first request after no one has used the app for a while that is really slow, but since my mobile app is very new, it has periods of inactivity, and I'm worried that cold-start lag could deter a new user. – cloudwalker Aug 11 '20 at 00:17
0

I would say you need more Micronaut or Quarkus in combination with GraalVM if you target performance. Define your services to be run as

My experience is primary with Micornaut serverless application and it is manageable to have api service running as function/lambda with boot time of 100-500 ms. Cold starts are not a big issue anymore if you enable provisioning (feature is available since 12.2019 in AWS), you could skip the so called warming.

How to make your lambda faster ?

  1. Keep your package size as small a possible (remove all big libraries where a fraction of it is used) - keep package size to max 20 MB. On every cold start this package is fetched and decompressed.
  2. If you use a JVM technology for your services, try to migrate them to Graalvm where the boot-up overhead is reduced to minimum.
    • micronaut + graalvm
    • quarkus + graalvm
    • helidon + graalvm
  3. Use cloud infrastructure configs to reduce the cold starts. This is what AWS provides, not sure about GPC

https://aws.amazon.com/about-aws/whats-new/2019/12/aws-lambda-announces-provisioned-concurrency/

Note: IMHO AWS has a better setup for serverless application so far compared to GCP in terms of boot-up and cold starts.

Traycho Ivanov
  • 2,887
  • 14
  • 24