1

I am familiar with firebase platform, but I am relatively a new user of the google cloud platform as whole. I am working on a project built using a microservices structure, and I do have so many question for which I cannot find an answer or better I cannot find any example. Unfortunately all the example that I am able to find are way to simple to be able to extrapolate a viable answer for my issues.

I adopted the new cloud run offer, and I decided to play with the full managed version (not kubernetes). I built few microservices (each service is built using express for node or flask for python - depending on what the services does). Each microservices expose it's own endpoint and has it's own api to call the methods - and I use a service account to allow the application to perform the internal calls.

I now want to expose the application to the external (specifically to my client built using vuejs technology), and I was trying to leverage another google product to create and expose an api: the google endpoints.

My question (specifically referred to the cloud run structure) is related to how is possible and what I need to do to create an api endpoints to communicate with the client app, that internally calls multiple services and combine their response in one.

Just to be clear, let's make an example:

  • Cloud run service 1 -> crud user api

  • Cloud run service 2 -> crud product api

  • Cloud endpoint external visible api -> get user from service 1, and after get products from service 2 and return the combined response all green products for user Jane Doe.

How I can aggregate the response directly in the endpoint gateway, check for failure and if everything goes smooth send the aggregate response to the client?

I need to build the aggregate endpoint in something else, like a cloud function for example? or I can do it directly in the google endpoints gateway?

Note that for cloud run the google endpoints is another cloud run container.

Thanks guys for some help, running pretty much out of option here.

Dinuz
  • 400
  • 4
  • 13

3 Answers3

4

As per my understanding, API Gateway should just work as a proxy, presenting all micro services as a single endpoint. To this scenarios I think you can have following 2 approaches :

1: Implement a new micro service (or on any of the existing one) which will do invocations and aggregation of responses.

2: Client(like UI) can invoke the services and do the aggregation on their side as well.

I feel, it is not a good idea to do it at api-gateway.

Manmay
  • 835
  • 7
  • 11
  • Manmay that is what I thought, but I was hoping to find something at the gateway level, and do the aggregation directly in the gateway. I would prefer to keep the whole structure independent (trying to preserve the full independence of the microservices), and your answer number 1 just defeat this. The problem with the answer number 2 instead is that in order to do aggregation on the client side, I will be forced to make multiple calls (the purpose of the gateway is to avoid the multiple calls). I know it's possible what I am looking for, I just don't know if it is possible in google endpoints. – Dinuz Oct 14 '19 at 13:53
  • As long as it works for you it sud be fine. But when in future you will have to combine/orchestrate multiple simple services like this, all need to be in gateway level and there you may break the single responsibility principle as doing a good part of business logic /orchestration logic in gateway. May result in gateway to be splitted in case of more complexity. Just a thought. – Manmay Oct 14 '19 at 13:58
  • 1
    I can suggest to go with : N Core Services With Own Domain (DB) & 1+ orchestration/combining micro services to expose services which need invocation to multiple core services (not maintaining own data) and a gateway api over it. But if your orchestration is simple & of very small amount, you can go with putting it in gateway level. But put them in gateway in such a way that it can be migrated to be a new orchestration microservice easily in future, if the complexity or volume grows. – Manmay Oct 14 '19 at 14:06
  • Manmay that is a great point! Check the last answer in this post by allan chua, you can esily do what I am looking for in ocelot, I don't see how I can apply the same idea in google endpoints (unfortunately I cannot find any example for google): https://stackoverflow.com/questions/52472261/how-to-merge-consolidate-responses-from-multiple-restful-microservices – Dinuz Oct 14 '19 at 14:07
3

In my opinion, from an architectural point of view, the best option for you is to create a new microservice which will take the responses from the other two and then, it will aggregate them.

I understand that you want to aggregate the responses in a api-geteway and you are not able to find code examples for it. Here I was able to find a guide on what are you wanting to implement. The full code implementation can be found in this repository.

Keep in mind though, this idea of implementation is not a best practice.

Andrei Tigau
  • 2,010
  • 1
  • 6
  • 17
  • Andrei thank you for the answer, and for the tutorial and example. – Dinuz Oct 17 '19 at 12:23
  • However, it's not correct that I am not able to find example for making a gateway, what I am not able to find is an example of what I want to realize using google endpoints. What you are suggesting is correct (create a new service to aggregate responses), but it has potentially 2 big issue: – Dinuz Oct 17 '19 at 12:32
  • (1) if the service is made to aggregate only one specific response, it is just overkilling; (2) if the service is made to be an aggregation service (and aggregate different responses in different endpoints) it becomes a coupling issue in the system (you don't have anymore independent services, they are coupled in the aggregator), and this defeat the microservices idea. – Dinuz Oct 17 '19 at 12:32
  • For these reasons, I was looking in doing it directly in the gateway (google endpoints) or to do something similar to what you suggested but using a function. What I want is having a microservices system completely independent and decoupled, and aggregate responses in a way that would allow me to just unplug an endpoint without touching the whole system (eg delete the function). Make sense? – Dinuz Oct 17 '19 at 12:33
  • I understand your point of view, Dinuz. But you still need to have some kind of coupling because of the logic of your workflow itself ( the term "aggregation" will involve some level of coupling by itself). It is true that you should maintain that coupling as light as possible. – Andrei Tigau Oct 17 '19 at 12:51
  • 1
    As about that tutorial it implements the exact logic that you want to achieve : "fetches data from several services and aggregates it to return a single rich response. Depending on the API consumer, data representation may change according to the needs". Yes, it is true that it is not made specifically for Google Cloud Endpoints, but that's just a matter of configuration. You can take the heart of the code from that example and adapt it with the configuration for Google Cloud Endpoints : https://cloud.google.com/endpoints/docs/openapi/get-started-cloud-run – Andrei Tigau Oct 17 '19 at 12:54
  • Andrei, I think your answer is the most complete and probably correct. What do you think if instead of creating a "new" service, I will implement the aggregation in a serverless function. This will allow me to have a coupling point limited to a function, and will allow me to unplug it easily if I need. Clearly, I would need to create multiple functions (all independent from each other) one for each coupling endpoint. The gateway endpoint, will then just be the port of entry from the outside world, and it will take care of authentication and authorization. Is this right? – Dinuz Oct 23 '19 at 23:23
  • Andrei, what I loved to do is something similar to what is done in this post https://www.pogsdotnet.com/2018/09/api-gateway-response-aggregation-with.html are you telling me that something like this is not possible in google cloud, specifically in cloud endpoints? – Dinuz Oct 23 '19 at 23:30
  • 1
    That is a good idea and I think it will work smoothly if you are aware with the time/rate/size quotas for Cloud Functions - https://cloud.google.com/functions/quotas. I am not saying that, I just say that you can do it exactly that way, since you do not Ocelot for GCP, you have Cloud Endpoints and the most similar example that I was able to find was the one in the repository from my answer – Andrei Tigau Oct 24 '19 at 09:33
-1

This is ok, only if those two services that are going to be combined are independent. Meaning there is no functional/business relation between them and the concurrency or inconsistency problem will not occur in the process of aggregating.

MikoLunar
  • 1