1

I'm trying to build a NodeJS REST API project based on the so called "micro architecture" (basically multiple smaller NodeJS projects that can run totally independent, but at the same time work together).

Currently users are able to upload images from the app, and my NodeJS backend then processes and saves them appropriately.

Now, what I want to do is the following:

User selects an image to upload from the app -> The app makes a request to the "Main API" endpoint -> The Main API endpoint then forwards this request to the "Image Service" -> Once the Image Service (which is a totally different server) has successfully finished, it should return the URL where the image is stored to the Main API server endpoint, which will then return the info back to the app.

My question is, how do I forward the image upload request from one server to another? Ideally, I don't want the Main API to store the image temporarily and then make a request to the Image Service.

What I'd like is try and forward the data the Main API receives straight to the Image Service server. I guess you could say I want to "stream" the data from one place to another without having to temporarily store on disk or memory. I literally just want it to "tunnel" from one server to another.

Is this possible and is this an efficient way? I just want 1 central point for the app to access, I don't want it to know about this Image Service server. I'd like the app to only ever make requests to the Main API, which will then call my other little services as required.

I'm using NodeJS, Express, Multer (for image uploads) and Digital Ocean hosting (if that should make any difference at all).

Jay Y
  • 48
  • 6

1 Answers1

1

What you would basically be doing is setting up a proxy server that will pass requests straight through to another machine and back. There are a few libraries out there to help with this, and this article in particular http://blog.vanamco.com/proxy-requests-in-node-js/ will explain how to go about setting it up even though they are really just trying to get around HTTPS, the same concept applies here.

In short, you get the file upload POST, and then immediately just make that same request to another server and when the response is returned, immediately return it back to the front end. Your entry point can be set up as a hub, and you can proxy requests through to other servers or even just handle them on the same server if necessary.

Scottux
  • 1,577
  • 1
  • 11
  • 22
  • Would this also work if "Main API" needs to do some additional logic (for example save the returned link from the "Image Service" in a database against the current user)? – Jay Y Dec 20 '15 at 18:19
  • in this case yes, because you are able to capture the return response from the upload service. so your main endpoint gets a request, takes that and makes a new request to a different server using the same request data, then when you get the response back you can log it or save it or transform it, right before returning the response back to the original request. – Scottux Dec 21 '15 at 14:32
  • Sorry for the stupid question, but if I proxy the request, how can I still catch the response? Wouldn't it automatically just return once the initial operation is completed by the ImageService? – Jay Y Dec 22 '15 at 10:12
  • It doesn't have to be a completely transparent proxy, what you would have is `Client --- (POST {imageStuff:'...'}) ---> Main --- (POST {imageStuff:'...'}) ---> ImageUploader --- (Save file data to disk, respond {margin: 'success', fileData:{imageStuff:'...'}}) ---> Main --- (Save message / name / data to DB, respond {margin: 'success', fileData:{imageStuff:'...'}}) ---> Client` I hope that makes sense as for a flow of things. – Scottux Dec 22 '15 at 14:31
  • So if I understand correctly, I still need to temporarily save the image on the "Main API" server, before forwarding the request to the "Image Service" server, correct? – Jay Y Dec 23 '15 at 10:27
  • Not at all, the upload data will stay in memory until you specifically save it to the file system. – Scottux Dec 23 '15 at 15:40
  • 1
    I ended up using this module here for proxying requests: https://github.com/nodejitsu/node-http-proxy Thanks for your help Scottux! :) – Jay Y Dec 26 '15 at 20:12