1

Brothers and sisters, I am building an Express API Endpoint that needs to consume an external API, perform some changing of keys and values, and return to the result to the client. Here is what I have thus far:

const external_endpoint = <external_api_end_point>;

app.get('/', function (req, res, next) {

  request({ url: external_endpoint}).pipe(res);
});

This returns the exact payload you would get from hitting the external_endpoint directly.

Isn't there something I can do to change res before it gets sent to the client? I tried a few things but nothings has worked. Any ideas or best practices associated with doing a transform on the incoming payload?

For the sake of simplicity. Lets say this is the payload obj.json:

{
    "sad": {
        "userid": 5,
        "username": "jsmith",
        "isAdmin": true
    }
}

and I am wanting to change sad to happy.

I know outside of the request I could do something like this:

obj = JSON.parse(JSON.stringify(obj).split('"sad":').join('"happy":'));

but throwing obj in place of res will not work. I have tried assigning the value of this res and res.body but no dice.

Thanks for you help in advance!

Matthew McGuff
  • 37
  • 1
  • 8
  • Can you clarify what ```request``` is? There are multiple libraries this could be. – Jim B. Nov 24 '18 at 15:57
  • right. Sorry for not clarifying this. I am using `request ^2.88.0`. Here is the link to the github repo for it https://github.com/request/request – Matthew McGuff Nov 25 '18 at 03:52
  • Got it. Yes, the basic ```request``` module handles streaming to express responses as a first class citizen. So you could use the ```event-stream``` method I outline in the answer. – Jim B. Nov 25 '18 at 05:39

1 Answers1

1

If you're using request-promise, you can simply make a new response and send it, or modify the response you got back:

app.get('/', function (req, res, next) {    
    request({ url: external_endpoint, json: true})
        .then(response => res.json({ happy: response.sad })))
        .catch(next);
});

(of course, you need to handle errors appropriately)

If you want to process it as a stream (which makes sense if you have a massive amount of data), you can use the original request module, and use event-stream to create your pipe:

const es = require('event-stream');

const swapper = es.through(
    function write(data) {
        this.emit("data", data.replace("sad", "happy"));
    },
    function end() {
        this.emit("end");
    }
);

request({ url: external_endpoint})
    .pipe(es.stringify())
    .pipe(swapper)
    .pipe(es.parse())
    .pipe(res);

Here's a sandbox to test the stream processing: https://codesandbox.io/s/3wqx67pq6

Jim B.
  • 4,512
  • 3
  • 25
  • 53
  • In the code above using `await` you need to add `async` just prior to the function or await will return as undefined. – Matthew McGuff Nov 25 '18 at 15:22
  • So on the `.pipe(es.map(j => j.replace('sad', 'happy'))` it would seem like we are missing a closing `)`. However even when I add this to the end of the line it doesn't work as expected. I tried adding to the end of `.pipe(res))` it returns the payload but without any replacing going on. – Matthew McGuff Nov 25 '18 at 16:19
  • Indeed, ```es.map()``` isn't right because we're receiving a string at that point. Replaced with ```es.through()``` – Jim B. Nov 25 '18 at 16:34
  • BTW, had trouble with the codesandbox link, but I think it's working now. – Jim B. Nov 25 '18 at 17:09
  • Awesome but looking to make this just a little better. Say Someone has the word `sad` in their username(or in any value for that matter). Whats to prevent or ensure that this `.replace` wont change that value? I attempted to change the `data.replace("sad", "happy")` to `data.replace('"sad":', '"happy":')` but including quotes seems to break the replace from working at all. Any ideas on how I can target just changing keys instead of data? You have been a huge help thus far. Thanks for everything. – Matthew McGuff Nov 26 '18 at 13:42
  • Just answered my own questions. `.replace` supports the use of regular expressions. This means that I could user a regular expression, something like And to further ensure that I don't replace something I don't intend I can use a regular expression as it appears that `.replace` supports the use of regular expressions. However when testing with something like `"sad":` or even `/"user":/` . It fails to replace though I fairly confident my regex is correct. Any ideas on this? – Matthew McGuff Nov 26 '18 at 14:07
  • Might be a topic for another question. Regexes are powerful but tricky. I recommend breaking it down and using node in REPL mode to debug. – Jim B. Nov 26 '18 at 15:17
  • agreed. Thats I why I marked this as answered. Thanks again! – Matthew McGuff Nov 26 '18 at 20:43