Dark magic going through res.json()

Question

To avoid the XY problem, here's my final goal: I want to fetch something, use the response body, and return the response (from which the user should be able to get the body) without returning the body separately. On the paper, this should work:

const res = await fetch("something")
const clone = res.clone()
const body = await clone.json()
// Use body
return res

If I get lucky, the process ends during await clone.json(). If I don't, it freezes. Forever.

As a reminder, res.json can't be called twice.

Does it think I'm not good enough to get an error message? Any help on that? Thank you.

Progess

I located the error: in node_modules/node-fetch/lib/index.js:416, the listener on the end of the action is never triggered. I still don't know why.

When replacing the URL with "https://jsonplaceholder.typicode.com/todos/1", everything works. This may be related to the server. I don't see any link between the origin of the request and whether the request could be cloned...

Placing res.text() before clone.json() magically fixes it, but it makes everything lose its purpose.

By messing with the module's code, I found that the response actually never ends. The JSON gets cut, and the chunk from the last data event call isn't the end of the JSON, so the end event never gets called. Extremely confusing.

I spent too much time on this issue, I will simply avoid it by returning the body separately.

does anything change if you put the clone/json logic inside a promise and avoid making the function async? — apoteet, Mar 14 '21 at 16:21
If you think there's an error, but you're not seeing it, try placing your code inside of `try-catch` block, like `try { /* your code */ } catch (e) { console.warn(e) }` — ΔO 'delta zero', Mar 14 '21 at 16:26
Yes, it was my first guess, I've already tried it.... I'll check again. — deb, Mar 14 '21 at 16:30
Sounds like the endpoint you are hitting isn't properly closing the connection once it's piped through the data. The reason why it works sometimes is likely because the endpoint terminates inactive connections as it shuts down/times out which then fires the 'end' listener. — samthecodingman, Mar 14 '21 at 16:59
@samthecodingman You're probably right, but why would it work when not cloning the response then? — deb, Mar 14 '21 at 17:02
I personally avoid cloning streams, because of these weird side effects. Out of curiosity, does it work when you consume the original and return the clone? `const res = await fetch("something"); const clone = res.clone(); const body = await res.json(); /* use body */; return clone;` — samthecodingman, Mar 14 '21 at 17:20
No, it doesn't work. It seems to mess up the original and the clone. As I said in the 3rd progress update though, strangely I can consume the clone after consuming the original... This makes absolutely no sense — deb, Mar 14 '21 at 17:22

score 1 · Answer 1 · answered Mar 14 '21 at 16:24

1

This seems to work just fine

const myFetch = async (url) => {
  const resp = await fetch(url),
        clone = resp.clone()
        json = await clone.json() 
  console.log('inner:', json.title)
  return resp
}

myFetch('https://jsonplaceholder.typicode.com/todos/1')
  .then(resp => resp.json())
  .then(json => console.log('outer:', json.title))

answered Mar 14 '21 at 16:24

ΔO 'delta zero'

3,506
1
19
31

Yes, it does work, even when copying it on my machine... Something else is wrong, let me investigate... – deb Mar 14 '21 at 16:29

Dark magic going through res.json()

Progess

1 Answers1