Getting an incomplete (chunked) JSON response on Angular proxy

Question

While developing an Angular 7 web app, I am using the proxy.conf.json file and --proxy-config option on the ng serve command to proxy my requests from https://localhost:4200/api to a remote url which hosts the actual REST API.

When doing an HTTP GET request directly to the remote url through my browser (Chrome) or Postman, I always get the full JSON response like I expect it to be.

When requesting this on the angular proxy url, I get only a part of the JSON response (most of the time - not always).

I am aware that the API returns response header Transfer-encoding 'chunked' and from what I understand, this means having to process the response as a stream.

One way to avoid this, is to let the API return a Content-Length header. But I'd rather want to leave the API unchanged.
I read that when proxying on an nginx server, there is an option 'proxy_buffering' to configure how to handle large HTTP responses. But I'm using the built-in angular proxy (which is a webpack dev server) and I haven't found a way to configure such property.

proxy.conf.json

...
 "/api/messages": {
    "target": "http://myRemoteHost:myRemotePort/myRemoteContext/v1/",
    "secure": false,
    "changeOrigin": true,
    "logLevel": "debug",
    "pathRewrite": {
      "^/api": ""
    }
  },
...

messages.service.ts

return this.http.get<Message[]>(this.endpoint, {params: httpParams});

Error in console

Error Code: 200
Message: Http failure during parsing for http://...

HttpErrorResponse:
error: SyntaxError: Unexpected end of JSON input at JSON.parse (<anonymous>) 
at XMLHttpRequest.onLoad (https://localhost:4200/hal/vendor.js:32570:51) 
at ZoneDelegate.push../node_modules/zone.js/dist/zone.js.ZoneDelegate.invokeTask (https://localhost:4200/hal/polyfills.js:2768:31) 
at Object.onInvokeTask (https://localhost:4200/hal/vendor.js:78088:33) at ZoneDelegate.push../node_modules/zone.js/dist/zone.js.ZoneDelegate.invokeTask (https://localhost:4200/hal/polyfills.js:2767:60) 
at Zone.push../node_modules/zone.js/dist/zone.js.Zone.runTask (https://localhost:4200/hal/polyfills.js:2540:47) 
at ZoneTask.push../node_modules/zone.js/dist/zone.js.ZoneTask.invokeTask [as invoke] (https://localhost:4200/hal/polyfills.js:2843:34) 
at invokeTask (https://localhost:4200/hal/polyfills.js:4089:14) 
at XMLHttpRequest.globalZoneAwareCallback (https://localhost:4200/hal/polyfills.js:4126:21)

message: "Unexpected end of JSON input"

I was expecting that when using the standard HttpClient from Angular, I always would get the full response (also for large HTTP responses). This does not seem to be the case. My gut feeling says the proxy is causing this.

So do I need to specifically change my code in order to process chunked http responses? Or is this (hopefully configurable) behaviour from the webpack server?

Does anyone know a possible solution?

maybe try to look here https://stackoverflow.com/questions/42556455/unexpected-end-of-json-input-in-http-get-request-from-angular-2-to-codeigniter "This causes problems when the api returns back an empty response. You'll then get the Unexpected end of json error. The solution is to set responseType as text" — Disaster, Aug 19 '19 at 09:57
@Disaster, Thanks for your suggestion but I already tried that. The response is not empty, but it is an incomplete JSON. It is just cut off causing the JSON to be invalid. It does not help to set the responseType as text, because I need to parse the response as json (and later to typescript objects) anyway. — Gert, Aug 19 '19 at 10:41

score 0 · Answer 1 · answered Jan 09 '20 at 20:12

0

One way to avoid this, is to let the API return a Content-Length header. But I'd rather want to leave the API unchanged.

This worked for us, our customers had proxy servers for security, and 25% of our JSON requests were broken because Transfer-Encoding was chunked. We had to remove compression for the time being.

Thank you kind stranger for posting this question.

answered Jan 09 '20 at 20:12

Nth.gol

746
9
20

Won't removing compression have any other side effects ? – user2868864 Jul 08 '22 at 13:11

score 0 · Answer 2 · answered Jan 17 '20 at 15:48

0

I had a similar problem using angular 6 and running the local server (with ng --serve) with node.js 8.10.

I resolved it by upgrading node.js to 12.13.0

answered Jan 17 '20 at 15:48

pwillemet

613
5
21

score 0 · Answer 3 · answered Jun 11 '20 at 13:33

0

Got around this by adding

"headers": {
    "Connection": "keep-alive"
  }

inside the proxy item declaration in proxy.conf.json, as follows:

"/api/messages": {
      "target": "http://myRemoteHost:myRemotePort/myRemoteContext/v1/",
      "secure": false,
      "changeOrigin": true,
      "logLevel": "debug",
      "headers": {
        "Connection": "keep-alive"
      }
}

Adding this, I managed to retrieve quite large datasets without the JSON response getting cut off.

Hope this can help someone, this has been haunting me for the past few days.

answered Jun 11 '20 at 13:33

Coert

1

Where do we find this usually ? I couldn't find any file like this on the existing code – user2868864 Jul 08 '22 at 13:12
It does not get generated automatically, you have to create the proxy.conf.json file in the root of the project. It may also have another name. Best is to look in the package.json file and see what file is referenced in the npm script start command with the --proxy flag. – Coert Jul 19 '22 at 07:44

Getting an incomplete (chunked) JSON response on Angular proxy

3 Answers3