Istio egress gives "upstream connect error or disconnect/reset before headers" errors from python micro-service

Question

When I am running a Python micro-service in a dockerized or kubernetes container it works just fine. But with Istio service mesh, it is not working.

I have added ServiceEntry for two of my outbound external http apis. It seems I can access the url content form inside the container using curl command which is inside service mesh. So, I think the service entries are fine and working.

But when I try from the micro-service which uses xml.sax parser in Python, it gives me the upstream connect error or disconnect/reset before headers though the same application works fine without Istio.

I think it is something related to Istio or Envoy or Python.

Update: I did inject the Istio-proxy side-car. I have also added ServiceEntry for external MySQL database and mysql is connected from the micro-service.

What does `kubectl -n namespace_name logs pod_name -c istio-proxy` command give? — nurgasemetey, Dec 22 '18 at 19:16
Are you sure the protocol is HTTP and not HTTPS? Are you able to access both external APIs by curl from a pod inside the service mesh? — Vadim Eisenberg, Dec 23 '18 at 08:37
@VadimEisenberg Yes, protocol is http and I can access both the APIs by curl from the pod. — rayhan, Dec 23 '18 at 16:56
Can you publish your `ServiceEntry` configurations? And which version of Istio you are using? — Vadim Eisenberg, Dec 23 '18 at 17:40
Yes, I am using mutual TLS istio configuration. I think I have found the problem. It is causing due to HTTP/1.0 request made by python xml.sax parser. So, its a legacy application issue as Envoy doesn't support HTTP/1.0 — rayhan, Dec 29 '18 at 12:59

score 2 · Accepted Answer · answered Dec 29 '18 at 13:05

I have found the reason for this not working. My Python service is using xml.sax parser library to parse xml form the internet, which is using the legacy urllib package which initiate http/1.0 request.

Envoy doesn't support http/1.0 protocol version. Hence, it is not working. I made the workaround by setting global.proxy.includeIPRanges="10.x.0.1/16" for Istio using helm. This actually bypass the entire envoy proxy for all outgoing connections outside the given ip ranges.

But I would prefer not to globally bypass Istio.

Istio egress gives "upstream connect error or disconnect/reset before headers" errors from python micro-service

1 Answers1