1

I'm using istio with ingress gateway, and added zipkin tracing. All my apps are using spring boot with sleuth zipkin.

I've deployed 2 zipkin for testing

  • deployed in istio-system by istio documentation
  • deployed manually in a zipkin namespace

the spring boot configuration are pointing to the zipkin namespace, with always sampled configuration.

Problem is when I'm using ingress gateway, the trace id looks like request id and it does propagate to my sub systems. But when I query to zipkin (deployed both in istio-system from istio documentation, and manually deployed to another namespace) the trace id are not present.

Interestingly when I do port-forward of my outer most system, and hit the spring boot with grpc, the trace id are being propagated to the sub systems, and it does show in the zipkin dashboard.

although the trace id are different when using ingressgateway and port-forward direct grpc call :

ingressgateway : 0672471566b9305f7dcaadecaf1a8c71 direct call : cdc337ec90b8c085

Thanks!

Jonathan Hosea
  • 11
  • 1
  • Are you sure that the ingress gateway is sending spans to ZIpkin? Maybe it is propagating the trace context but has no knowledge of Zipkin as such? – Marcin Grzejszczak Aug 27 '20 at 07:30
  • Did you configure trace sampling as mentioned in istio [documentation](https://istio.io/latest/docs/tasks/observability/distributed-tracing/zipkin/#before-you-begin)? Everything is working when you point to zipkin in istio-system namespace? – Jakub Aug 27 '20 at 07:53
  • Are you using zipkin service which is present in istio and that is pointing to tracing component. As istio usually point to zipkin service and service will point to tracing pod which contain Jaeger based API, Please check in istio-system namespace you have jaeger-query service. Try to port-forward the service and check are you getting any traces. – Anurag Jain Aug 31 '20 at 13:48
  • I've actually fixed it using zipkin deployed in istio-system using this https://discuss.istio.io/t/apply-zipkin-adapter-w-tracespan-template-not-working/3071 But for some reason, if the trace started from ingress gateway, the trace id is visible on istio-system local, and when I deploy zipkin outside cluster it only registers when I invoke the system using port-forward of the node. – Jonathan Hosea Sep 06 '20 at 05:17

0 Answers0