0

I am new to Jaeger and Kafka,

I am trying to use Kafka as intermediate buffer.

I am using OpenTelemetry to send data to Jaeger-Collector directly using -Dotel.exporter.jaeger.endpoint. Jaeger-Collector is deployed on Kubernetes and the Kafka is on another network but is accessible. I can confirm that the traces are sent to Jaeger-collector.

On hitting the /metrics of collector and output tells me that spans were written successfully to Kafka.

jaeger_kafka_spans_written_total{status="success"} 21

The Collector logs indicate what topic I am writing to

{"Brokers":["myKafkaBroker......"}},"topic":"tp6"}

I want to get this (Span) data from Kafka Queue to ElasticSearch. To do this I am starting the Jaeger Ingester as follows

docker run -e "SPAN_STORAGE_TYPE=elasticsearch" jaegertracing/jaeger-ingester:1.22 --kafka.consumer.topic=tp6 --kafka.consumer.brokers='myKafkaBroker' --es.tls.skip-host-verify

But the container is stopped after fatal error

{"level":"fatal","ts":1615546463.7784193,"caller":"command-line-arguments/main.go:64","msg":"Failed to init storage factory","error":"failed to create primary Elasticsearch client: health check timeout: Head \"http://127.0.0.1:9200\": dial tcp 127.0.0.1:9200: connect: connection refused: no Elasticsearch node available","stacktrace":"main.main.func1\n\tcommand-line-arguments/main.go:64\ngithub.com/spf13/cobra.(*Command).execute\n\tgithub.com/spf13/cobra@v0.0.7/command.go:838\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tgithub.com/spf13/cobra@v0.0.7/command.go:943\ngithub.com/spf13/cobra.(*Command).Execute\n\tgithub.com/spf13/cobra@v0.0.7/command.go:883\nmain.main\n\tcommand-line-arguments/main.go:113\nruntime.main\n\truntime/proc.go:204"}

The elasticsearch and ingester are being run on the same machine using docker. The elasticsearch is running on docker using

docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node"ocker.elastic.co/elasticsearch/elasticsearch:7.11.2

I have disabled TLS so that shouldn't be a problem. I am unable to get this to work.

Magellan
  • 71
  • 1
  • 2
  • 11
  • I don't see you telling the container `jaegertracing/jaeger-ingester` where your elastic search is and it is trying to find it inside the container itself – Tarun Lalwani Mar 12 '21 at 11:27
  • i had tried using es.server-urls:localhost:9200 as well. But isn't the es container accessible to the ingester container ? (127.0.0.1:9200) – Magellan Mar 12 '21 at 11:43
  • If the jaeger ingester and elasticsearch are running on different containers, you can't use `127.0.0.1` as the elasticsearch host, `127.0.0.1` is the localhost loopback address, it would only work if both applications were on the same host. You need to pass the ip address or acessible hostname of the container running elasticsearch. – leandrojmp Mar 12 '21 at 13:07
  • oh boy, I got confused. I thought since the service is available on host machine @ localhost:9200, other container will be able to access using localhost:9200 but then oi forgot to take into account that containers are machine in itself. Thanks Tarun and leandrojmp – Magellan Mar 12 '21 at 14:19

0 Answers0