I'm currently setting up an opentelemetry project, which sends data to jaeger to view the created spans. As long as I am using the jaeger-all-in-one with in memory storage it works. As soon as I use elastic for persistent data storage it stops working.
I am running everything on a windows system, no docker. Default elasticsearch, no changes to settings, with the elasticsearch.bat from the website.
I start jaeger-all-in-one with the following command:
jaeger-all-in-one.exe --es.server-urls=https://localhost:9200 --es.username=elastic --es.password=TvpO0Q6fffoHVlmjYWXSo --es.tls.enabled=true --es.tls.skip-host-verify --es.create-index-templates=false --es.tags-as-fields.all=true --es.version=7
It all starts correct, going to localhost:16686 displays the Jaeger UI correctly, once. As soon as I refresh, I see the following messages in my elasticsearch:
[2023-04-27T15:25:50,515][INFO ][o.e.c.m.MetadataCreateIndexService] [PCNAME] [jaeger-span-2023-04-27] creating index, cause [auto(bulk api)], templates [], shards [1]/[1]
[2023-04-27T15:25:50,582][INFO ][o.e.c.m.MetadataCreateIndexService] [PCNAME] [jaeger-service-2023-04-27] creating index, cause [auto(bulk api)], templates [], shards [1]/[1]
[2023-04-27T15:25:51,035][INFO ][o.e.c.m.MetadataMappingService] [PCNAME] [jaeger-service-2023-04-27/8s9ZRaa_Rd2aa0AhRjVJ2A] create_mapping
[2023-04-27T15:25:51,116][INFO ][o.e.c.m.MetadataMappingService] [PCNAME] [jaeger-span-2023-04-27/ZtqawFQTRRSTPaeLYK_P4A] create_mapping
[2023-04-27T15:25:51,225][INFO ][o.e.c.m.MetadataMappingService] [PCNAME] [jaeger-span-2023-04-27/ZtqawFQTRRSTPaeLYK_P4A] update_mapping [_doc]
[2023-04-27T15:25:59,536][INFO ][o.e.c.m.MetadataMappingService] [PCNAME] [jaeger-span-2023-04-27/ZtqawFQTRRSTPaeLYK_P4A] update_mapping [_doc]
After that, my Jaeger UI throws the error
HTTP Error: search services failed: elastic: Error 400 (Bad Request): all shards failed [type=search_phase_execution_exception]
I've read that this has something to do with ssl, so I tried turning that off on my elasticsearch, and omitting the parameters --es.tls.enabled=true --es.tls.skip-host-verify but jaeger fails to connect to the elasticsearch if I do that.
Using Jaeger version 1.44 and ElasticSearch version 7