0

How can we ingest data to elastic search through java without logstash and beats is there any option like kafka or something like using only java without any tools

Nate
  • 10,361
  • 3
  • 33
  • 40
harish kumar
  • 13
  • 5
  • 1
    you can `POST` data directly to `elasticserch`, no `java` needed: https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html – Nate Aug 12 '20 at 23:54

1 Answers1

0

I am not sure why you dont want to consider Filebeats --> Elastic. But yes, there are other ways to send your logs to Elastic search.

Also, you did not mention whats the source, whether you want to insert app logs, database. Assuming you want to send microservices logs also, and below options holds good for sending other data too.

As you dont want to use Filebeat, you should add custom code to collect, refine, format and publish the logs.

  • you can use Kafka Sink Connector to Elastic search to move all your logs
  • Also, you can use UDP protocol to send(client) logs and listen(server), then implement buffer and ingest to Elastic.

you can develop a commons lib which holds all this code and use in all your java applications.

Simple udp client server code - https://github.com/suren03/udp-server-client

Surendra S
  • 91
  • 1
  • 6
  • 1
    `Elastic listens on udp port 54328` ?? This is new to me. Where did you see that? – Val Aug 13 '20 at 03:35
  • @Val - I meant implementing udp client and server and mentioned the port we are using :).Thanks for checking. – Surendra S Aug 14 '20 at 17:21