Not able to read data from elasticsearch using alpakka

Question

I was trying to read document(json data stored in ES) from elasticsearch using alpakka. I got this alpakka-Elasticsearch. Here it says that you can stream messages from or to Elasticsearch using the ElasticsearchSource, ElasticsearchFlow or the ElasticsearchSink. I tried to impliment ElasticsearchSource method. So my code looks like this

  val url = "http://localhost:9200"
  val connectionSettings = ElasticsearchConnectionSettings(url)
  val sourceSettings = ElasticsearchSourceSettings(connectionSettings)
  val elasticsearchParamsV7 = ElasticsearchParams.V7("category_index")
        val copy = ElasticsearchSource
          .typed[CategoryData](
            elasticsearchParamsV7,
            query = query,
            sourceSettings
          ).map { message: ReadResult[CategoryData] =>
          println("Inside message==================>  "+message)
          WriteMessage.createIndexMessage(message.id, message.source)
        } .runWith(
          ElasticsearchSink.create[CategoryData](
            elasticsearchParamsV7,ElasticsearchWriteSettings(connectionSettings)
          )
        )
  println("Final data==============>.  "+copy)

At the end, copy value returning Future[Done]. But I was not able to read data from ES.
Is there Something I missing?
And also is there any other way using akka http client api to do the same?
What is preferred way to use ES in akka?

val query = Json.obj( "query" -> Json.obj( "bool" -> Json.obj( "must" -> Seq( Json.obj( "match" -> Json.obj( "isDeleted"-> false) ) ) )) ).toString() — md samual, Jul 03 '21 at 03:04
The above comment is my query which I want to execute on category_index. But still not able to read data. — md samual, Jul 03 '21 at 03:06

earthling paul · Answer 1 · 2021-07-21T05:26:52.133

To read data from Elasticsearch, sth like this should be enough:

  val matchAllQuery = """{"match_all": {}}"""
  val result = ElasticsearchSource
    .typed[CategoryData](
      elasticsearchParamsV7,
      query = matchAllQuery,
      sourceSettings
    ).map { message: ReadResult[CategoryData] =>
    println("Read message==================>  "+message)
  }.runWith(Sink.seq)

  result.onComplete(res => res.foreach(col => println(s"Read: ${col.size} records")))

If the type CategoryData does not match correctly to what is stored in the index, the query may not return results.

If in doubt, it's possible to read raw JSON:

  val elasticsearchSourceRaw = ElasticsearchSource
    .create(
      elasticsearchParamsV7,
      query = matchAllQuery,
      settings = sourceSettings
    )

Not able to read data from elasticsearch using alpakka

1 Answers1