Embedded ES instance failing when running unit tests

Question

I currently have the following code. The following is setting up a local elasticsearch instance using the elastic4s library

val essettings = Settings
  .settingsBuilder().put("cluster.name", "elasticsearch")
  .put("path.home", "/tmp/v").build()

ElasticsearchConnection.setInstance(ElasticClient.local(essettings))

val mockESItem1 = Map("title" -> "Worth It",
  "artist" -> "Fifth Harmony",
  "countries" -> Seq("AI", "CA", "IT", "AU", "GB", "IO", "IE", "GI", "US", "SH", "MS", "KY"),
  "thumbnails" -> Seq(Map("default" -> "http://cache.vevo.com/Content/VevoImages/video/WRONGVEVOPICTURE.jpg")),
  "videoId" -> "VEVOID1", 
  "videoType" -> "type", 
  "ytVideoId" -> "YTID1", 
  "features" -> Seq(),
  "duration" -> 230)

Here I am creating a new index and then inserting the mockES item above.

client.execute( create index "videos" shards 1 replicas 5 mappings(
  "video" as (
    "artist" typed StringType,
    "title" typed StringType,
    "countries" typed StringType,
    "thumbnails" typed ObjectType,
    "videoId" typed StringType,
    "videoType" typed StringType,
    "ytVideoId" typed StringType,
    "features" typed StringType,
    "duration" typed IntegerType
    )
  )
)

client.execute(
  bulk(
    index into "videos"/"video" id 1 fields mockESItem1
  )
).await

However if I then run a query to find that item in any of my tests, similar to the following:

es.execute {
  search in "videos" / "video" limit 5 query bool {
    must(
      queryStringQuery("Worth It").field("title"),
      queryStringQuery("Fifth Harmony").field("artist").field("features"),
      matchQuery("videoType","type"),
      matchQuery("countries","US")
    )
  }
}.await

The program breaks with the following error:

org.elasticsearch.action.NoShardAvailableActionException: null
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.start(TransportSearchTypeAction.java:151) [elasticsearch-2.0.1.jar:2.0.1]
at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:64) [elasticsearch-2.0.1.jar:2.0.1]
at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:53) [elasticsearch-2.0.1.jar:2.0.1]
...
org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:228) ~[elasticsearch-2.0.1.jar:2.0.1]
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.start(TransportSearchTypeAction.java:151) ~[elasticsearch-2.0.1.jar:2.0.1]
at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:64) ~[elasticsearch-2.0.1.jar:2.0.1]
at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:53) ~[elasticsearch-2.0.1.jar:2.0.1]

I've been trying to find a solution to this issue off and on for a few days and haven't been able to find anything that helps me solve this issue, so now looking to see if anyone on SE has had similar issues and found a fix.

UPDATE

I've rewritten what I have so it's a much simpler example, since previous there were still a couple classes from my own code that I was using.

import com.sksamuel.elastic4s.ElasticClient
import com.sksamuel.elastic4s.ElasticDsl._
import com.sksamuel.elastic4s.mappings.FieldType.{IntegerType, ObjectType, StringType}
import org.elasticsearch.action.delete.DeleteRequest
import org.elasticsearch.common.settings.Settings
import org.scalatest._
class ESTest extends FlatSpec with Matchers {
  val essettings = Settings
    .settingsBuilder().put("cluster.name", "elasticsearch")
    .put("path.home", "/tmp/v").build()

  val client = ElasticClient.local(essettings)

  val mockESItem1 = Map("title" -> "Worth It",
    "artist" -> "Fifth Harmony",
    "countries" -> Seq("AI", "CA", "IT", "AU", "GB", "IO", "IE", "GI", "US", "SH", "MS", "KY"),
    "thumbnails" -> Seq(Map("default" -> "http://cache.vevo.com/Content/VevoImages/video/WRONGVEVOPICTURE.jpg")),
    "videoId" -> "VEVOID1",
    "videoType" -> "type",
    "ytVideoId" -> "YTID1",
    "features" -> Seq(),
    "duration" -> 230)

  client.execute(create index "videos" shards 1 replicas 5 mappings (
    "video" as(
      "artist" typed StringType,
      "title" typed StringType,
      "countries" typed StringType,
      "thumbnails" typed ObjectType,
      "videoId" typed StringType,
      "videoType" typed StringType,
      "ytVideoId" typed StringType,
      "features" typed StringType,
      "duration" typed IntegerType
      )
    )
  ).await


  val exists = client.execute {
    index exists "videos"
  }.await.isExists

  println(exists)
  println(mockESItem1.get("videoId").get)

  client.execute(
    index into "videos" / "video" id mockESItem1.get("videoId").get fields mockESItem1
  ).await
  println("hi")

  val resp = client.execute {
    search in "videos" / "video" limit 5 query bool {
      must(
        matchQuery("videoType", "type"),
        matchQuery("countries", "US")
      )
    }
  }.await

  println(resp.hits.head.sourceAsString)
  client.client.delete(new DeleteRequest("videos"))
  client.close
}

Currently if you run this it will fail to execute the bulk insert, then receive the NoShardAvailableActionException when the "test" tries to run the search. Could this be an issue caused by specs2?

score 0 · Answer 1 · answered Dec 22 '15 at 07:59

This works for me in 1.7.4 (if using 2.0+ change ImmutableSettings to Settings):

object Test extends App {

  import ElasticDsl._

  val essettings = ImmutableSettings
    .settingsBuilder().put("cluster.name", "elasticsearch")
    .put("path.home", "/tmp/v").build()

  val client = ElasticClient.local(essettings)

  val mockESItem1 = Map("title" -> "Worth It",
    "artist" -> "Fifth Harmony",
    "countries" -> Seq("AI", "CA", "IT", "AU", "GB", "IO", "IE", "GI", "US", "SH", "MS", "KY"),
    "thumbnails" -> Seq(Map("default" -> "http://cache.vevo.com/Content/VevoImages/video/WRONGVEVOPICTURE.jpg")),
    "videoId" -> "VEVOID1",
    "videoType" -> "type",
    "ytVideoId" -> "YTID1",
    "features" -> Seq(),
    "duration" -> 230)

  client.execute(create index "videos" shards 1 replicas 5 mappings (
    "video" as(
      "artist" typed StringType,
      "title" typed StringType,
      "countries" typed StringType,
      "thumbnails" typed ObjectType,
      "videoId" typed StringType,
      "videoType" typed StringType,
      "ytVideoId" typed StringType,
      "features" typed StringType,
      "duration" typed IntegerType
      )
    )
  )

  client.execute(
    bulk(
      index into "videos" / "video" id 1 fields mockESItem1
    )
  ).await

  val resp = client.execute {
    search in "videos" / "video" limit 5 query bool {
      must(
        matchQuery("videoType", "type"),
        matchQuery("countries", "US")
      )
    }
  }.await

  println(resp.hits.head.sourceAsString)
  sys.exit(0)
}

Since this is your code with the parts you missed out added in, I can only assume your error is in some of the code you are not showing (like ElasticsearchConnection). I also adjusted your query slightly.

I have an updated set of code without any custom "fluff" from my actual tests. This is using elasticsearch 2.0. I'm still receiving the `NoShardAvailableActionException`, it looks like this is possibly due to no index being created? Then that would explain why it's failing to insert items and find anything. I don't know why the index wouldn't be created though. — rebnat, Dec 22 '15 at 22:29
Added an await, also wrapped the insert in a try catch and am doing single insert instead of bulk (updated the code in my question). I am catching a 10 second futures timeout now. However adding the .await did seem to fix the NoShardsAvailable issue, since now I do not get that exception. — rebnat, Dec 25 '15 at 01:53
I changed the update code to a barebones test. I am creating the index and then checking to make sure it exists, which it does. It is never getting to the println("hi") line however, due to a 10 second futures timeout. Any thoughts on why this is? — rebnat, Dec 29 '15 at 20:00

score 0 · Answer 2 · answered May 03 '16 at 13:22

I had a similar problem: subsequent test runs failed with a NoShardAvailableActionException when I tested on an in-memory Elasticsearch node. As I realized that this only happened when the tests were run for 2 or more times my solution was to delete the old index. Roughly, I did:

if (testsDoRunInInternalMode()) {
  client.admin().indices().delete(new DeleteIndexRequest(myIndex)).actionGet();
}

Maybe that helps.

Embedded ES instance failing when running unit tests

UPDATE

2 Answers2