1

I am trying to test elastic search with the following instruction:

http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-gsg.html

When I try the above and upload the following text which is also in the instruction:

{
   "index": {
    "_index": "movies",
    "_type": "listings",
    "_id": "2"
}
 } {
"director": "Frankenheimer, John",
"genre": ["Drama", "Mystery", "Thriller"],
"year": 1962,
"actor": ["Lansbury, Angela", "Sinatra, Frank", "Leigh, Janet", "Harvey, Laurence", "Silva, Henry", "Frees, Paul", "Gregory, James", "Bissell, Whit", "McGiver, John", "Parrish, Leslie", "Edwards, James", "Flowers, Bess", "Dhiegh, Khigh", "Payne, Julie", "Kleeb, Helen", "Gray, Joe", "Nalder, Reggie", "Stevens, Bert", "Masters, Michael", "Lowell, Tom"],
"title": "The Manchurian Candidate"
 } {
"index": {
    "_index": "movies",
    "_type": "listings",
    "_id": "3"
}
} {
"director": "Baird, Stuart",
"genre": ["Action", "Crime", "Thriller"],
"year": 1998,
"actor": ["Downey Jr., Robert", "Jones, Tommy Lee", "Snipes, Wesley", "Pantoliano, Joe", "Jacob, Ir\u00e8ne", "Nelligan, Kate", "Roebuck, Daniel", "Malahide, Patrick", "Richardson, LaTanya", "Wood, Tom", "Kosik, Thomas", "Stellate, Nick", "Minkoff, Robert", "Brown, Spitfire", "Foster, Reese", "Spielbauer, Bruce", "Mukherji, Kevin", "Cray, Ed", "Fordham, David", "Jett, Charlie"],
"title": "U.S. Marshals"
} {
"index": {
    "_index": "movies",
    "_type": "listings",
    "_id": "4"
}
} {
"director": "Ray, Nicholas",
"genre": ["Drama", "Romance"],
"year": 1955,
"actor": ["Hopper, Dennis", "Wood, Natalie", "Dean, James", "Mineo, Sal", "Backus, Jim", "Platt, Edward", "Ray, Nicholas", "Hopper, William", "Allen, Corey", "Birch, Paul", "Hudson, Rochelle", "Doran, Ann", "Hicks, Chuck", "Leigh, Nelson", "Williams, Robert", "Wessel, Dick", "Bryar, Paul", "Sessions, Almira", "McMahon, David", "Peters Jr., House"],
"title": "Rebel Without a Cause"
 }

After when I use this command :

curl -XGET 'search-movies-4f3nw7eiia2xiynjr55a2nao2y.us-west-1.es.amazonaws.com/movies/_search?q=Frankenheimer'

I never get anything back. always returns 0. Here is what I get back:

enter image description here

I am not if I am doing it right but I think I should be able to see at least something because I have Frankenheimer in the director key of the json data uploaded. Can anyone shed light on it?

Hamed Minaee
  • 2,480
  • 4
  • 35
  • 63
  • Do you get any results just doing a `curl -XGET search-movies-4f3nw7eiia2xiynjr55a2nao2y.us-west-1.es.amazonaws.com/movies/_search` ? This should ensure the data is actually in the index – John Veldboom Oct 07 '17 at 01:48
  • @JohnVeldboom Intresting. I never tried that. YES I do get that. So then why am I not getting when I add search query? – Hamed Minaee Oct 07 '17 at 01:50
  • @JohnVeldboom Here is what I get when I try yours: took":1,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":1,"max_score":1.0,"hits":[{"_index":"movies","_type":"movie","_id":"1","_score":1.0,"_source":{"director": "Burton, Tim", "genre": ["Comedy","Sci-Fi"], "year": 1996, "actor": ["Jack Nicholson","Pierce Brosnan","Sarah Jessica Parker"], "title": "Mars Attacks!"}}]}} – Hamed Minaee Oct 07 '17 at 01:51

1 Answers1

1

Try searching the whole index with curl -XGET search-movies-4f3nw7eiia2xiynjr55a2nao2y.us-west-1.es.amazon‌​aws.com/movies/_sear‌​ch which will return the first 10 results by default. If it returns less than 10, the data is likely not in the index.

You can verify that by changing the query to:

curl -XGET search-movies-4f3nw7eiia2xiynjr55a2nao2y.us-west-1.es.amazon‌​aws.com/movies/_sear‌​ch?q=director:Burton

Check out the Elasticsearch URI Search for the complete list of the parameters.

John Veldboom
  • 2,049
  • 26
  • 29
  • Thanks yes you are right. So a question if I have some columns already in mysql should I grab the columns data I want to add for search and insert tehm it using above to elastic search? – Hamed Minaee Oct 07 '17 at 03:31
  • 1
    Yes, that is correct. I've used Elasticsearch for this previously - works pretty well. You will just need some kind of job that indexes that data into ES. Two recommendations on that: 1) use a last updated field in MySQL which will allow you to only re-index data that's changed. 2) Use the records's unique id MySQL as the ES id. This will allow you to update records easily in ES. If you need more help, feel free to hit me up on Twitter @jveldboom – John Veldboom Oct 07 '17 at 12:48
  • Thanks so if I am not mistken I may need to use logstash or may be kenisis+ lamda to migrate all the data I have in the mysql db and also I need to post whatever new data that will come to mysql db to Elastic search at the same time of insertion to mysql db. Am right? Is the second step looks good? – Hamed Minaee Oct 07 '17 at 16:08
  • 1
    If you have a large amount of data to be index, then yes, Logstash or Kinesis will work. But if it's relatively small, a simple script that loops over the data and inserts each row into ES would work too. Also, if you need the data to be immediately searchable, then yes POSTing the data to ES at insertion time. But the search-ability can wait, you could batch process it for a later time. – John Veldboom Oct 07 '17 at 21:38