0

I"m new to elasticsearch and i"m trying to connect it with mongodb. I tried to do it with mongo-connector. so i"m running mongod and this is the output i get:

2016-11-21T10:03:24.213+0200 I CONTROL  [initandlisten] MongoDB starting : pid=7244 port=27017 dbpath=C:\data\db\ 64-bit host=DESKTOP-7QLMJRC
2016-11-21T10:03:24.215+0200 I CONTROL  [initandlisten] targetMinOS: Windows Vista/Windows Server 2008
2016-11-21T10:03:24.216+0200 I CONTROL  [initandlisten] db version v3.2.4
2016-11-21T10:03:24.216+0200 I CONTROL  [initandlisten] git version: e2ee9ffcf9f5a94fad76802e28cc978718bb7a30
2016-11-21T10:03:24.217+0200 I CONTROL  [initandlisten] allocator: tcmalloc
2016-11-21T10:03:24.217+0200 I CONTROL  [initandlisten] modules: none
2016-11-21T10:03:24.217+0200 I CONTROL  [initandlisten] build environment:
2016-11-21T10:03:24.217+0200 I CONTROL  [initandlisten]     distarch: x86_64
2016-11-21T10:03:24.217+0200 I CONTROL  [initandlisten]     target_arch: x86_64
2016-11-21T10:03:24.218+0200 I CONTROL  [initandlisten] options: {}
2016-11-21T10:03:24.220+0200 I -        [initandlisten] Detected data files in C:\data\db\ created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2016-11-21T10:03:24.223+0200 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=8G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2016-11-21T10:03:24.886+0200 I NETWORK  [HostnameCanonicalizationWorker] Starting hostname canonicalization worker
2016-11-21T10:03:24.886+0200 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory 'C:/data/db/diagnostic.data'
2016-11-21T10:03:24.891+0200 I NETWORK  [initandlisten] waiting for connections on port 27017

Then I run the elasticsearch and this is the output I get:

>

[2016-11-21T10:07:33,859][INFO ][o.e.n.Node               ] [] initializing ...
[2016-11-21T10:07:33,945][INFO ][o.e.e.NodeEnvironment    ] [YzvQTk1] using [1] data paths, mounts [[(C:)]], net usable_space [66.7gb], net total_space [223.1gb], spins? [unknown], types [NTFS]
[2016-11-21T10:07:33,946][INFO ][o.e.e.NodeEnvironment    ] [YzvQTk1] heap size [1.9gb], compressed ordinary object pointers [true]
[2016-11-21T10:07:33,947][INFO ][o.e.n.Node               ] [YzvQTk1] node name [YzvQTk1] derived from node ID; set [node.name] to override
[2016-11-21T10:07:33,954][INFO ][o.e.n.Node               ] [YzvQTk1] version[5.0.1], pid[6968], build[080bb47/2016-11-11T22:08:49.812Z], OS[Windows 10/10.0/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_111/25.111-b14]
[2016-11-21T10:07:35,406][INFO ][o.e.p.PluginsService     ] [YzvQTk1] loaded module [aggs-matrix-stats]
[2016-11-21T10:07:35,407][INFO ][o.e.p.PluginsService     ] [YzvQTk1] loaded module [ingest-common]
[2016-11-21T10:07:35,407][INFO ][o.e.p.PluginsService     ] [YzvQTk1] loaded module [lang-expression]
[2016-11-21T10:07:35,407][INFO ][o.e.p.PluginsService     ] [YzvQTk1] loaded module [lang-groovy]
[2016-11-21T10:07:35,407][INFO ][o.e.p.PluginsService     ] [YzvQTk1] loaded module [lang-mustache]
[2016-11-21T10:07:35,407][INFO ][o.e.p.PluginsService     ] [YzvQTk1] loaded module [lang-painless]
[2016-11-21T10:07:35,407][INFO ][o.e.p.PluginsService     ] [YzvQTk1] loaded module [percolator]
[2016-11-21T10:07:35,407][INFO ][o.e.p.PluginsService     ] [YzvQTk1] loaded module [reindex]
[2016-11-21T10:07:35,407][INFO ][o.e.p.PluginsService     ] [YzvQTk1] loaded module [transport-netty3]
[2016-11-21T10:07:35,408][INFO ][o.e.p.PluginsService     ] [YzvQTk1] loaded module [transport-netty4]
[2016-11-21T10:07:35,408][INFO ][o.e.p.PluginsService     ] [YzvQTk1] no plugins loaded
[2016-11-21T10:07:39,324][INFO ][o.e.n.Node               ] [YzvQTk1] initialized
[2016-11-21T10:07:39,325][INFO ][o.e.n.Node               ] [YzvQTk1] starting ...
[2016-11-21T10:07:39,924][INFO ][o.e.t.TransportService   ] [YzvQTk1] publish_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}, {[::1]:9300}
[2016-11-21T10:07:44,059][INFO ][o.e.c.s.ClusterService   ] [YzvQTk1] new_master {YzvQTk1}{YzvQTk1WT_asjjzSFEeaOw}{DhwO889xQDGHqsQuKBkq4Q}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
[2016-11-21T10:07:44,145][INFO ][o.e.g.GatewayService     ] [YzvQTk1] recovered [0] indices into cluster_state
[2016-11-21T10:07:44,577][INFO ][o.e.h.HttpServer         ] [YzvQTk1] publish_address {127.0.0.1:9200}, bound_addresses {127.0.0.1:9200}, {[::1]:9200}
[2016-11-21T10:07:44,577][INFO ][o.e.n.Node               ] [YzvQTk1] started

Then I run the mongo-connector to connect them with this command: "mongo-connector -m localhost:27017 -t localhost:9200 -d elastic_doc_manager"

The output is: Logging to mongo-connector.log.

So it seems every thing works,but when i go to this link http://localhost:9200/foo/_search?pretty=true&q={'matchAll':{''}}

The output i get is:

"error" : {
"root_cause" : [
  {
    "type" : "index_not_found_exception",
    "reason" : "no such index",
    "resource.type" : "index_or_alias",
    "resource.id" : "foo",
    "index_uuid" : "_na_",
    "index" : "foo"
  }
],
"type" : "index_not_found_exception",
"reason" : "no such index",
"resource.type" : "index_or_alias",
"resource.id" : "foo",
"index_uuid" : "_na_",
"index" : "foo"

}, "status" : 404 }

So the connection is failed, but I don't know why.

I have no plugins in my elastic search and I read somthing about replica set but i"m not sure I know what is it, maybe this is the problam. My mongo data-base is in my internal storage.

Please help me! what am I doing wrong, or what should I do to sync the data between mongo and elasticsearch. I have read about river plugin but I failed to install it, so if that is answer please tell me how to install it on windows, all the tutorials i found were on linux.

I realy need some help i"m stuck for way too long on this.

Community
  • 1
  • 1
Ben Kalifa
  • 17
  • 6

1 Answers1

0

The query mentioned is trying to access foo index in ElasticSearch. Error is due to the index not being present in the same.

P.S.
Query: http ://localhost:9200/foo/_search?pretty=true&q={'matchAll':{''}}

Graham
  • 7,431
  • 18
  • 59
  • 84
manishrw
  • 429
  • 5
  • 17
  • How can i see the whole database in elasticsearch? I want to make sure mongo and elastic search are synced. – Ben Kalifa Nov 21 '16 at 09:57
  • Use command curl 'localhost:9200/_cat/indices?v' to get list of indices or just run the link in the browser. – manishrw Nov 21 '16 at 10:23
  • You can use mongoosastic lib to sync with mongo while using it with NodeJS. Else to manually sync try transporter. This [doc](https://www.linkedin.com/pulse/5-way-sync-data-from-mongodb-es-kai-hao) might come handy. – manishrw Nov 21 '16 at 10:24
  • 1
    [doc](https://www.digitalocean.com/community/tutorials/how-to-sync-transformed-data-from-mongodb-to-elasticsearch-with-transporter-on-ubuntu-14-04) for transporter. – manishrw Nov 21 '16 at 10:26
  • Thank you very much so far. I work in windows, do you have a doc for that? in addition these "health status index uuid pri rep docs.count docs.deleted store.size pri.store.size" are the results that came back as my indexes, how can i see the data inside the index for example the value of "docs.count". and the most important, i want to see somehow my database with the elasticdearch, that is the reason I decided to use mongo-connector. how can do it? is there a query that can show me all of my db? – Ben Kalifa Nov 21 '16 at 10:40