mongo-connector is not syncing data from MongoDB to Solr

Question

I need to syncing data from MongoDB to Solr.

when i issue the command

mongo-connector -m dbserver:27018 -n shop -t http://localhost:8983/solr/Search --auto-commit-interval=0 -d solr_doc_manager

the error log is:

2016-12-13 15:21:40,461 [CRITICAL] mongo_connector.oplog_manager:630 - Exception during collection dump
Traceback (most recent call last):
  File "c:\program files (x86)\python35-32\lib\site-packages\mongo_connector\oplog_manager.py", line 583, in do_dump
    upsert_all(dm)
  File "c:\program files (x86)\python35-32\lib\site-packages\mongo_connector\oplog_manager.py", line 567, in upsert_all
    dm.bulk_upsert(docs_to_dump(namespace), mapped_ns, long_ts)
  File "c:\program files (x86)\python35-32\lib\site-packages\mongo_connector\util.py", line 32, in wrapped
    return f(*args, **kwargs)
  File "c:\program files (x86)\python35-32\lib\site-packages\mongo_connector\doc_managers\solr_doc_manager.py", line 288, in bulk_upsert
    batch = list(next(cleaned) for i in range(self.chunk_size))
  File "c:\program files (x86)\python35-32\lib\site-packages\mongo_connector\doc_managers\solr_doc_manager.py", line 288, in <genexpr>
    batch = list(next(cleaned) for i in range(self.chunk_size))
  File "c:\program files (x86)\python35-32\lib\site-packages\mongo_connector\doc_managers\solr_doc_manager.py", line 286, in <genexpr>
    cleaned = (self._clean_doc(d, namespace, timestamp) for d in docs)
  File "c:\program files (x86)\python35-32\lib\site-packages\mongo_connector\oplog_manager.py", line 509, in docs_to_dump
    database, coll = namespace.split('.', 1)
ValueError: not enough values to unpack (expected 2, got 1)
2016-12-13 15:21:40,464 [ERROR] mongo_connector.oplog_manager:638 - OplogThread: Failed during dump collection cannot recover! Collection(Database(MongoClient(host=['dbserver:27018'], document_class=dict, tz_aware=False, connect=True, replicaset='shop'), 'local'), 'oplog.rs')
2016-12-13 15:21:41,460 [ERROR] mongo_connector.connector:304 - MongoConnector: OplogThread <OplogThread(Thread-2, started 4972)> unexpectedly stopped! Shutting down

i don't know where i am wrong. please help me to solve this. thank you.

score 1 · Answer 1 · answered Dec 13 '16 at 11:41

1

From the documentation of the -n parameter:

List of collections to read from MongoDB. Collection names should be given as database_name.collection_name

So you're missing either the database_name or collection_name of the argument, depending on how your MongoDB is structured.

answered Dec 13 '16 at 11:41

MatsLindh

49,529
4
53
84

can i link a full database `shop` in to my solr core? because my collection creation is random. – Telen Dec 13 '16 at 11:55
If `shop` is your database name, you can use `shop.*` to get all collections under `shop`. – MatsLindh Dec 13 '16 at 12:33
i tried this. the mongo-connector started successfully without any error. but the data is not syncing now. – Telen Dec 13 '16 at 13:40

mongo-connector is not syncing data from MongoDB to Solr

1 Answers1