1

I have joined 2 mysql tables by JOIN query and have imported data by DIH in SOLR.

But, Half of records deleted automatically. I come to know about this by reading numDocs , maxDocs & deletedDocs values.

I have checked all fields in tables that I have already defined in data-config.xml as well as in schema.xml files.

What is the problem in this?

Help me in this please.

Abhijit Bashetti
  • 8,518
  • 7
  • 35
  • 47
Mayur Champaneria
  • 239
  • 5
  • 16
  • deletedDocs : I think it must be a replace of an existing document... whats the numDocs, maxDocs & deletedDocs count? – Abhijit Bashetti May 21 '15 at 11:36
  • Num Docs:821 Max Doc:1981 Deleted Docs:1160 – Mayur Champaneria May 21 '15 at 11:48
  • This count is remain same when I delete my core and reimport data – Mayur Champaneria May 21 '15 at 11:49
  • numDocs+DeletedDocs = MaxDocs (821+1160=1981)...numDocs represents the number of searchable documents in the index (and will be larger than the number of XML files since some files contained more than one ). maxDoc may be larger as the maxDoc count includes logically deleted documents that have not yet been removed from the index. if you reindex/repost numDocs will never increase, because the new documents will constantly be replacing the old...try optimizing the index, it might remove the logically deleted documents/indexs. – Abhijit Bashetti May 21 '15 at 11:53

1 Answers1

2

Its a replace of an existing document

numDocs+DeletedDocs = MaxDocs 
 (821+1160=1981)

numDocs represents the number of searchable documents in the index maxDoc may be larger as the numDoc count as it includes logically deleted documents that have not yet been removed from the index.

If you reindex/repost numDocs will never increase, because the new documents will constantly be replacing the old...try optimizing the index, it might remove the logically deleted documents/indexs.

The Unique Key Field

<uniqueKey></uniqueKey>

The <uniqueKey> declaration can be used to inform Solr that there is a field in your index which should be unique for all documents. If a document is added that contains the same value for this field as an existing document, the old document will be deleted.

Abhijit Bashetti
  • 8,518
  • 7
  • 35
  • 47
  • Thanks for this. But, my question is why its been deleted? My mysql data is 1981 but why it only allows 821 docs? and why its deleted ? – Mayur Champaneria May 21 '15 at 12:18
  • whats the uniqueKey in your schema.xml? and how many entries your mysql database has for that? I mean is the primary key in the mysql as well? – Abhijit Bashetti May 21 '15 at 12:21
  • Main **User** table has **ID** unique key. Another table has same unique key but **user.id=tal.userid** – Mayur Champaneria May 21 '15 at 12:35
  • Today I have joined 3 tables **user**, **country** & **cities**. **User** table has uniqueKey **ID**, **Country** table has uniqueKey **country_id** & **Cities** table has uniqueKey **city_id** and its joined perfectly and by importing that data It had not deleted any data. So i come to know that all joined tables have different **uniqueKey** that works perfectly. But in my second **JOIN**, I have same uniqueKey in 2 different tables and It have remains same problem that deletes my docs. So how do i define 2 uniqieKeys for 2 different tables in **Schema.xml** file ?? – Mayur Champaneria May 22 '15 at 06:55
  • whats the in defined in your schema.xml? – Abhijit Bashetti May 22 '15 at 06:58
  • when joining 2 tables **User -> id**, **Country -> country_id**, **Cities -> city_id** there is no problem. But joining these **User -> id**, **User_works -> id** 2 tables then it does problem – Mayur Champaneria May 22 '15 at 07:16
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/78497/discussion-between-abhijit-bashetti-and-mayur-champaneria). – Abhijit Bashetti May 22 '15 at 07:18