0

I have to import some mdb files to solr. some of mdb files are indexed well as document but there are others don't. I use Solr 4.10.0 and ver. 2.0.9 The follwoing is a screen shot from the log:

enter image description here

For some missing fields values (in the screen shot case 6 fields) I have set onError="continue" in the dataimport-config:

<document>
    <entity name="Book" dataSource="a" query="select bkid AS id, bkid AS BookID,bk AS BookTitle, betaka AS BookInfo, cat as cat from 0bok WHERE bkid = 29435">
                   <field column="id" name="id"/>
                   <field column="BookID" name="BookID"/>
                   <field column="BookTitle" name="BookTitle"/>
                   <field column="cat" name="cat"/>    
        <entity name="Category" dataSource="a" query="select name as CatName, catord as CatWeight, Lvl as CatLevel from 0cat where id = ${Book.CAT}">
                    <field column="CatName" name="CatName"/>
                    <field column="CatWeight" name="CatWeight"/>
                    <field column="CatLevel" name="CatLevel"/>
        </entity>

        <entity name="Pages" dataSource="a1" onError="continue" query="SELECT nass AS PageContent, page AS PageNum FROM book ORDER BY page">
                <field column="PageContent" name="PageContent"/>
                <field column="PageNum" name="PageNum"/>
                <entity name="Titles" dataSource="a1" onError="continue" query="SELECT * FROM title WHERE id = ${Pages.PAGENUM} ORDER BY sub">
                        <field column="ID" name="TitleID"/>
                        <field column="TIT" name="PageTitle"/>
                        <field column="SUB" name="TitleWeight"/>
                        <field column="LVL" name="TitleLevel"/>
                </entity>
        </entity>


    </entity>
  </document>

This is a screen shot for the table regarded in the database with the 6 undefined data fields:

enter image description here

At the end of dataimporting for this mdb file I got the following response:

Last Update: 09:12:04 Requests: 31,952, Fetched: 78,980, Skipped: 0, Processed: 0 Started: 18 minutes ago

Which it is shown that 0 processed!

There are other mdb files are proceed i.e 1 processed is generated in the response but I have got the folwing errors in the log:

10/7/2014 9:28:08 AM ERROR SolrWriter Exception while solr commit.

this writer hit an OutOfMemoryError; cannot commit...

and

SolrIndexWriter Error closing IndexWriter this writer hit an

OutOfMemoryError; cannot flush...

How could I solve this issue? and why Solr requests and fetched all this records and then process and index none?!

SaidbakR
  • 13,303
  • 20
  • 101
  • 195
  • Look into your solr config and make sure auto commit is set to some reasonable limits. See `Update Handler Section` here http://wiki.apache.org/solr/SolrConfigXml. Try indexing fewer docs and see if they are indexed properly by Solr. – arun Oct 13 '14 at 23:32
  • @arun I tried and another approach and it has been succeeded. In Tomcat configuration panel, I increased the maximum memory limit from 512 MB to 1024 MB. – SaidbakR Oct 14 '14 at 17:41
  • 1
    Sure, if your entire index fits in 1 GB, then that would work too. But to scale properly, it is better to use auto commit. Notice that you need not open a new searcher after auto commit. Just set `false` in your auto-commit settings and your searcher will be updated only after a commit is issued. – arun Oct 14 '14 at 17:45

0 Answers0