0

I've been chained dataimporthandler with schemaless mode. add config param "add-unknown-fields-to-the-schema" to dataimporthandler as seen below and configure everything. Dataimporthandler imports data and works well but solr can't add unknown field to schema. It only works for existing fields. Here is dataimporthandler answer to request.

{ "responseHeader": { "status": 0, "QTime": 0 }, "initArgs": [ "defaults", [ "update.chain", "add-unknown-fields-to-the-schema", "config", "db-data-config.xml" ] ], "command": "status", "status": "idle", "importResponse": "", "statusMessages": { "Total Requests made to DataSource": "1", "Total Rows Fetched": "273", "Total Documents Processed": "273", "Total Documents Skipped": "0", "Full Dump Started": "2016-06-21 13:29:48", "": "Indexing completed. Added/Updated: 273 documents. Deleted 0 documents.", "Committed": "2016-06-21 13:29:49", "Time taken": "0:0:0.543" } }

You can see that my configuration is alright. below is solrconfig file. 

<requestHandler name="/dataimport"   class="solr.DataImportHandler">
    <lst name="defaults">
         <str name="update.chain">add-unknown-fields-to-the-schema</str>
         <str name="config">db-data-config.xml</str>
   </lst>

I want to be able to add unknown fields from database.

Halis Yılboğa
  • 850
  • 10
  • 12

1 Answers1

2

DataImportHandler precedes schemaless mode. If you are trying to auto-map it (column names to field names), it is implemented to check whether the field exist in schema to know how to map it. That way you were able to define only the fields you wanted in the schema and it would ignore the rest. It does not know that schemaless mode can compensate for that, nor does it provide any alternative mechanism for field exclusion (which you would need if accept every field was the default).

If you define the field mapping explicitly in the DIH, it should work.

If you truly do not know what field you will have, you might be better off generating a CSV, XML or JSON file from the database content and pushing it to Solr, instead of pulling it with DIH.

Alexandre Rafalovitch
  • 9,709
  • 1
  • 24
  • 27
  • DIH also include some chain processors and if it except schema generator it should use it. I know I have option to export it some where but it doesn't feet my needs. – Halis Yılboğa Jun 23 '16 at 08:55