6

The latest documentation (8.8) for Apache Solr says that the Data Import Handler is deprecated is going to be removed in future versions. It only lists a third-party plugin maintained on Github, and no other native alternative to import data from relational databases. Am I missing something or has Solr dropped native support for this?

U754V
  • 75
  • 2
  • 9
  • There will be no native support; the DataImportHandler has been moved to the separate repository and will be developed externally from Solr in the future. If you still need it, you can install it from that repository as a contrib module. That being said, you should really consider alternatives if you want to get better performance and far more flexibility. – MatsLindh Feb 10 '21 at 13:47
  • Could you tell me what some of those alternatives could be? – U754V Feb 11 '21 at 03:48
  • 1
    Usually you'll write the code yourself in your language of choice - using a Solr client and the preferred library for connecting to your data source. That way you can easily preprocess the content in any way necessary, as well as parallelize indexing by submitting your content to more than one single node in a cluster setup (since DIH only runs on a single node). You can also run multiple copies of your program or threads to process content on multiple cores on the computer doing the retrieval and submission, allowing for great speedups. – MatsLindh Feb 11 '21 at 09:10
  • @MatsLindh Can you share the separate repository URL? Does it support mssql? how to build and attach to Solr 9? Are docs ready? – Krunal May 27 '21 at 11:42
  • Development happens in https://github.com/rohitbemax/dataimporthandler as far as I know. I'd think it'd support anything that the old version supported through JDBC, but the details page says only the MariaDB driver works for now. I'd plan on deprecating any usage you have of DIH at the moment and instead write a light weight indexing script that can properly parallelize imports. So for the two other questions; as far as I can tell there is no new documentation except for the README in the repository. – MatsLindh May 27 '21 at 11:54

3 Answers3

2

Data import handler will still be there, but as a contrib package.

I'll do what I did before data import handler existed and write a separate program. I prefer Python, where reading a database and sending JSON updates is pretty simple.

Be sure to batch the updates.

Walter Underwood
  • 1,201
  • 9
  • 11
1

Alternative solr DIH https://github.com/saro-lab/solr-db-importer

I made an alternative program to DIH and released the source code and manual.

PARK Yong Seo
  • 11
  • 1
0

I want to implement your solr-sb-importer But it seems stay stuck at running. I suspect the loading of the schema is not working http://localhost:8983/solr/succesboeken_sb/sb-schema I get: Exception in thread "main" org.springframework.web.client.HttpClientErrorException$NotFound: 404 Not Found: "

Searching for Solr?
You must type the correct path.
Solr will respond.

"

while excecuting: java -jar solr-db-importer-1.4.jar

I am using Solr 9.1.0

Have you encountered this before?

Hansl Kerstens
  • 1
  • As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Jan 08 '23 at 01:58