1

In order to ingest data from a single database I usually implement a process to load it through DataImportHandler. It is pretty easy to setup, appears to be very efficient in terms of time to load and it works really well for me. It is easy to load, reload and keep it up to date without having to employ any extra effort.

Now I have a different scenario. I need to ingest data from multiple databases to feed a single solr collection however it appears that DataImportHandler might not the way to go but I'm unsure.

1. Is there any way to ingest data from multiple databases into a single solr collection using DIH?

2. If not possible; what is the best strategy to achieve it employing the less possible effort?

user4157124
  • 2,809
  • 13
  • 27
  • 42
user1778669
  • 13
  • 3

1 Answers1

0

Yes, It is possible to have more than one datasources for a configuration. In order to configure an extra datasource , add an another 'dataSource' tag to the data-config.xml.

There is an implicit attribute "name" for a datasource. If there are more than one, each extra datasource must be identified by a unique name 'name="datasource-2"' .

It could be something like below.

<dataSource type="JdbcDataSource" name="ds-1" driver="com.mysql.jdbc.Driver" url="jdbc:mysql://db1-host/dbname" user="db_username" password="db_password"/>
<dataSource type="JdbcDataSource" name="ds-2" driver="com.mysql.jdbc.Driver" url="jdbc:mysql://db2-host/dbname" user="db_username" password="db_password"/>

These can be used in the entities as below.

<entity name="one" dataSource="ds-1" ...>
   ..
</entity>
<entity name="two" dataSource="ds-2" ...>
   ..
</entity>
Abhijit Bashetti
  • 8,518
  • 7
  • 35
  • 47
  • So whenever I need to populate the index with data from different datasources, the solr way to get this to work is to add a new entity within the document pointing to the desired datasource and each record retrieved by each entity query will become a record within the collection. Did I get it right? Thanks again for your help! – user1778669 Mar 27 '20 at 05:24
  • node consist of entities...here all entities data will become one record – Abhijit Bashetti Mar 27 '20 at 05:27