22

Is it possible to create new cores in SOLR via HTTP? I can't seem to find a definitive answer to what seems like a very simple question...

Each core I want to create will have the same schema/configuration (multi-tenant SaaS architecture).

I found this page:

Which makes me think it's possible, but it's not very clear on specifically what each parameter I'm supposed to pass in actually is:

  • instanceDir - is this the path to where I want the new core to reside, or where the existing "template" core resides? is it a full file path, or a relative one? relative to what?

  • config - is this a full file path to the existing config file? or a relative one? relative to what?

  • schema - same as above

  • dataDir - is this the data dir of the existing core, or the new one? full file path? relative? relative to what? does it already have to exist, or will SOLR create it for me?

Keith Palmer Jr.
  • 27,666
  • 16
  • 68
  • 105
  • 2
    In my experience (and confirmed on the solr-user listserv), you cannot create a new core over HTTP. Yes, there's an API for it, but the API assumes you've already created the conf directory and put the right files in it, which cannot be done over HTTP: you must have file system acccess. The "config sets" feature may be a way around this, but it didn't work as advertised for me and when I asked on solr-user, the response was, "it's new and still buggy, so we're not surprised that it doesn't work." – Mark E. Haase Mar 27 '15 at 17:59
  • 1
    This does work, I have marked the working answer below as accepted now. We've been using this in production for quite some time now with great success. – Keith Palmer Jr. Mar 27 '15 at 18:33
  • Keith, are you using the "Solr Cloud" configuration (i.e. with ZooKeeper) or a single-node Solr instance? – Mark E. Haase Mar 28 '15 at 15:43
  • Single node instance. – Keith Palmer Jr. Mar 29 '15 at 11:36
  • @Keith. I seems as if I'm facing now the same problem as you. Can you, please, share with us the right answer: `http://what should be here???` Since buddy86's answer with `http://'localhost':8983...` does not look right at first glance. – Jacobian Jun 24 '15 at 18:12
  • And besides I followed buddy86 solution dot by dot and got exactly the same error message as you had reported earlier: `"Error CREATEing SolrCore..."`. So, it seems as if the answer which is marked as correct, in fact, needs some tuning. – Jacobian Jun 24 '15 at 18:15
  • The exact URL we hit is: http://(the hostname):8080/solr/admin/cores?action=CREATE&name=the_new_name&instanceDir=the_new_name&dataDir=data&config=solrconfig.xml&schema=schema.xml&loadOnStartup=false&transient=true This is the same as what the correct answer is indicating, and works fine for us. Obviously you'll have to adjust your hostname, port, and core/dir name to what's appropriate for your environment. If you're having trouble, I'd suggest that you try posting the URL you're using, your configuration, and the error message you're getting so that someone might be able to help you. – Keith Palmer Jr. Jun 24 '15 at 19:32
  • @KeithPalmerJr. But have you previously placed the file `solrconfig.xml` into the right location, `data/the_new_name/solrconfig.xml`, before you hit the URL? In other words, as well as HTTP access to the solr instance, do I also require direct write-access to the file system? – Daniel Earwicker Nov 29 '19 at 11:07
  • @DanielEarwicker We do not do anything except hit that URL via HTTP. No direct filesystem access required. – Keith Palmer Jr. Nov 29 '19 at 15:54

3 Answers3

11

Yes, you can create the Solr cores via HTTP. You have found the correct URL (https://wiki.apache.org/solr/CoreAdmin) to look into. All the above parameters are optional.

instanceDir - This is the path where your new core will be created. It'll create the folder structure provided by you, under "/example/solr/". If you don't provide this parameter, it'll automatically create a new core (with the collection no) like collection1.

config - If you want to have a different solrconfig.xml for the new core, then provide this. Otherwise, it'll share the existing core's (collection1) config by default.

schema - If you want to have a different schema.xml for the new core, then provide this. Otherwise, it'll share the existing core's (collection1) schema by default.

dataDir - This is the path where your new core's data will be stored. It'll create the folder structure provided by you under the new core folder.

If you really don't want some different configuration for your new core, follow the sample HTTP URL

http://localhost:8983/solr/admin/cores?action=CREATE&name=core_name&numShards=2&replicationFactor=2

Hope this will help.

Keith Palmer Jr.
  • 27,666
  • 16
  • 68
  • 105
buddy86
  • 1,434
  • 12
  • 20
  • So that collection number is just an auto-incrementing value? In the explanation of instanceDir, it sounds like you're saying that "collection1" is the new core name. But then in config comments, you refer to "collection1" as the existing core. Am I just confused on that, or? Is dataDir like a base level directory, and it creates a directory for the core in that directory, or? – Keith Palmer Jr. Feb 07 '14 at 14:34
  • Running the sample URL you provided does not work. It says "Error CREATEing SolrCore 'new_core_name': Unable to create core: new_core_name – Keith Palmer Jr. Feb 07 '14 at 14:36
  • 1
    The logs show: Failed to load file /usr/share/solr/new_core_name/solrconfig.xml. So it looks like it's not really trying to use any existing solrconfig.xml. – Keith Palmer Jr. Feb 07 '14 at 14:37
  • "collection1" is your existing solr core. If a solr core is already running in a solr instance, then only you'll be able to create another Solr core. Yes, collection number is the auto-increment value, if you are not providing the collection name explicitly. If there is a core present named collection1, the next collection name will be collection2 by default. collection2 will share the configuration of collection1 by default. – buddy86 Feb 10 '14 at 07:24
  • First, when you start a solr instance, it creates a core "collection1" by default. Then try creating new solr cores. – buddy86 Feb 10 '14 at 07:27
  • 7
    @buddy86 Are these instructions for "Solr Cloud" or for single-node Solr? Using your example URL on single-node Solr doesn't work: `Error CREATEing SolrCore 'core_name': Unable to create core [core_name] Caused by: Can't find resource 'solrconfig.xml' in classpath or '/var/solr/data/core_name/conf'` – Mark E. Haase Mar 28 '15 at 15:46
  • 1
    @KeithPalmerJr. Do you know of any reason why the instructions might not work? They don't appear to work as an independent way of creating a new core over HTTP - you have to put a `solrconfig.xml` file into a directory by some other means first or you get the same error described above by Mark E Haase. – Daniel Earwicker Nov 29 '19 at 10:54
  • @DanielEarwicker it appears you can upload configsets (schema and config.xml files) via the ConfigSet API https://solr.apache.org/guide/8_4/configsets-api.html#configsets-api – Chris Seufert Jul 13 '21 at 00:47
8

I had the exact same issue and I was not able to make it work from the description on https://wiki.apache.org/solr/CoreAdmin#CREATE

What solved it for me was updating to Solr 5 and using a configSet (introduced in 4.8) which defines the shared solrconfig/schema. The following URL can be used to create several cores with the same configuration:

http://localhost:8983/solr/admin/cores?action=CREATE&name=new_core&configSet=basic_configs

The 'basic_configs' is the name of a configSet that comes with the Solr 5 download (solr-5.0.0/server/solr/configsets/basic_config). The URL worked out-of-the-box after starting the Solr server.

https://cwiki.apache.org/confluence/display/solr/Config+Sets

molholm
  • 1,992
  • 3
  • 25
  • 29
  • 2
    Did you use the installation script for Solr 5? I'm encountering the same problem as you but using the `configSet` is not working for me. I'm using the exact same URL as you (my Solr is also on `localhost:8983`) but I get an error: `Could not load configuration from directory /var/solr/data/configsets/basic_configs`. I can't figure out why it's looking in `/var/solr` — my `solr.install.dir` is `/opt/solr` but my `solr.solr.home` is `/var/solr`. Did you see this same problem? – Mark E. Haase Mar 26 '15 at 06:11
  • No i did not, my test setup is running on windows. I just unpacked the zip and started the server. As far as I know the configsets folder should be in the solr home (./server/solr/ in the zip), so it does not seem wrong that it is looking in /var/solr.. but I do not know why it is adding the data folder in-between. Is the configsets located in ./opt/solr/... ? – molholm Mar 27 '15 at 12:07
  • 1
    Yes, my config sets are in `/opt/solr/server/solr/...` I posted this on the solr-user list and the reply was basically, "config sets are new and still buggy." One person said that config sets don't even apply when using a single node of Solr (as opposed to "solr cloud"), but nobody can point at documentation that corroborates that belief. I just gave up and added `solr create -c foo` to my installation manual: admins have to do it themselves. I'm still jealous that it works in your case, though... – Mark E. Haase Mar 27 '15 at 17:56
  • 3
    For those who are still looking for answer, actually, this one is correct. I wonder why buddy86's wrong answer was accepted. It may mislead other newbies like me. – Jacobian Jun 24 '15 at 18:30
  • For those where API request is still claiming "Could not load configuration from directory /var/solr/data/configsets/basis_configs", see https://cwiki.apache.org/confluence/display/solr/Format+of+solr.xml#Formatofsolr.xml-TheElement and add `configSetBaseDir` to solr.xml, restart Solr and try adding core again. That worked for me. – rabudde Mar 10 '16 at 10:10
  • Two updates to this: first, if you may try this, note that the congSets argument in that URL is CASE-sensitive--so using configsets will cause the same failing message as if you didn't set it at all. Second, in later Solr versions, there is no built-in configset called basic_config, but there is one called _default (with that leading underscore). Use that. You will see it in the configsets folder, as discussed above. – charlie arehart Aug 25 '22 at 19:12
0

Copy the configsets folder recursively to var/solr/data/your_new_core_name then add the your_new_core_name from the admin panel

The configsets path usually should be here.

/solr-8.6.3/server/solr/configsets/_default/conf

Shankar Narayana Damodaran
  • 68,075
  • 43
  • 96
  • 126
zawhtut
  • 8,335
  • 5
  • 52
  • 76