Can I generate tire+elasticsearch index locally , and push the index file to the server later?

Question

it's consuming memory when generating index.

Can I generate tire+elasticsearch index locally , and push the index file to the server later?

Thanks

Answer 1

I guess you could make good use of the elasticsearch distributed capabilities to achieve what you want.

You can index data on a node (even your local machine), then whenever you add a new node to the same cluster (they need to have the same cluster name) the shards are automatically distributed over the cluster. You can control the way the shards are distributed updating the settings through the update settings api. For example using the cluster.routing.allocation.exclude directive you could tell elastichsearch that you don't want to allocate any shards on a specific node. After the settings update you just have to wait till all the shards have been moved to the other node.

Otherwise you could have a look at the cluster reroute api to move shards around, but beware that after you move a shard elasticsearch always try to evenly balance them unless you disable this automatic behaviour.