ElasticSearch Bulk Indexing

Question

I am using version 2.1 and the 2.0 (alpha) client with the Nest C# client... I am attempting to bulk insert some records, however, the server is returning an error. I am utilizing the client on a windows client speaking to a Linux server (I don't believe this should matter however).

    public static void AddQuestionsToElasticSearch()
    {
        var es = new ElasticsearchClient(new ConnectionConfiguration(
            new Uri("http://elasticserver:9200"))
        );

        var foobars = new FooBar().Parse();
        var descriptor = new BulkDescriptor();

        descriptor.CreateMany<FooBar>(foobars, (bd, q) => bd.Id(q.Id.ToString()).Index("foobars"));

        Console.WriteLine($"Inserting foobars into ES...");
        var sw = new Stopwatch();
        sw.Start();

        var result = es.Bulk<FooBar>(descriptor);

        sw.Stop();
        Console.WriteLine($"Finished inserting foobars {GetTimeTaken(sw.Elapsed)}");
    }

Update - Error Info

The error I'm getting is in the response returned from the Bulk() method... the two properties on the BulkResponse returned are:

OriginalException: "The remote server returned an error: (400) Bad Request"

ServerError.Error: "Validation Failed: 1: no requests added"

Are you sure `new FooBar().Parse();` returns actually any items? — Rob, Jan 12 '16 at 09:08
Yes. It does return valid data.... to emulate the data source simply new []{new FooBar(), new FooBar(), .... }; — bbqchickenrobot, Jan 12 '16 at 15:13

score 5 · Accepted Answer · edited Sep 12 '18 at 23:37

5

You've made a simple mistake - you're using the low level ElasticsearchClient from Elasticsearch.Net to make the request, but sending it a strongly typed bulk request from NEST. To rectify is simple enough, you just need to use the ElasticClient from NEST

public static void AddQuestionsToElasticSearch()
{
    var es = new ElasticClient(new Uri("http://elasticserver:9200"));

    var foobars = new FooBar().Parse();
    var descriptor = new BulkDescriptor();

    descriptor.CreateMany<FooBar>(foobars, (bd, q) => bd.Id(q.Id.ToString()).Index("foobars"));

    Console.WriteLine($"Inserting foobars into ES...");
    var sw = new Stopwatch();
    sw.Start();

    var result = es.Bulk(descriptor);

    sw.Stop();
    Console.WriteLine($"Finished inserting foobars {GetTimeTaken(sw.Elapsed)}");
}

ElasticClient from NEST is the high level client and uses ElasticsearchClient from Elasticsearch.Net under the covers.

edited Sep 12 '18 at 23:37

Frederik Struck-Schøning

12,981
8
59
68

answered Jan 12 '16 at 21:56

Russ Cam

124,184
33
204
266

Thanks @Russ - Will try that now. I'm brandnew to ES so forgive me but I have another minor question - should I be using CreateMany<>() or IndexMany()? Couldn't tell the diff from the docs (probably missed somen). – bbqchickenrobot Jan 12 '16 at 22:59
4

`CreateMany()` will fail for any individual create call where a document already exists in the index with the same type and `Id`, whereas `IndexMany` will insert the document or overwrite an existing document with the same type and `Id` (unless you've specified type, it will be inferred from T). Essentially, `CreateMany` is useful in situations where you want to utilize Optimistic Concurrency Control. See https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html – Russ Cam Jan 12 '16 at 23:16

ElasticSearch Bulk Indexing

1 Answers1