Elasticsearch 5.0 batch document ingestion with pipeline

Question

I'm upgrading elasticsearch 2.1 to 5.0. I used a document ingestion plugin for 2.1 which works most excellently with a batch ingest.

For 5.0 I've installed the ingest-attachment in 5.0.

I've created a pipeline:

{
  "attachment": {
  "description": "Attachment ingestion",
  "processors": [
    {
      "attachment": {
        "field": "data"
      }
    }]
  }
}

The problem is, with the previous plugin I was ingesting using bulk, but I can't find in the documentation how to do a bulk ingest whilst utilising a pipeline?

You'd do the exact same way using bulk, the only change is that you need to specify the pipeline name in your bulk call. — Val, Dec 01 '16 at 13:55

score 0 · Accepted Answer · edited May 08 '17 at 15:56

0

You're right @val, it was a case of rtm!

return new Promise((resolve, reject) => {
  client.bulk({
    body: documentArray,
    pipeline: 'attachment',//this worked
  }, (error, response) => {
    if (error) {
      return reject(error.message);
    }

    resolve(response);
  });
});

edited May 08 '17 at 15:56

Funk Forty Niner

74,450
15
68
141

answered Dec 01 '16 at 13:58

Mark Robson

1,298
3
15
40

Glad you figured it out! – Val Dec 01 '16 at 13:59

Elasticsearch 5.0 batch document ingestion with pipeline

1 Answers1