How to retry indexing with elasticsearch-py when using bulk streaming?

Question

I have occasional BulkIndexError when using streaming_bulk helper. Is there any way to configure client to retry on such errors? What is the best way to handle errors when using helpers?

score 2 · Answer 1 · edited May 23 '17 at 12:16

2

Well, you could set up your streaming pipeline in a way, so as to retry on errors (I believe, this will be a BulkIndexError).

The response from streaming_bulk is a tuple that looks like ok, item [see this]. Now, if you wrap the request to streaming_bulk in a try, and in your except, not empty out your list of actions, you could have this try-except block in an infinite loop, and break out when your list of actions is empty.

edited May 23 '17 at 12:16

Community

1
1

answered Sep 18 '16 at 09:59

Debosmit Ray

5,228
2
27
43

Do you have an example? – Oleksiy Sep 20 '16 at 22:32

How to retry indexing with elasticsearch-py when using bulk streaming?

1 Answers1