AuthorizationException: on Scan

Question

I am trying to run what seems like a very simple Scan but I'm running into an odd exception. It seems like the generator created by the scan hits an exception and I have no idea why. I tried wrapping the interior of my for loop in a try/except but it still throws the exception so I'm guessing it dies when hitting es. Here is what I've got if you can help identify what is going on:

old_stuff_query = {"query": {"filtered": {
    "filter": {
        "range": {
            "expire_on": {'lte': datetime.datetime.now() - datetime.timedelta(weeks=53)}}

    }}}}

scanResp = scan(client=es, query=old_stuff_query, scroll="1m", index="myIndex", doc_type="myDoc")
counter = 0
for resp in scanResp:
    try:
        print("YAH")
    except:
        print("BOO")

But I get the following:

    for resp in scanResp:
  File "/home/will/.virtualenvs/side_project/local/lib/python2.7/site-packages/elasticsearch/helpers/__init__.py", line 314, in scan
    client.clear_scroll(body={'scroll_id': [scroll_id]}, ignore=(404, ))
  File "/home/will/.virtualenvs/side_project/local/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/home/will/.virtualenvs/side_project/local/lib/python2.7/site-packages/elasticsearch/client/__init__.py", line 688, in clear_scroll
    params=params, body=body)
  File "/home/will/.virtualenvs/side_project/local/lib/python2.7/site-packages/elasticsearch/transport.py", line 327, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
  File "/home/will/.virtualenvs/side_project/local/lib/python2.7/site-packages/elasticsearch/connection/http_requests.py", line 84, in perform_request
    self._raise_error(response.status_code, raw_data)
  File "/home/will/.virtualenvs/side_project/local/lib/python2.7/site-packages/elasticsearch/connection/base.py", line 113, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.AuthorizationException: <exception str() failed>

The most frustrating part is this happens completely at random. It has error'd out after less than 10k docs and more than 250k docs. Any help would be awesome, thanks!

I ran into the same problem, and it seems to be unicode handling that is causing it, probably because of Python2/python3 compatibility I added (with future). In pure python2.7 it seems to works. — Pierre.Sassoulas, Sep 06 '17 at 10:07

score 3 · Answer 1 · edited Jun 21 '18 at 07:21

So I also had the same error and found the reason: If you create a own class with special __str__, and in this method or a other called method throw an error with the instance from what it tries to make a string, the error message will be swap out with '<exception str() failed>':

>>> class X:
...     def __str__(self):
...         raise ValueError(self)
... 
>>> print(X())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in __str__
ValueError: <exception str() failed>

In your case it seems to be sloppy programming on api side.

AuthorizationException: on Scan

1 Answers1