incomplete read error with python google app engine, using webapp2, WSGI

Question

I get this internal server error whenever I run the code. I'm trying to save the content of a page I retrieved, into datastore by splitting it into a list of words. basically it's a crawler, and has to save each word and the url of the page as a pair.

here is the traceback:

Traceback (most recent call last):
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1535, in __call__
rv = self.handle_exception(request, response, e)
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1102, in __call__
return handler.dispatch()
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "C:\Users\Samson\Documents\searchengineapp\searchapp\crawlHandler.py", line 26, in post
if spider.crawl_web():
File "C:\Users\Samson\Documents\searchengineapp\searchapp\crawl\crawler.py", line 52, in crawl_web
add_page_to_index(page, content)
File "C:\Users\Samson\Documents\searchengineapp\searchapp\crawl\indexin.py", line 42, in add_page_to_index
add_to_index(word, url)
File "C:\Users\Samson\Documents\searchengineapp\searchapp\crawl\indexin.py", line 29, in add_to_index
index.put()
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\ext\ndb\model.py", line 3432, in _put
return self._put_async(**ctx_options).get_result()
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\ext\ndb\tasklets.py", line 326, in get_result
self.check_success()
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\ext\ndb\tasklets.py", line 369, in _help_tasklet_along
value = gen.throw(exc.__class__, exc, tb)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\ext\ndb\context.py", line 810, in put
key = yield self._put_batcher.add(entity, options)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\ext\ndb\tasklets.py", line 369, in _help_tasklet_along
value = gen.throw(exc.__class__, exc, tb)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\ext\ndb\context.py", line 343, in _put_tasklet
keys = yield self._conn.async_put(options, datastore_entities)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\ext\ndb\tasklets.py", line 455, in _on_rpc_completion
result = rpc.get_result()
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\api\apiproxy_stub_map.py", line 613, in get_result
return self.__get_result_hook(self)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\datastore\datastore_rpc.py", line 1881, in __put_hook
self.check_rpc_success(rpc)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\datastore\datastore_rpc.py", line 1371, in check_rpc_success
rpc.check_success()
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\api\apiproxy_stub_map.py", line 579, in check_success
self.__rpc.CheckSuccess()
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\api\apiproxy_rpc.py", line 157, in _WaitImpl
self.request, self.response)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\ext\remote_api\remote_api_stub.py", line 201, in MakeSyncCall
self._MakeRealSyncCall(service, call, request, response)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\ext\remote_api\remote_api_stub.py", line 227, in _MakeRealSyncCall
encoded_response = self._server.Send(self._path, encoded_request)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\appengine_rpc.py", line 394, in Send
response = f.read()
File "C:\Python27\lib\socket.py", line 351, in read
data = self._sock.recv(rbufsize)
File "C:\Python27\lib\httplib.py", line 549, in read
return self._read_chunked(amt)
File "C:\Python27\lib\httplib.py", line 603, in _read_chunked
raise IncompleteRead(''.join(value))
IncompleteRead: IncompleteRead(52 bytes read)

I honestly don't know where the problem is coming from. please I need help. Thanks

The error is a big hint have you looked into the meaning of that error ? Just posting a stack trace with out any supporting code is going to make it difficult for anyone to tell you what the problem is. — Tim Hoffman, Nov 21 '15 at 00:22
IncompleteRead is indeed a strong hint -- and http://stackoverflow.com/questions/14149100/incompleteread-using-httplib gives you a thorough analysis and a couple of (dubious but practical) workarounds. Note that it's not an app engine problem -- rather the problem is with the server you're getting data from, and Python's standard httplib being insufficiently paranoid about server misbehavior. — Alex Martelli, Nov 21 '15 at 16:38

incomplete read error with python google app engine, using webapp2, WSGI

0 Answers0