1

I'm trying to benchmark a simple Tornado Server made using tornado.httpserver and I get the following error, when using apache bench with 1000 or more concurrent connections on a Solaris machine.

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/tornado-2.3-py2.7.egg/tornado/iostream.py", line 304, in wrapper
  File "/usr/local/lib/python2.7/site-packages/tornado-2.3-py2.7.egg/tornado/httpserver.py", line 227, in _on_headers
LookupError: unknown encoding: latin1

I believe this is due to a race condition that leaves the python codec table in an invalid state. My understanding is that latin1 is included in the python standard library, so it should be present as long as nothing is modifying pythons codec table.

I've looked through the tornado src and it doesn't seem to modify the codecs table, so im wondering if anyone out there knows of other methods or issues that may cause this or a similar problem in a multiprocess python application.

Thanks!!

Link to server code: https://www.dropbox.com/s/xrgj59sv0y5y31n/server_ioloop.py

RickyG
  • 11
  • 3
  • Looks like it's a Solaris issue: https://groups.google.com/forum/#!msg/python-tornado/qmu8Yv6sW_I/47RIUS8HLPcJ%5B1-25%5D – Cole Maclean Aug 15 '12 at 07:09
  • Thanks for the link, however, the person that started the thread has seen the same issue on OSX Lion. I've also replaced the default ioloop implementation with an implementation using event ports and devpoll and have still seen the issue, so I think it may be unrelated to select running out of file descriptors on solaris. – RickyG Aug 15 '12 at 17:59
  • I've found the following workaround for this issue. Changing the following line in httpserver._on_headers from:`data = native_str(data.decode('latin1'))` to `data = native_str(codecs.latin_1_decode(data)[0])` fixes the issue for me, but it would be nice to know the underlying cause. – RickyG Aug 16 '12 at 18:04

1 Answers1

0
import pandas as pd
import numpy as np
d = pd.read_csv('/content/zomato.csv',encoding='latin-1')
d.head(1)
S.B
  • 13,077
  • 10
  • 22
  • 49
AKSHAY KUMAR
  • 1