0

System Info

$ cat /etc/centos-release
CentOS Linux release 7.6.1810 (Core)

$ uname -a
Linux lb-cam-11 3.10.0-1160.36.2.el7.x86_64 #1 SMP Wed Jul 21 11:57:15 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$ python --version
Python 3.7.1

$ openssl version
OpenSSL 1.0.1h 5 Jun 2014

>>> cherrypy.__version__
'18.1.0'

>>> import ssl
>>> [(ele, sys.modules[ele]) for ele in sys.modules if 'ssl' in ele]
[('ssl',
  <module 'ssl' from '/tools/oss/packages/x86_64-centos7/python/3.7.1/lib/python3.7/ssl.py'>),
 ('_ssl',
  <module '_ssl' from '/tools/oss/packages/x86_64-centos7/python/3.7.1/lib/python3.7/lib-dynload/_ssl.cpython-37m-x86_64-linux-gnu.so'>)]
>>>

>>> cheroot.__version__
'6.5.4'

Hi

I am attempting to set up a cherrypy server that uses https.

The ssl related cherrypy configuration I am using looks like:

server_config={
    'server.socket_host': chrysocketserver,
    'server.socket_port':int(chrysocketport),
    'server.ssl_module':'pyopenssl',
    'server.ssl_certificate':sslcertificate,
    'server.ssl_private_key':privatekey,
    'server.ssl_certificate_chain':certificatechain,
}
cherrypy.config.update(server_config)

With small pages this looks like it works fine, but when I try to load a large page (I have not yet worked out at what point the problem starts occurring) the cheroot back-end (I know nothing about this) throws an exception that looks like:

Traceback (most recent call last):
  File "/tools/oss/packages/x86_64-centos7/python/3.7.1/lib/python3.7/site-packages/cheroot/server.py", line 1252, in communicate
    req.respond()
  File "/tools/oss/packages/x86_64-centos7/python/3.7.1/lib/python3.7/site-packages/cheroot/server.py", line 1056, in respond
    self.server.gateway(self).respond()
  File "/tools/oss/packages/x86_64-centos7/python/3.7.1/lib/python3.7/site-packages/cheroot/wsgi.py", line 147, in respond
    self.write(chunk)
  File "/tools/oss/packages/x86_64-centos7/python/3.7.1/lib/python3.7/site-packages/cheroot/wsgi.py", line 228, in write
    self.req.write(chunk)
  File "/tools/oss/packages/x86_64-centos7/python/3.7.1/lib/python3.7/site-packages/cheroot/server.py", line 1112, in write
    self.conn.wfile.write(chunk)
  File "/tools/oss/packages/x86_64-centos7/python/3.7.1/lib/python3.7/site-packages/cheroot/makefile.py", line 36, in write
    self._flush_unlocked()
  File "/tools/oss/packages/x86_64-centos7/python/3.7.1/lib/python3.7/site-packages/cheroot/makefile.py", line 45, in _flush_unlocked
    n = self.raw.write(bytes(self._write_buf))
  File "/tools/oss/packages/x86_64-centos7/python/3.7.1/lib/python3.7/socket.py", line 607, in write
    return self._sock.send(b)
  File "/tools/oss/packages/x86_64-centos7/python/3.7.1/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1729, in send
    self._raise_ssl_error(self._ssl, result)
  File "/tools/oss/packages/x86_64-centos7/python/3.7.1/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1616, in _raise_ssl_error
    raise WantWriteError()
OpenSSL.SSL.WantWriteError

(Some of the page is loaded, but not all)

Has anyone seen this issue before ? and know what might be causing it ? On the surface it looks like the socket being written to becomes unwritable (because the buffer is full ?) and when it encounters the SSL.WantWriteError it simply raises an exception (rather than waiting and trying again). But I am probably being naive in my reading of it.

Does anyone know I how I might get round this problem ?

You're incorrectly assuming that the stdlib ssl

Yes post-posting I noticed this and replied as an Answer (repeated below) which got deleted (probably because it wasn't actually an answer) - the actual version used is 1.0.2m:

Apologies - the ssl version I posted was simply the output of which openssl, however it the python is using a later version (1.0.2m):

>>> ssl.OPENSSL_VERSION
'OpenSSL 1.0.2m  2 Nov 2017'

Is that also too old? (I believe it supports TLS 1.2, but I do not know what cyphers it supports or otherwise)

I did think it unlikely to be an ssl issue since small pages are loaded fine.

Either way in this environment my hands are tied regarding the ssl version being used. In order to test I need to build python + openssl (plus any other dependencies) on a local machine, so it may be a while before I find time to test it.

Shaggy1
  • 21
  • 7
  • Your openssl versions looks pretty outdated. Most likely it is causing the problems because it does only support outdated TLS versions and ciphers. See also https://stackoverflow.com/questions/69371800/how-to-link-python3-to-use-openssl11-or-latest-version-of-openssl-1-1-1-on-c – Robert Feb 03 '22 at 16:48
  • You're incorrectly assuming that the stdlib `ssl` module is used. It's not, because you've chosen to use pyOpenSSL instead. Now, that is a library from PyCA that, if installed from their wheels will probably use its bundled OpenSSL copy, not the system one. So maybe OpenSSL being in use is newer. I think `python -m OpenSSL.debug` will print out what you actually have. The `WantWriteError` bit may be a bug in Cheroot — if you can come up with a minimal reproducer, make sure to post it on the tracker. – webknjaz -- Слава Україні Feb 04 '22 at 17:39
  • Looks like there was an incomplete attempt to fix it @ https://github.com/cherrypy/cheroot/issues/245 / https://github.com/cherrypy/cheroot/pull/332 that needs tests to be implemented. – webknjaz -- Слава Україні Feb 04 '22 at 17:42
  • Thank you for the issues link - it looks like that is what I am hitting and suggests this is a bug in cheroot. Not sure there is much I can do about that, There is a suggestion that using the built in ssl_module rather than pyopenssl so I will try that. – Shaggy1 Feb 07 '22 at 10:49

1 Answers1

0

To wrap up and give an answer to my question.

There appears to be a bug in pyopenssl ssl_module related to the way OpenSSL.SSL.WantReadError and OpenSSL.SSL.WantWriteError are handled as documented here: https://github.com/cherrypy/cheroot/issues/245

If I configure cherrypy using the 'builtin' module as below:

server_config={
    'server.socket_host': chrysocketserver,
    'server.socket_port':int(chrysocketport),
    'server.ssl_module':'builtin',
    'server.ssl_certificate':sslcertificate,
    'server.ssl_private_key':privatekey,
    'server.ssl_certificate_chain':certificatechain,
}
cherrypy.config.update(server_config)

it works fine (at least in my environment)

Shaggy1
  • 21
  • 7