1

I am developing a Pyramid 1.7 web app using Python 3.5, uWSGI 2.0.11 and SQLAlchemy 1.0.9. Seems like when using uWSGI with multiple workers we should use a uWSGI postfork function to connect to the Cassandra cluster ensuring each fork will use a independent connection to the pool. I have tried the following Pyramid implementation (file my_app/__init__.py:

#my_app/__init__.py
def main(global_config, **settings):

    try:
        from uwsgidecorators import postfork
    except ImportError:
        # We're not in a uWSGI context, no need to hook dbs connection
        # to the postfork event.
        connection.setup(
                     [settings['cassandra.host']],
                     settings['cassandra.keyspace'],
                     port=int(settings['cassandra.port'])
        )

    else:
        @postfork
        def init():
            """ Initialize dbs connexions in the context.
                Ensures that a new connexion is returned for every new request.
            """
            if cql_cluster is not None:
                cql_cluster.shutdown()
            if cql_session is not None:
                cql_session.shutdown()

            connection.setup(
                     [settings['cassandra.host']],
                     settings['cassandra.keyspace'],
                     port=int(settings['cassandra.port'])
            )


    config = Configurator(settings=settings, root_factory=my_factory)
    config.scan()
    return config.make_wsgi_app()

But I am getting connections timeouts when running the app in production using uWSGI. If you install the libev library Cassandra will detect the library an use it by default. But I don't know if some changes need to be done in uWSGI in order to be compatible (Monkeypatching must be disabled for sure).

Is this the right way to configure Pyramid + Cassandra + uWSGI using the uWSGI forking mode? Or am I missing something?

Ander
  • 5,093
  • 7
  • 41
  • 70
  • Are you starting uwsgi with lazy-apps = true?. This will instruct uWSGI to load the applications after each worker’s fork() – webjunkie Sep 21 '16 at 00:26
  • 1
    **lazy-apps = true** would do the trick, but that will consume way more memory than using [forking](http://uwsgi-docs.readthedocs.io/en/latest/articles/TheArtOfGracefulReloading.html#preforking-vs-lazy-apps-vs-lazy) which only loads the app once and then shares the result with the other workers. I was interested specifically on how to configure Cassandra with Pyramid +uWSGI using [forking mode](http://uwsgi-docs.readthedocs.io/en/latest/articles/TheArtOfGracefulReloading.html#preforking-vs-lazy-apps-vs-lazy) (I've updated my question). – Ander Sep 21 '16 at 20:19
  • uWSGI has some events the process can receive pre/post fork, but I have never used them myself. Maybe you can use them to detect forks and set pools accordingly. – Mikko Ohtamaa Oct 01 '16 at 23:48

0 Answers0