Timeout due to flask-assets pipeline when debug is turned off

Question

I have a Flask based app that uses flask-assets to bundle, compile and minify css and Javascript. A timeout occurs when I start the app with gunicorn using

gunicorn --bind=0.0.0.0:8000 --workers=3 --log-level=INFO manage:app

The timeout message states that:

[2018-04-01 11:15:13 +0000] [10] [INFO] Starting gunicorn 19.6.0
[2018-04-01 11:15:13 +0000] [10] [INFO] Listening at: http://0.0.0.0:8000 (10)
[2018-04-01 11:15:13 +0000] [10] [INFO] Using worker: sync
[2018-04-01 11:15:13 +0000] [16] [INFO] Booting worker with pid: 16
[2018-04-01 11:15:13 +0000] [17] [INFO] Booting worker with pid: 17
[2018-04-01 11:15:13 +0000] [18] [INFO] Booting worker with pid: 18
[2018-04-01 11:15:46 +0000] [10] [CRITICAL] WORKER TIMEOUT (pid:17)
[2018-04-01 11:15:47 +0000] [17] [INFO] Worker exiting (pid: 17)
[2018-04-01 11:15:47 +0000] [19] [INFO] Booting worker with pid: 19

I have defined the assets in an assets.py file with the content

from flask_assets import Bundle

app_css = Bundle('app.scss', filters='libsass', output='styles/app.css')

app_js = Bundle('app.js', filters='jsmin', output='scripts/app.js')

vendor_css = Bundle(
    'vendor/semantic.css',
    filters='cssmin',
    output='styles/vendor.css')

vendor_js = Bundle(
    'vendor/jquery.min.js',
    'vendor/semantic.min.js',
    'vendor/tablesort.min.js',
    'vendor/zxcvbn.js',
    filters='jsmin',
    output='scripts/vendor.js')

The relevant part in the main app __init__.py is

from flask_assets import Environment

from .assets import app_css, app_js, vendor_css, vendor_js

def create_app(config_name):
    ...
    # Set up asset pipeline
    assets_env = Environment(app)
    dirs = ['assets/styles', 'assets/scripts']
    for path in dirs:
        assets_env.append_path(os.path.join(basedir, path))
    assets_env.url_expire = True

    assets_env.register('app_css', app_css)
    assets_env.register('app_js', app_js)
    assets_env.register('vendor_css', vendor_css)
    assets_env.register('vendor_js', vendor_js)

The question is, how can the timeout be avoided? Is it possible to precompile the assets before gunicorn starts waiting?

score 0 · Accepted Answer · answered Apr 01 '18 at 12:06

The solution requires two changes to work, enabling preload for gunicorn and run the asset pipeline in the factory. To enable preloading, add --preload to the gunicorn command:

gunicorn --bind=0.0.0.0:8000 --workers=3 --preload --log-level=INFO manage:app

If not otherwise specified, the asset pipeline will be run in a lazy manner, i.e., when its assets are first required by a request. This means that if build() is not explicitly called, the first request will block and possibly timeout. Therefore, for each Bundle object call bundle in the create_app function:

app_css.build()
app_js.build()
vendor_css.build()
vendor_js.build()

If only the preloading is enabled in gunicorn or the Bundle not explicitly built before, the build command is called with the first request and then times out.

Bonus: To speed up your Travis CI build, add app/static/.webassets-cache to the cache in .travis.yml with:

cache:
  pip: true
  directories:
    - core/app/static/.webassets-cache

Timeout due to flask-assets pipeline when debug is turned off

1 Answers1