mod_wsgi flask app not seeing jinja2 to import?

Question

I have a test script, test_wsgi.py. See below. This is working. I can hit https://opencalaccess.org/test_wsgi and I get a good result, so I think that mod_wsgi is properly set up.

But when I access my main script, I get a 500.

Here is the bit in my conf:

        WSGIScriptAlias /test_wsgi /var/www/opencalaccess_org/test_script.py

        WSGIDaemonProcess opencalaccess_org threads=5 python-home=/var/www/opencalaccess_org/contracts_scc/.venv
        WSGIScriptAlias /data /var/www/opencalaccess_org/contracts_scc/app.py

        <Directory /var/www/opencalaccess_org/contracts_scc/>
                WSGIProcessGroup opencalaccess_org
                WSGIApplicationGroup %(GLOBAL)
                Require all granted
        </Directory>

My app has:

activate_this = '/var/www/opencalaccess_org/contracts_scc/.venv/bin/activate_this.py'
with open(activate_this) as file_:
    exec(file_.read(), dict(__file__=activate_this))

import re

from jinja2 import Environment, PackageLoader

from flask import Flask
from flask import request

import data

app = Flask(__name__)
env = Environment(loader=PackageLoader('app', 'pages'))
# env = Environment(bytecode_cache=MyCache(), loader=PackageLoader('app', 'pages'))


@app.route('/')
def hello_world():
    return '<h2><a href="contracts/scc">Launch</a></h2>'

Now I see something in the error.log:

[wsgi:info] [pid 229274] mod_wsgi (pid=229274, process='opencalaccess_org', application='%(GLOBAL)'): Loading Python script file '/var/www/opencalaccess_org/contracts_scc/app.py'.
[wsgi:error] [pid 229274] mod_wsgi (pid=229274): Failed to exec Python script file '/var/www/opencalaccess_org/contracts_scc/app.py'.
[wsgi:error] [pid 229274] mod_wsgi (pid=229274): Exception occurred processing WSGI script '/var/www/opencalaccess_org/contracts_scc/app.py'.
[wsgi:error] [pid 229274] Traceback (most recent call last):
[wsgi:error] [pid 229274]   File "/var/www/opencalaccess_org/contracts_scc/app.py", line 7, in <module>
[wsgi:error] [pid 229274]     import jinja2
[wsgi:error] [pid 229274]   File "/var/www/opencalaccess_org/contracts_scc/.venv/lib/python3.8/site-packages/jinja2/__init__.py", line 5, in <module>
[wsgi:error] [pid 229274]     from .bccache import BytecodeCache as BytecodeCache
[wsgi:error] [pid 229274]   File "/var/www/opencalaccess_org/contracts_scc/.venv/lib/python3.8/site-packages/jinja2/bccache.py", line 26
[wsgi:error] [pid 229274]     def get(self, key: str) -> bytes:
[wsgi:error] [pid 229274]                      ^
[wsgi:error] [pid 229274] SyntaxError: invalid syntax

I can run my app manually, accessing it via a hard-coded port, using the same .venv that the conf file is pointing to.

By the way, from my pip freeze:

click==8.1.3
Flask==2.2.2
itsdangerous==2.1.2 
Jinja2==3.1.2 
MarkupSafe==2.1.1
Werkzeug==2.2.2

cat test_wsgi.py:

def application(environ,start_response):
    status = '200 OK'
    html = '\n' \
           '\n' \
           ' mod_wsgi is working\n' \
           '\n' \
           '\n'
    response_header = [('Content-type','text/html')]
    start_response(status,response_header)
    return [html]

Now I have:

renamed app.py to contract_scc.py
changed the WSGIDaemonProcess line to:

WSGIDaemonProcess contracts_scc threads=5 user=ray python-home=/var/www/opencalaccess_org/contracts_scc/.venv

changed my routes in my contracts_scc.py to:

contracts_scc = Flask(name) 
env = Environment(loader=PackageLoader('contracts_scc', 'pages')) 

@contracts_scc.route('/') 
def hello_world(): 
  return '<h2><a href="contracts/scc">Launch</a></h2>'

Now, I am seeing, in the error.log:

either:

[] Traceback (most recent call last): 
[] File "/var/www/opencalaccess_org/contracts_scc/contracts_scc.py", line 10, in <module> 
[] import data

or

Target WSGI script '/var/www/opencalaccess_org/contracts_scc/contracts_scc.py' does not contain WSGI application 'application'.

What the f? –

Don't know if this is really directly related to the error you are experiencing, but the `mod_wsgi` author wrote a note that `activate_this` is not recommended and the `python-home` should be configured instead. Links: https://stackoverflow.com/a/48219099/5378816 and https://stackoverflow.com/a/42662020/5378816 — VPfB, Oct 26 '22 at 09:55
BTW, `https://opencalaccess.org/test_wsgi` gives a 500 error — Andrey Sobolev, Oct 28 '22 at 07:25
Please include the full traceback for "import data" in your post — Angus L'Herrou, Oct 28 '22 at 23:17

score 0 · Answer 1 · answered Oct 28 '22 at 23:27

0

The SyntaxError on def get(self, key: str) -> bytes: would be because mod_wsgi is attempting to run your script with a Python version before type annotations were added (probably 2.7, but could be anything below 3.5).

The mod_wsgi docs say that Python scripts run through WSGIDaemonProcess need to be of the same version of Python that mod_wsgi was compiled for. Make sure you are using a compatible version of mod_wsgi, and not one that was built for an older version of Python, like Python 3.4 or Python 2.

Without more information (namely, a full traceback) about the error you're running into after making the changes to your WSGIDaemonProcess call and your script, not much more can be said.

answered Oct 28 '22 at 23:27

Angus L'Herrou

429
3
11

I am using a virtual environment. I can run, using that venv, on the command line at the location I have installed the app. and I point the WSGI process to the python inside of the venv. It is possible that one can work and not the other? If the venv has the right python version, why would that not work when it is invoked by mod_wsgi? – Ray Kiddy Oct 31 '22 at 17:50
@RayKiddy how is mod_wsgi installed? Did you build it with `./configure --with-python=/path/to/venv/bin/python`? – Angus L'Herrou Oct 31 '22 at 23:02
It had not occurred to me that I should try to build mod_wsgi myself. I am not seeing very many docs on building. I looked at about 100 documents on this issue and I saw zero documents on building mod_wsgi. But I am willing to try it. – Ray Kiddy Nov 01 '22 at 16:02
And again: Target WSGI script '/var/www/opencalaccess_org/contracts_scc/contracts_scc.py' does not contain WSGI application 'application'. W. T. F? – Ray Kiddy Nov 01 '22 at 16:39

Ray Kiddy · Accepted Answer · 2022-11-01T18:15:09.750

Wow.

contracts_scc = Flask(__name__)
application = contracts_scc

What a pain in the derrier....

Well, I am glad that I figured this out before I created a new server and completely reproduced my entire server environment.

So. Given the complexities of the symptoms, what actually solved the issues? Good question. I believe that it was these things:

First, the reasons for the complications. I believe that the issues were that I may have been using a version of mod_wsgi built for the wrong version of python, I am using virtualenv to manage my python version, I have python 3.8.10 on my server and it is fairly difficult to see when suggestions on the internet are not relevant to your version, one has to put the right bits in your apache conf file and one has to put the correct things into one's main script file to make it deployable.

It is very, very easy to run a python flask app in development. It is probably easy, most of the time, to set up a simple deployment. But if there is any problem at all, it can become very complicated to see what to do and how to look past all of the "And look at how easy this is!" documentation.

The things that I believed fixed the issues.

I put my project in my vertual host's directory. This may not have been necessary, but it made things clearer. So, I have:

$ pwd
/var/www/opencalaccess_org/contracts_scc
$ ls
contracts_scc.py  data.py  example.py  _mycache.py_  pages
pdfs              __pycache__  requirements.txt  www
$ cat requirements.txt 
cffi==1.15.1
click==8.1.3
cryptography==38.0.1
Flask==2.2.2
greenlet==1.1.3
importlib-metadata==5.0.0
itsdangerous==2.1.2
Jinja2==3.1.2
MarkupSafe==2.1.1
pycparser==2.21
PyMySQL==1.0.2
python-dotenv==0.21.0
SQLAlchemy==1.4.41
Werkzeug==2.2.2
zipp==3.9.0
$

Build the correct version of mod_wsgi. Just get the source on github and add the --with-python=exec where "exec" is the path to the python inside your virtualenv's bin directory.

For example:

./configure --with-python=/var/www/opencalaccess_org/contracts_scc/.venv/bin/python

I also have a file to test if mod_wsgi is working at all.

$ cat ../test_script.py
def application(environ,start_response):
    status = '200 OK'
    html = '\n' \
           '\n' \
           ' mod_wsgi is working\n' \
           '\n' \
           '\n'
    response_header = [('Content-type','text/html')]
    start_response(status,response_header)
    return [bytes(html, encoding='utf-8')]

I have this in my apache conf file for my virtual domain:

WSGIScriptAlias /test_wsgi /var/www/opencalaccess_org/test_script.py

        # for setting up contracts_scc
        WSGIDaemonProcess contracts_scc threads=5 user=ray python-home=/var/www/opencalaccess_org/contracts_scc/.venv

        WSGIProcessGroup contracts_scc
        WSGIApplicationGroup %{GLOBAL}

        WSGIScriptAlias /data /var/www/opencalaccess_org/contracts_scc/contracts_scc.py

        <Directory /var/www/opencalaccess_org/contracts_scc/>
                Require all granted
        </Directory>

That top bit allows me to hit https://opencalaccess.org/test_wsgi and the next things give me my proper app: https://opencalaccess/data/.

The bits in my main script. This is at the top:

import re
import sys

from flask import Flask, request
from jinja2 import Environment, PackageLoader

sys.path.append("/var/www/opencalaccess_org/contracts_scc/")

import data

contracts_scc = Flask(__name__)
application = contracts_scc

env = Environment(loader=PackageLoader('contracts_scc', 'pages'))
# env = Environment(bytecode_cache=MyCache(), loader=PackageLoader('app', 'pages'))

and this is at the bottom:

if __name__ == '__main__':
    contracts_scc.run()

Anywho. This is what I can brain-dump now. Is everything here necessary to make this work? I have no idea. But it is what it is.

Sufficient unto the day is the evil thereof.

mod_wsgi flask app not seeing jinja2 to import?

2 Answers2