0

I am attempting to write a web application using the Twisted framework for python. I want the application to work if run as a standalone server (ala twistd), or if Apache reverse proxies to it. E.g.

Apache https://example.com/twisted/ --> https://internal.example.com/

After doing some research, it seemed like I needed to use the vhost.VHostMonsterResource to make this work. So I set up apache with the following directive:

ProxyPass /twisted https://localhost:8090/twisted/https/127.0.0.1:443

Here is my basic SSL server:

from twisted.web import server, resource, static
from twisted.internet import reactor
from twisted.application import service, internet
from twisted.internet.ssl import SSL
from twisted.web import vhost

import sys
import os.path
from textwrap import dedent

PORT = 8090
KEY_PATH = "/home/waldbiec/projects/python/twisted"
PATH = "/home/waldbiec/projects/python/twisted/static_files"

class Index(resource.Resource):
    def render_GET(self, request):
        html = dedent("""\
            <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
            <html>
            <head>
                <title>Index</title>
            </head>
            <body>
                <h1>Index</h1>
                <ul>
                    <li><a href="/files/">Files</a></li>
                </ul>
            </body>
            </html>
            """)
        return html

class ServerContextFactory:
    def getContext(self):
        """
        Create an SSL context.

        Similar to twisted's echoserv_ssl example, except the private key
        and certificate are in separate files.
        """
        ctx = SSL.Context(SSL.SSLv23_METHOD)
        ctx.use_privatekey_file(os.path.join(KEY_PATH, 'serverkey.pem'))
        ctx.use_certificate_file(os.path.join(KEY_PATH, 'servercert.pem'))
        return ctx

class SSLService(internet.SSLServer):
    def __init__(self):
        root = resource.Resource()
        root.putChild("", Index())
        root.putChild("twisted", vhost.VHostMonsterResource())
        root.putChild("files", static.File(PATH))

        site = server.Site(root)
        internet.SSLServer.__init__(self, PORT, site, ServerContextFactory())

application = service.Application("SSLServer")
ssl_service = SSLService()
ssl_service.setServiceParent(application)

It almost works-- but the "files" link on the index page does not behave how I want it to when using apache as a reverse proxy, because it is an absolute link.

My main question is, other than using a relative link, is there some way to compute what the full URL path of the link ought to be in such a way that the link still works in standalone server mode? A second question would be, am I using VHostMonsterResource correctly? I did not find much documentation, and I pieced together my code from examples I found on the web.

Carl
  • 695
  • 8
  • 21

3 Answers3

1

This seems like too much work. Why use VHostMonsterResource at all? You may have very specific reasons for wanting some of this but....Most times:

  • Have apache handle the ssl. apache then passes off to your twisted app serving non SSL goodies back to apache. Documentation all over the net on the apache config stuff.

  • you can sill add another server on an ssl port if you really want to

Haven't tested but structure more like:

root = resource.Resource()
root.putChild("", Index())
root.putChild("files", static.File(PATH))

http = internet.TCPServer(8090, server.Site(root))
# change this port # to 443 if no apache
https= internet.SSLServer(8443, server.Site(root), ServerContextFactory())

application = service.Application("http_https_Server")
http.setServiceParent(application)
https.setServiceParent(application)

Dev tip: During development, for the cost of a couple of extra lines you can add an ssl server so that you can ssh into the running web_server and inspect variables and other state. Way cool.

ssl = internet.TCPServer(8022, getManholeFactory(globals(), waldbiec ='some non-system waldbiec passwork')) 
ssl.setServiceParent(application)
Phil Cooper
  • 5,747
  • 1
  • 25
  • 41
  • How does any of this address URLs? – Jean-Paul Calderone Jul 14 '12 at 10:33
  • In a number of ways, primarily by removing the need the `VHostMosterResource` if eliminates some of the url scrabble that OP was having to go through. In my sample I even removed the `/twisted` route although it could be put back in without any harm. Virtual hosting for `https` never really makes sense since only one domain name can be served on 443 , or any port for that matter. – Phil Cooper Jul 14 '12 at 18:08
  • `VHostMonsterResource` isn't the cause of the problem. Having to generate any URLs at all is the cause of the problem. – Jean-Paul Calderone Jul 15 '12 at 10:29
  • 1
    SSL and vHostmonster aside, does this actually solve the issue if apache is proxying from some non-root path. In my original example, where apache proxies from /twisted to / on the internal server, won't a full path of "/files/" still work out to "/files/" instead of "/twisted/files/" on the apache side of things? – Carl Jul 17 '12 at 02:31
0

Configure the Twisted application so that it knows its own root location. It can use that information to generate URLs correctly.

Jean-Paul Calderone
  • 47,755
  • 6
  • 94
  • 122
  • I considered creating a config file/setting that had something like WEBROOT_PREFIX, but it would not allow the application to behave correctly in local and proxy modes *simultaneously*. I am not sure if that was stated clearly enough in my original post. I would like to be able to browse to the apache address or the internal address and still have the app work without having to reconfigure the app each time. I am not sure if I am understanding your suggestion correctly, or if it just does not account for the scenario I am attempting to describe. – Carl Jul 17 '12 at 02:26
  • It is not that I don't want configuration. What I want to do is to be able to access the application from in front of or behind Apache without 2 different configurations. Ideally, I want the Apache configuration to be able to let the back-end service know what the subtree being proxied is so that the back end can dynamically figure out how to prefix links. I figured out how to do this by looking at the vhost.VHostMonsterResource source. – Carl Jul 22 '12 at 19:29
0

So after digging into the vhost.VHostMonsterResource source, I determined I could create another resource that could let the reverse proxied URL prefix be specified by an additional marker in the Apache ProxyPass URL.

Firstly, I finally figured out that vhost.VHostMonsterResource is supposed to be a special URL in your back end web site that figures out the reverse proxy host and port from data encoded in the URL path. The URL path (sans scheme and net location) looks like:

/$PATH_TO_VHMONST_RES/$REV_PROXY_SCHEME/$REV_PROXY_NETLOC/real/url/components/

$PATH_TO_VHMONST : Path in the (internal) twisted site that corresponds to the VHostMonsterResource resource.
$REV_PROXY_SCHEME : http or https that is being used by the reverse proxy (Apache).
$REV_PROXY_NETLOC : The net location (host and port) or the reverse proxy (Apache).

So you can control the configuration from the reverse proxy by encoding this information in the URL. The result is that the twisted site will understand the HTTP request came from the reverse proxy.

However, if you are proxying a subtree of the external site as per my original example, this information is lost. So my solution was to create an additional resource that can decode the extra path information. The new proxy URL path becomes:

/$PATH_TO_MANGLE_RES/$REV_PROXY_PATH_PREFIX/$VHOSTMONST_MARKER/$REV_PROXY_SCHEME/$REV_PROXY_NETLOC/real/url/components/

$PATH_TO_MANGLE_RES : The path to the resource that decodes the reverse proxy path info.
$REV_PROXY_PATH_PREFIX : The subtree prefix of the reverse proxy.
$VHOSTMONST_MARKER : A path component (e.g. "vhost") that signals a VHostMonster Resource should be used to further decode the path.
Carl
  • 695
  • 8
  • 21