I have a Zope/Plone server which has a browser (let's call it thebrowser) which sends signed requests to a ReSTful web service, receives XML, transforms it to HTML and delivers it to the client via AJAX (thus, from the client's point of view, everything takes place on my server). Most of the time this works, but for certain requests it won't, and I can't find out why. The failing requests take some seconds, and the data length is about 100 kB, but we're not talking about minutes or Megabytes here.
Strangely, the ZServerHTTPResponse object looks quite ok, but it arrives at the web client's side (Firefox, Firebug) as an empty string!
The toolchain looks like this:
In a skin, I have a Script (Python):
## Script (Python) "theservice"
##bind container=container
##bind context=context
##bind namespace=
##bind script=script
##bind subpath=traverse_subpath
##parameters=
##title=
##
thebrowser = context.getBrowser('thebrowser')
return thebrowser.get(thedict={'path':'/'.join(traverse_subpath)})
This makes requests to /theservice/some/sub/path work; some/sub/path is put in the path key.
The get method of thebrowser takes the path argument, does the general processing (sending a signed request to https://remote.service.somewhere/some/sub/path), feeds the XML response to the right transform method according to the path, and returns the response, which is used e.g. by a jquery accordion.
The browser thebrowser looks like this (error handling etc. stripped for clarity):
def transformer(func):
"""
decorator function for generic processing
"""
def f2(self, txt, req=None, auth_tup=None):
global HTML_HEAD, HTML_TAIL
ham = func(self, txt, auth_tup=auth_tup)
res = list(HTML_HEAD)
res.extend(ham)
res.extend(HTML_TAIL)
return u''.join(res)
f2.__name__ = func.__name__
return f2
class Browser(BrowserView):
def get(self, propagate=True, thedict=None):
"""
send a request to the remote ReSTful service
and transform the received XML text to HTML
pretty URL:
/theservice/path/as/sub/path
"""
context = self.context
request = context.request
response = request.response
if thedict is None:
thedict = request.form
path = thedict['path']
# ... set the 'pseudonym', 'theurl' variables ...
auth_tuples = self._auth_tuples(pseudonym)
code = None
text = None
fo = urlopen(theurl)
code = fo.code
headers = fo.info().headers
# set the headers explicitly; might be unnecessary:
for line in headers:
name, text = line.split(':', 1) # found: ': '
handle_header(response, name, text)
raw = fo.read().strip()
text = unicode(raw, 'utf-8') # -> unicode
fo.close()
text = self.transform(path, text, response, theurl, auth_tuples)
response.setBody(text)
return response
# -------------------------------------------------------- get
def transform(self, path, txt, response, theurl=None, auth_tup=None):
"""
Choose a transformer function, apply it to the given text,
and return the HTML text generated
"""
transformer = self.getTransformer(path) # not included here
html = transformer(self, # e.g. tf_xy; see below
txt.decode('utf-8'),
theurl or path,
auth_tup=auth_tup)
if html is not None:
response.setHeader('Content-Type', 'text/html; charset=utf-8')
return html
return txt.encode('utf-8')
@transformer
def tf_xy(self, txt, auth_tup=None):
"""
transform function for path xy
"""
root = ElementTree.fromstring(txt)
res = []
for child in root.getchildren():
if child.tag == 'foo':
res.append(''.join(('<p>', child.text, '</p>'))) # example
# ...
return res
When I apply another decorator to the get method to log the results, I get the following information about the ZServerHTTPResponse object:
headers = {'connection': 'close',
'content-length': '110626',
'content-type': 'text/html; charset=utf-8',
'date': 'Wed, 17 Apr 2013 09:04:13 GMT',
'status': '200 OK',
'transfer-encoding': 'chunked'}
accumulated_headers = 'Set-Cookie: JSESSIONID=471108159E6EF40EB9F2F7305E77EB69; Path=/; Secure\r\n'
body = '<!DOCTYPE html><html><head><meta h(... 110575 bytes total ...)iv></body></html>'
errmsg = 'OK'
status = 200
... which looks quite reasonable to me. Anyway, for some requests (possibly those with big responses or longest processing times) the client browser sees an empty string.
Versions: Zope 2.10.13, Python 2.4.6 (64bit), Plone 3.3 (yes, we'll switch to Plone 4, but it will take some time...)
Update:
I thought my problem was solved when I fixed the ElementTree imports to make use of the compiled version:
try:
from celementtree import fromstring # modern Python versions
except ImportError:
try:
from cElementTree import fromstring # Python 2.4
except ImportError:
from elementtree import fromstring # emergency fallback
Unfortunately, this worked for me, but not for the customer.
The Zope site is served in an Apache VirtualHost where the usual RewriteRule is active:
RewriteRule ^/(.*) http://localhost:8082/VirtualHostBase/http/%{HTTP_HOST}:80/theportalroot/VirtualHostRoot/$1 [L,P]
According to Firebug, there is no content-length header present (can this have been stripped by Apache?!). Here are the complete response headers:
Connection Keep-Alive
Content-Encoding gzip
Content-Type text/html; charset=utf-8
Date Thu, 18 Apr 2013 12:31:41 GMT
Keep-Alive timeout=15, max=99
Server Zope/(Zope 2.10.7-final, python 2.4.6, linux2) ZServer/1.1 Plone/3.2.2 ShellEx Server/4.0.0
Set-Cookie JSESSIONID=308B09957635CA02829AF5C362FB60E3; Path=/; Secure
Transfer-Encoding chunked
Vary Accept-Encoding
I try to get the response directly by Zope, but to do this I must apparently access the Zope root, and I have a permission problem there.
