How to make spydlay module to work like httplib/http.client?

Question

I have to test server based on Jetty. This server can work with its own protocol, HTTP, HTTPS and lastly it started to support SPDY. I have some stress tests which are based on httplib /http.client -- each thread start with similar URL (some data in query string are variable), adds execution time to global variable and every few seconds shows some statistics. Code looks like:

t_start = time.time()
connection.request("GET", path)
resp = connection.getresponse()
t_stop = time.time()
check_response(resp)
QRY_TIMES.append(t_stop - t_start)

Client working with native protocol shares httplib API, so connection may be native, HTTPConnection or HTTPSConnection.

Now I want to add SPDY test using spdylay module. But its interface is opaque and I don't know how to change its opaqueness into something similar to httplib interface. I have made test client based on example but while 2nd argument to spdylay.urlfetch() is class name and not object I do not know how to use it with my tests. I have already add tests to on_close() method of my class which extends spdylay.BaseSPDYStreamHandler, but it is not compatibile with other tests. If it was instance I would use it outside of spdylay.urlfetch() call.

How can I use spydlay in a code that works based on httplib interfaces?

Jetty also provides a SPDY client that you can use for load testing, see for example: https://github.com/eclipse/jetty.project/blob/master/jetty-spdy/spdy-server/src/test/java/org/eclipse/jetty/spdy/server/SynDataReplyDataLoadTest.java. — sbordet, Jan 22 '14 at 13:54
I already have working SPDY client: `spydlay` works, but works differently from `httplib`. It is my problem that I want to use it in my test application and I want it to work similar to other clients. — Michał Niklas, Jan 22 '14 at 16:26
@sborder -- I consider your idea about using Jetty client to test Jetty server :-) I will give it a try with Jython. It may be even easier to use than `spydlay` because it do not use native libraries. — Michał Niklas, Jan 23 '14 at 08:33

score 0 · Answer 1 · answered Jan 24 '14 at 08:04

My only idea is to use global dictionary where url is a key and handler object is a value. It is not ideal because:

new queries with the same url will overwrite previous response
it is easy to forget to free handler from global dictionary

But it works!

import sys
import spdylay

CLIENT_RESULTS = {}


class MyStreamHandler(spdylay.BaseSPDYStreamHandler):
    def __init__(self, url, fetcher):
        super().__init__(url, fetcher)
        self.headers = []
        self.whole_data = []

    def on_header(self, nv):
        self.headers.append(nv)

    def on_data(self, data):
        self.whole_data.append(data)

    def get_response(self, charset='UTF8'):
        return (b''.join(self.whole_data)).decode(charset)

    def on_close(self, status_code):
        CLIENT_RESULTS[self.url] = self


def spdy_simply_get(url):
    spdylay.urlfetch(url, MyStreamHandler)
    data_handler = CLIENT_RESULTS[url]
    result = data_handler.get_response()
    del CLIENT_RESULTS[url]
    return result


if __name__ == '__main__':
    if '--test' in sys.argv:
        spdy_response = spdy_simply_get('https://localhost:8443/test_spdy/get_ver_xml.hdb')

I hope somebody can do spdy_simply_get(url) better.

How to make spydlay module to work like httplib/http.client?

1 Answers1