4

I'm trying to set up an Open Device Lab.

This question focuses on the network routing of and injection of code into the responses to requests made by the test devices.

I want people to be able to come in with their laptops, and test as painlessly as possible. I want developers to be able to:

  • inspect the mobile browsers,
  • not have to refresh every page on file change, and
  • not have to navigate each device separately.

To achieve this, I have chosen to use the following technologies (respectively):

All of these technologies require JavaScript snippets in the requested web pages in order to open and maintain socket connections with a server program. I want to use a proxy server to inject these snippets.

Below is an image of how I plan to set up the testing network:

https://i.stack.imgur.com/klGJF.png

I will quickly explain the roles of the different parts of the lab:

Local web server + test device

This machine is the machine that a developer would come in with, running a web server of their choice (Apache, Nginx, IIS, whatever, it shouldn't matter). This doubles up as a test device. The developer will have to install some software that notifies the LiveReload server when a file has changed (like guard-livereload). It will connect to the router via WiFi.

Test devices

These are the mobile devices that are the reason the lab is here. The web pages they render will be inspectable using weinre, refresh on file change using LiveReload, and navigate along when any other device navigates using Shim. They will connect via WiFi to the router.

Router

The router will use the proxy server for all requests served on port 80. It will also use the proxy server machine as a DNS server.

Proxy server

This server is more than just a proxy server. Its tasks are to be:

  • a DNS server (so that local test domains will work without having to configure each device separately, thinking of using Smbind for this),
  • a proxy server (in order to inject the JavaScript snippets),
  • a weinre server (to be able to inspect any connected WebKit client),
  • a LiveReload server (to signal to all connected browsers to refresh the whole page or some of its resources when appropriate),
  • a Shim server (to signal all connected browsers to navigate when appropriate), and
  • a network speed throttle (to simulate slow connections).

TL;DR

What I am struggling with is the following:

  • Which proxy software can inject code into requests?
  • How do I only inject the code into local port 80 requests that are HTML (as indicated by the MIME type) and contain </body> ('local' refers to the test network, not requests that go out to the wider Internet)? Also, the code needs to be injected right in front of </body>.
  • How do I transparently route all traffic through the proxy (so that I can inject code and throttle speed), i.e. will my current set up even work?

I would very much appreciate your thoughts.

Community
  • 1
Xandor Schiefer
  • 151
  • 1
  • 6

2 Answers2

2

The feature you're possibly looking for is ICAP (Internet Content Adaptation Protocol).

There's a Python framework that handles it (http://icap-server.sourceforge.net/), and Squid itself might have the feature now (at least in development).

There is more info (and a list of other ICAP servers) on the Squid Wiki at http://wiki.squid-cache.org/Features/ICAP#ICAP_Servers

voretaq7
  • 79,879
  • 17
  • 130
  • 214
sw00
  • 121
  • 2
  • This answers the question of how to get the proxy to inject code. I still don't know how to route all traffic through the proxy. – Xandor Schiefer Feb 01 '13 at 09:37
  • I had to use [pythonbrew](https://github.com/utahta/pythonbrew) to run Python 2.1.1 (which is was icap-server was based on), and delete a line of code that tried to import module `FCNTL` right after importing `fcntl` to get this to work. – Xandor Schiefer Feb 19 '13 at 14:36
  • 1
    I also found I needed an exact version of PyXML (0.7.1), not the latest one. I got that version [here](http://pkgs.fedoraproject.org/repo/pkgs/PyXML/PyXML-0.7.1.tar.gz/6a2e7f0b95961b08dab95b0416cebf9d/PyXML-0.7.1.tar.gz). – Xandor Schiefer Feb 20 '13 at 14:32
  • Unselected answer because ICAP does not seem to help me. All the open source versions are out of date and/or badly documented. – Xandor Schiefer Feb 22 '13 at 14:23
1

Although I appreciate the ICAP suggestion, the open source ICAP servers are out of date and/or badly documented. I spent a few days trying to set up the ICAP solution, with no dice.

So I turned to Apache as the proxy. I configured it as a transparent forward proxy, and configured the filter chain to inflate (if content was deflated), do a substitution, and deflate (if originally deflated). This works like a charm.

In terms of routing all the traffic through the proxy, I have separate access points for all the devices, and the local web server. They are on different subnets, thus the router (same as the proxy machine) needs to route traffic between the two, and sends traffic through the proxy.

With only one access point I had the problem that outbound traffic went through the proxy, but local traffic did not.

I will post a much more detailed write-up of how I set up the device lab once it is finished. I'm still figuring some stuff out (unrelated to this question).

Xandor Schiefer
  • 151
  • 1
  • 6