Strange wait on static files from node.js express behind nginx proxy

Question

OK, so the title might be a little confusing, but this issue really is!

First things first though: a little background.

I'm setting up a CI server for a customer, which should be used for automatic testing and building and whatever CI-related tasks they come up with. The solutions being tested are mainly web-solutions, which all run on the MII/NetWeaver web server.

This web server presents the developer with a tool, that allows you to insert "code-snippets" into your HTML, which will then be replaced by a customizable value before it's sent to the browser. Pretty smart.

But, to be able to test the client-side code, I need to make the test-system as decoupled from this MII web server as possible. So i've created a tiny Node.js server app, which does just that - replaces the snippets in the HTML with configured values.

This works great!

But, I can't test the client-side code without having access to the server API. So, using nginx, I've created a reverse proxy, which forwards all /XMII/* requests to the MII server. Simple, and works great! To make everything work together, I've also created a reverse proxy to my Node.js application. This way the 2 separate servers can be accessed on the same port, just as the user would expect.

But, when pairing the 2 (Node.js server and nginx reverse proxy), I consequently see a 1 minute (exactly!) delay on loading some of the client-side files! It varies from being just 1 file, to being several files.

Here are some screenshots, showing exactly what's happening, from Google Chrome's POV:

Exact 1min loads

It says, that it's been waiting for 1 min for the file, and received it in 0ms? Loong wait

This looks a lot more normal. Not so long wait

Here are my config files for both the Node.js server and the nginx proxy. I've removed the code that tampers with the HTML body, since commenting it out made no difference to performance.

Node.js server:

var BASE = 'C:/Users/twj/Documents/git';
var CUSTOM_ATTRIBUTES = null;

var http = require('http')
   ,express = require('express')
   ,tamper = require('tamper') //Used to tamper with the HTML
   ,request = require('request') //Get the configuration
   ,parseString = require('xml2js').parseString; //Parse the configuration

var app = express();

app.use(express.bodyParser());
app.use(app.router);

app.use(express.static(BASE, { maxAge: 86400000 }));

app.post('*', function(req, res){
  res.sendfile(BASE + req.url);
});

app.listen(3000);
console.log('MII Forward Manipulation running on localhost:3000');

nginx.conf:

worker_processes  2;


events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  text/html;

    sendfile        on;

    keepalive_timeout  65;

    gzip  off;

    server {
        listen       1337;
        server_name  localhost;

        autoindex on;

        location /XMII {
            proxy_redirect off;
            proxy_buffering off;
            proxy_pass  http://miiserver:50000/XMII;
        }

        location / {
            proxy_redirect off;
            proxy_buffering off;
            proxy_pass http://localhost:3000/;
        }

        location ~ /\.ht {
            deny  all;
        }
    }
}

I know there's a LOT of information to process here, but I really hope there's someone who have either experienced this themselves (and not made an answer available on google! ;) ), or someone who can work out a solution of some sort.

As a starting point, set up your logs to show timings on both node.js and nginx. For instance: (nodejs) `app.use(express.logger({format: ':req[X-Forwarded-For] - - [:date] ":method :url HTTP/:http-version" :status :res[content-length] - :response-time ms', stream: access_logfile }));` and (nginx) `log_format timed '$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" $request_time $upstream_response_time';`. Compare the timings and you can start to narrow what the cause is. (In general, try to serve static files with nginx directly) — cyberx86, Aug 10 '13 at 00:51
Hi, thanks for your comment, but I've solved it by taking nginx out of the equation, and using node-http-proxy for my proxying needs. Runs fast as lightning now :) — Thor Jacobsen, Aug 10 '13 at 15:21

score 4 · Answer 1 · answered Nov 30 '13 at 22:34

Ok, maybe a little late, but I had exactly the same problem and found a solution after some hours.

I pointed the Nginx proxy to localhost, as you did. It worked on my old server, but not on the new one (to where I copied the config).

The difference was, that the old server had no IPv6 configured, but the new one does.

Nginx sometimes tries http://[::1] for localhost. My backend doesn't know v6, so it just didn't answer.

After exactly one minute there is a timeout (it is also in error log when set to info) and Nginx tries v4 and - voila! Response!

So, solution: Don't use localhost, use 127.0.0.1. :)

Or just fix the broken backend. – Michael Hampton Nov 30 '13 at 22:53 — Michael Hampton, Nov 30 '13 at 22:53

score 0 · Accepted Answer · answered Aug 10 '13 at 15:22

0

I've solved the issue by removing nginx from the equation. Now I'm using node-http-proxy instead.

answered Aug 10 '13 at 15:22

Thor Jacobsen

111
5

Strange wait on static files from node.js express behind nginx proxy

2 Answers2