Apache ReverseProxy 502 to local jenkins process

Question

I have apache listening on port 443 running https and performing a ProxyPass back to a standalone jenkins process on the same server.

The bulk of the time, this works fine, but it often produces a 502 error. I can reproduce this by loading the jenkin's web page up and having it auto refresh. Often within 20 minutes of doing so, I can get a 502 page.

I enabled debug logging in Apache and provided my config + apache logs below. Jenkins logs reveal nothing (like it never received the request).

I need some more pointers on tracking this down (and ultimately resolve).

Apache/2.2.3 Jenkins ver. 1.451

    <Location />
    ProxyPass http://jenkins.example.com:8080/
    ProxyPassReverse http://jenkins.example.com:8080/
    </Location>


/usr/lib/jvm/jre-1.6.0/bin/java -Dcom.sun.akuma.Daemon=daemonized -Djava.awt.headless=true -DJENKINS_HOME=/var/lib/jenkins -jar /usr/lib/jenkins/jenkins.war --logfile=/var/log/jenkins/jenkins.log --daemon --httpPort=8080 --ajp13Port=8009 --debug=5 --handlerCountMax=100 --handlerCountMaxIdle=20

Myself getting the error: 192.168.186.207 - - [13/Mar/2013:09:23:16 -0400] "GET /?auto_refresh=true HTTP/1.1" 502 473

[Wed Mar 13 09:23:16 2013] [info] Initial (No.1) HTTPS request received for child 10 (server jenkins.example.com:443)
[Wed Mar 13 09:23:16 2013] [debug] mod_proxy_http.c(56): proxy: HTTP: canonicalising URL //jenkins.example.com:8080/
[Wed Mar 13 09:23:16 2013] [debug] proxy_util.c(1505): [client 192.168.186.207] proxy: http: found worker http://jenkins.example.com:8080/ for http://jenkins.example.com:8080/?auto_re
fresh=true, referer: https://jenkins.example.com/?auto_refresh=true
[Wed Mar 13 09:23:16 2013] [debug] mod_proxy.c(986): Running scheme http handler (attempt 0)
[Wed Mar 13 09:23:16 2013] [debug] mod_proxy_http.c(1982): proxy: HTTP: serving URL http://jenkins.example.com:8080/?auto_refresh=true
[Wed Mar 13 09:23:16 2013] [debug] proxy_util.c(2007): proxy: HTTP: has acquired connection for (jenkins.example.com)
[Wed Mar 13 09:23:16 2013] [debug] proxy_util.c(2063): proxy: connecting http://jenkins.example.com:8080/?auto_refresh=true to jenkins.example.com:8080
[Wed Mar 13 09:23:16 2013] [debug] proxy_util.c(2189): proxy: connected /?auto_refresh=true to jenkins.example.com:8080
[Wed Mar 13 09:23:16 2013] [error] [client 192.168.186.207] (104)Connection reset by peer: proxy: error reading status line from remote server jenkins.example.com, referer: https://jenkins.example.com/?auto_refresh=true
[Wed Mar 13 09:23:16 2013] [debug] mod_proxy_http.c(1484): [client 192.168.186.207] proxy: NOT Closing connection to client although reading from backend server jenkins.example.com failed., r
eferer: https://jenkins.example.com/?auto_refresh=true
[Wed Mar 13 09:23:16 2013] [error] [client 192.168.186.207] proxy: Error reading from remote server returned by /, referer: https://jenkins.example.com/?auto_refresh=true
[Wed Mar 13 09:23:16 2013] [debug] proxy_util.c(2025): proxy: HTTP: has released connection for (jenkins.example.com)
[Wed Mar 13 09:23:16 2013] [debug] ssl_engine_kernel.c(1823): OpenSSL: Write: SSL negotiation finished successfully
[Wed Mar 13 09:23:16 2013] [info] [client 192.168.186.207] Connection closed to child 10 with standard shutdown (server jenkins.example.com:443)
[Wed Mar 13 09:23:17 2013] [info] [client 192.168.186.207] Connection to child 7 established (server jenkins.example.com:443)

Did you ever solve this problem? It has been a while with no solution. — spuder, Aug 15 '14 at 00:26

score 1 · Accepted Answer · answered Mar 13 '13 at 16:13

1

"502" means that there is a problem with the backend server, not with Apache. You can also see this in the log:

[Wed Mar 13 09:23:16 2013] [error] [client 192.168.186.207] (104)Connection reset by peer:       proxy: error reading status line from remote server jenkins.example.com, referer: https://jenkins.example.com/?auto_refresh=true

It's your Jenkins Server that is hanging up on the apache server. Try if the problem also appears if you do this directly on your Jenkins instance (port 8080).

answered Mar 13 '13 at 16:13

Krist van Besien

1,862
13
16

Per your suggestion, I connected directly to Jenkins on 8080 and let it auto-refresh at 1300 Eastern time. It's been 1.5 hours and no timeout. However, going through Apache, I have timed out several times. – ytjohn Mar 13 '13 at 18:37
What is the timeout of your Jenkins server? – Krist van Besien Mar 13 '13 at 20:41
I'm not sure what you mean. I found some documentation on build timeouts, but nothing for the jenkins http component itself. – ytjohn Mar 15 '13 at 13:27

Apache ReverseProxy 502 to local jenkins process

1 Answers1