passenger(mod_rails) fails to start puppet master under nginx

Question

On the server

[root@bangvmpllDA02 logs]# ruby -v
ruby 1.8.7 (2011-06-30 patchlevel 352) [x86_64-linux]

[root@bangvmpllDA02 logs]# puppet --version
3.0.1

and

[root@bangvmpllDA02 logs]# service nginx configtest
nginx: the configuration file /apps/nginx/nginx.conf syntax is ok
nginx: configuration file /apps/nginx/nginx.conf test is successful
[root@bangvmpllDA02 logs]# service nginx status
nginx (pid 25923 25921 25920 25917 25908) is running...
[root@bangvmpllDA02 logs]#

however none of my agents are able to connect to the master, they all fail with errors like so

[amisr1@blramisr195602 ~]$ puppet agent --test --verbose --server bangvmpllda02.XXX.com
Info: Creating a new SSL certificate request for blramisr195602.XXX.com
Info: Certificate Request fingerprint (SHA256): 26:EB:08:1F:82:32:E4:03:7A:64:8E:30:A3:99:93:26:E6:66:B9:B0:49:B6:08:F9:67:CA:1B:0C:00:B9:1D:41
Error: Could not request certificate: Error 405 on SERVER: <html>
<head><title>405 Not Allowed</title></head>
<body bgcolor="white">
<center><h1>405 Not Allowed</h1></center>
<hr><center>nginx</center>
</body>
</html>

Exiting; failed to retrieve certificate and waitforcert is disabled

when I check logs on puppet master

[root@bangvmpllDA02 logs]# tail puppet_access.log
[05/Dec/2012:17:45:18 +0530] "GET /production/certificate/ca? HTTP/1.1" 404 162 "-" "Ruby"
[05/Dec/2012:18:32:23 +0530] "PUT /production/certificate_request/sl63anadi.XXX.com HTTP/1.1" 405 166 "-" "-"
[05/Dec/2012:18:33:33 +0530] "GET /production/certificate/sl63anadi.XXX.com? HTTP/1.1" 404 162 "-" "-"
[05/Dec/2012:18:33:33 +0530] "GET /production/certificate_request/sl63anadi.XXX.com? HTTP/1.1" 404 162 "-" "-"
[05/Dec/2012:18:33:33 +0530] "PUT /production/certificate_request/sl63anadi.XXX.com HTTP/1.1" 405 166 "-" "-"

and the error logs show that nginx is not really able to process the request well

2012/12/05 18:33:33 [error] 25920#0: *23 open() "/etc/puppet/rack/public/production/certificate/sl63anadi.XXX.com" failed (2: No such file or directory), client: 10.209.47.26, server: , request: "GET /production/certificate/sl63anadi.XXX.com? HTTP/1.1", host: "bangvmpllda02.XXX.com:8140"
2012/12/05 18:33:33 [error] 25920#0: *24 open() "/etc/puppet/rack/public/production/certificate_request/sl63anadi.XXX.com" failed (2: No such file or directory), client: 10.209.47.26, server: , request: "GET /production/certificate_request/sl63anadi.XXX.com? HTTP/1.1", host: "bangvmpllda02.XXX.com:8140"
2012/12/05 18:47:56 [error] 25923#0: *27 open() "/etc/puppet/rack/public/production/certificate/ca" failed (2: No such file or directory), client: 10.209.47.31, server: , request: "GET /production/certificate/ca? HTTP/1.1", host: "bangvmpllda02.XXX.com:8140"
2012/12/05 18:47:56 [error] 25923#0: *28 open() "/etc/puppet/rack/public/production/certificate_request/blramisr195602.XXX.com" failed (2: No such file or directory), client: 10.209.47.31, server: , request: "GET /production/certificate_request/blramisr195602.XXX.com? HTTP/1.1", host: "bangvmpllda02.XXX.com:8140"

Passenger does not show any application groups either

[root@bangvmpllDA02 nginx]# passenger-status 
----------- General information -----------
max      = 15
count    = 0
active   = 0
inactive = 0
Waiting on global queue: 0

----------- Application groups -----------
[root@bangvmpllDA02 nginx]#

here's my nginx configuration

[root@bangvmpllDA02 logs]# cat ../nginx.conf

user  puppet;
worker_processes  4;

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
error_log  logs/error.log  info;

#pid        logs/nginx.pid;


events {
    use epoll;
    worker_connections  1024;
}


    http {
        include       mime.types;
        default_type  application/octet-stream;

        log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                          '$status $body_bytes_sent "$http_referer" '
                          '"$http_user_agent" "$http_x_forwarded_for"';

        access_log  logs/access.log  main;

        sendfile        on;
        #tcp_nopush     on;
        server_tokens off;
        #keepalive_timeout  0;
        keepalive_timeout  120;

        gzip  on;
        gzip_http_version 1.1;
        gzip_disable "msie6";
        gzip_vary on;
        gzip_min_length 1100;
        gzip_buffers 64 8k;
        gzip_comp_level 3;
        gzip_proxied any;
        gzip_types text/plain text/css application/x-javascript text/xml application/xml;

        server {
            listen       80;
            server_name  bangvmpllda02.XXXX.com;

            charset utf-8;

            #access_log  logs/http.access.log  main;

            location / {
                root   html;
                index  index.html index.htm index.php;
            }

            #error_page  404              /404.html;

            # redirect server error pages to the static page /50x.html
            #
            error_page   500 502 503 504  /50x.html;
            location = /50x.html {
                root   html;
            }

            # proxy the PHP scripts to Apache listening on 127.0.0.1:80
            #
            #location ~ \.php$ {
            #    proxy_pass   http://127.0.0.1;
            #}

            # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
            #
            location ~ \.php$ {
                root           html;
                fastcgi_pass   unix:/var/run/php-fpm/php-fpm.sock;
                fastcgi_index  index.php;
                fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
                fastcgi_param  SCRIPT_NAME  $fastcgi_script_name;
                include        fastcgi_params;
            }

            # deny access to .htaccess files, if Apache's document root
            # concurs with nginx's one
            #
            location ~ /\.ht {
            access_log off;
            log_not_found off; 
                deny  all;
            }

        location ~* \.(jpg|jpeg|gif|png|css|js|ico|xml)$ {
            access_log        off;
            log_not_found     off;
            expires           2d;
        }   
        }

        # Passenger needed for puppet
        passenger_root  /usr/lib/ruby/gems/1.8/gems/passenger-3.0.18;
        passenger_ruby  /usr/bin/ruby;
        passenger_max_pool_size 15;

        server {
        ssl                on;
        listen                     8140 default ssl;
            server_name                bangvmpllda02.XXXX.com; 
        passenger_enabled          on;
        passenger_set_cgi_param    HTTP_X_CLIENT_DN $ssl_client_s_dn; 
        passenger_set_cgi_param    HTTP_X_CLIENT_VERIFY $ssl_client_verify; 
        passenger_min_instances    5;

        access_log                 logs/puppet_access.log;
        error_log                  logs/puppet_error.log;

        root                       /etc/puppet/rack/public;

        ssl_certificate            /var/lib/puppet/ssl/certs/bangvmpllda02.XXX.com.pem;
        ssl_certificate_key        /var/lib/puppet/ssl/private_keys/bangvmpllda02.XXX.com.pem;
        ssl_crl                    /var/lib/puppet/ssl/ca/ca_crl.pem;
        ssl_client_certificate     /var/lib/puppet/ssl/certs/ca.pem;
        ssl_ciphers                SSLv2:-LOW:-EXPORT:RC4+RSA;
        ssl_prefer_server_ciphers  on;
        ssl_verify_client          optional;
        ssl_verify_depth           1;
        ssl_session_cache          shared:SSL:128m;
        ssl_session_timeout        5m;
        }
    }

and the puppet.conf

[main]
    # The Puppet log directory.
    # The default value is '$vardir/log'.
    logdir = /var/log/puppet

    # Where Puppet PID files are kept.
    # The default value is '$vardir/run'.
    rundir = /var/run/puppet
    dns_alt_names = devops.XXXX.com,devops
    confdir = /etc/puppet
    vardir = /var/lib/puppet
    storeconfigs = true
    storeconfigs_backend = puppetdb
    thin_storeconfigs = false
    async_storeconfigs = false
    ssl_client_header = SSL_CLIENT_S_D
    ssl_client_verify_header = SSL_CLIENT_VERIFY

    # Where SSL certificates are kept.
    # The default value is '$confdir/ssl'.
    ssldir = $vardir/ssl

any ideas where am I going wrong? I checkthe directory permissions; /usr/share/puppet, /etc/puppet and /var/lib/puppet (and files inside them) are owned by puppet user.

Solved

The simple solution to my complicated problem was that I had placed the config.ru in wrong place

moved it to /etc/puppet/rack , it was in /etc/puppet/rack/public

Well!!! :-/

I disbaled SELinux `setentforce 0` and reboot the system, to see if that was causing any issues, I still get same results as above. When I check in general about nginx 405 issues, it does show issues with HTTP PUT requests for static files in newer versions of Nginx. trying to test it using http put for a js file to assert if that is the case. — Anadi Misra, Dec 08 '12 at 05:05

score 1 · Accepted Answer · answered Dec 09 '12 at 04:36

The problem was that I had wrongly placed config.ru inside /etc/puppet/rack/public folder, moved it to /etc/puppet/rack. It all works fine now.

Not related to the problem but good to point out:

changed the server_name nginx configuration from

server_name 0.0.0.0;

to

server_name _;

also re-enabled SELinux (my system was setup on permissive).

passenger(mod_rails) fails to start puppet master under nginx

1 Answers1