1

I'm doing a capistrano Setup here with nginx 1.6.2 and Unicorn. But Under my current setup nginx doesn't create the server i have written in con file. Im sure its a permission error for my user's directory as thats where the conf files are located under two rails app dir.

My nginx file is as below:

user  mjp nginx;
worker_processes  1;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                  '$status $body_bytes_sent "$http_referer" '
                  '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

   keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

/etc/nginx/conf.d/*.conf; is empty.

/etc/nginx/sites-enabled/; directory contains 2 symlinks:

[mjp@centos nginx]$ ll sites-enabled/
total 4
lrwxrwxrwx. 1 root root 61 Jan  5 06:58 mjp-portal_production -> /home/mjp/apps/mjp-portal_production/shared/config/nginx.conf
lrwxrwxrwx. 1 root root 58 Jan  3 21:03 mjp-portal_staging -> /home/mjp/apps/mjp-portal_staging/shared/config/nginx.conf

All Permission leading unto those con files:

[mjp@centos ~]$ ll
total 4
drwxrwxr-x. 4 mjp nginx 4096 Jan  5 06:58 apps

[mjp@centos ~]$ ll apps/
total 8
drwxr-xr-x. 5 mjp nginx 4096 Jan  5 07:27 mjp-portal_production
drwxrwxr-x. 5 mjp nginx 4096 Jan  3 21:11 mjp-portal_staging


[mjp@centos ~]$ ll apps/mjp-portal_staging/
total 16
lrwxrwxrwx. 1 mjp nginx   57 Jan  3 21:11 current -> /home/mjp/apps/mjp-portal_staging/releases/20150103210756
drwxrwxr-x. 4 mjp nginx 4096 Jan  3 21:07 releases
drwxrwxr-x. 7 mjp nginx 4096 Jan  3 21:04 repo
-rwxrwxr-x. 1 mjp nginx   71 Jan  3 21:11 revisions.log
drwxrwxr-x. 9 mjp nginx 4096 Jan  3 21:05 shared


[mjp@centos ~]$ ll apps/mjp-portal_staging/shared/
total 28
drwxrwxr-x. 2 mjp nginx 4096 Jan  3 21:10 bin
drwxrwxr-x. 3 mjp nginx 4096 Jan  3 21:05 bundle
drwxrwxr-x. 2 mjp nginx 4096 Jan  5 07:46 config
drwxrwxr-x. 2 mjp nginx 4096 Jan  3 21:11 log
drwxrwxr-x. 3 mjp nginx 4096 Jan  3 21:04 public
drwxrwxr-x. 5 mjp nginx 4096 Jan  3 21:04 tmp
drwxrwxr-x. 3 mjp nginx 4096 Jan  3 21:04 vendor

[mjp@centos ~]$ ll apps/mjp-portal_staging/shared/config/
total 24
-rwxrwxr-x. 1 mjp nginx  136 Jan  3 21:03 database.example.yml
-rwxrwxr-x. 1 mjp nginx  155 Jan  3 21:06 database.yml
-rwxrwxr-x. 1 mjp nginx  188 Jan  3 21:03 log_rotation
-rwxrwxr-x. 1 mjp nginx  814 Jan  5 07:46 nginx.conf
-rwxrwxr-x. 1 mjp nginx 1996 Jan  3 21:03 unicorn_init.sh
-rwxrwxr-x. 1 mjp nginx 1327 Jan  3 21:03 unicorn.rb

mjp-portal_production -> /home/mjp/apps/mjp-portal_production/shared/config/nginx.conf:

upstream unicorn1 {
  server unix:/tmp/unicorn.mjp-portal_production.sock fail_timeout=0;
}

server
{
  server_name 185.48.117.98;
  listen 8080 default;
  root /home/mjp/apps/mjp-portal_production/current/public;

  #access_log /home/mjp/apps/mjp-portal_production/shared/log/nginx_access.log;
  #error_log  /home/mjp/apps/mjp-portal_production/shared/log/nginx_error.log;

  location ^~ /assets/ {
    gzip_static on;
    expires max;
    add_header Cache-Control public;
  }

  try_files $uri/index.html $uri @unicorn;
  location @unicorn {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_redirect off;
    proxy_pass http://unicorn1;
    proxy_buffering off;
  }

  error_page 500 502 503 504 /500.html;
  client_max_body_size 4G;
  keepalive_timeout 10;
}

mjp-portal_staging -> /home/mjp/apps/mjp-portal_staging/shared/config/nginx.conf: 

upstream unicorn {
  server unix:/tmp/unicorn.mjp-portal_staging.sock fail_timeout=0;
}

server
{
  server_name 185.48.117.98;
  listen 8081 default;
  root /home/mjp/apps/mjp-portal_staging/current/public;

  #access_log /home/mjp/apps/mjp-portal_staging/shared/log/nginx_access.log;
  #error_log  /home/mjp/apps/mjp-portal_staging/shared/log/nginx_error.log;

  location ^~ /assets/ {
    gzip_static on;
    expires max;
    add_header Cache-Control public;
  }

  try_files $uri/index.html $uri @unicorn;
  location @unicorn {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_redirect off;
    proxy_pass http://unicorn;
    proxy_buffering off;
  }

  error_page 500 502 503 504 /500.html;
  client_max_body_size 4G;
  keepalive_timeout 10;
}

Even when i set to run the nginx process ("worker") as root. Still nginx can't create the server and start listening to it.

netstat -anp doesn't show the ports that are open by nginx. In this case port 8080 and port 8081.

What am i doing wrong. All the permissions seem to be right. Am i still missing something? When i put code for these two symlinks in /etc/nginx/conf.d/. It does opens those ports although i get502 bad gateway` which makes me think that its a permission error. on those app directories.

What am i doing wrong?

Saad Masood
  • 179
  • 3
  • 10
  • What is the output of `nginx -t`? – Michael Hampton Jan 05 '15 at 13:26
  • Its says that configuration ok. And test successful. – Saad Masood Jan 05 '15 at 13:29
  • Ok. I got it running. But im still confused. I usually start nginx with `sudo service nginx start` But this time i stopped it. and then just ran `sudo nginx` which runs and i can see that nginx is listening on `8081` and `8080`. But when i try to stop it with `sudo service nginx stop` it says that it doesn't have permission for `/var/run/nginx.pid` which is weird as i'm doing a sudo on it. – Saad Masood Jan 05 '15 at 13:35
  • @MichaelHampton so the question is How is `sudo nginx` different than `sudo service nginx start`??? – Saad Masood Jan 05 '15 at 13:37

2 Answers2

7

This is an selinux problem.

When you run sudo nginx it starts nginx as unconfined_t, when you run sudo service nginx start it starts nginx as httpd_t.

By initially starting with just sudo it creates a bunch of files and initializes its state as unconfined_t. For example the pid file will be the wrong context. Thus when using service nginx stop to terminate it there is insufficient privileges for httpd_t to read files written by the unconfined_t.

You should really always start using service which will avoid this problem. To correct it you will need to relabel stateful files that exist in the filesystem, for example running restorecon /var/run/nginx.pid will correct the incorrect label set on that pid file.

I am not sure if there are any more files that get written out when the service is created which will also need correcting. You can get a list of which files that these might be doing ausearch -ts recent -m avc.

Matthew Ife
  • 23,357
  • 3
  • 55
  • 72
3

Some additional information for those who want to extend a little bit knowledge on selinux and to debug issues with selinux:

https://www.nginx.com/blog/nginx-se-linux-changes-upgrading-rhel-6-6/

tl;dr

Debug issues with SElinux permissions:

  1. set permissive mode ( informs about security breach in audit.log and performs actions )
  2. check audit.log ( for centos and probaly all RH familly /var/log/audit/audit.log )
  3. apply proper permissions on SElinux or on files

Tool:

ausearch -i -m avc

will help to read audit.log in human readable format any AVC (SElinux) issue

You can also try to add:
-ts recent
-ts today

to narrow scope of search.

Krzysztof Zawadzki
  • 31
  • 3