I have two Centos 7 systems that should be identical (other than data).
I have a server process on server A that is being denied write access to a particular directory. The denial shows up as an SELinux denial.
On server B, the same service is not being denied write access to the same directory.
I've compared file perms, directory perms, ACLs, and SELinux contexts between both servers and they appear identical -- except for that A gets the "denied" error and B does not.
As a workaround I set SELinux to "Permissive" and now the server process on A can write to the directory.
At this point, I'm really just looking for ideas of what else I can troubleshoot; I'm happy to provide technical details if required. I am pretty experienced generally with Linux administration but not very experienced specifically with SELinux, so at this point I'm stumped.
Edit -- tech details
The service I've been having trouble with is IPA. Actually, though, it is Apache, as bundled with IPA, not being allowed to write to the memcached dir. Specifically, user "apache" is being denied write access to "/var/run/ipa_memcached/".
Here are commands I ran which came back literally identically between servers (except PIDs). (These were run as root, so lines starting with "$" are the command, what follows is the output)
$ semanage fcontext -l | grep memc
/var/run/memcached(/.*)? all files system_u:object_r:memcached_var_run_t:s0
/var/run/ipa_memcached(/.*)? all files system_u:object_r:memcached_var_run_t:s0
/usr/bin/memcached regular file system_u:object_r:memcached_exec_t:s0
/etc/rc\.d/init\.d/memcached regular file system_u:object_r:memcached_initrc_exec_t:s0
$ ls -ldZ /var/run/ipa_memcached/; ls -lZ /var/run/ipa_memcached/
drwx------. apache apache system_u:object_r:memcached_var_run_t:s0 /var/run/ipa_memcached/
srwx------. apache apache system_u:object_r:memcached_var_run_t:s0 ipa_memcached
-rw-r--r--. apache apache system_u:object_r:memcached_var_run_t:s0 ipa_memcached.pid
$ sudo -u apache id -Z
unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
$ ps auxZ | grep http
system_u:system_r:httpd_t:s0 apache 12321 0.0 6.6 674224 124860 ? Sl 19:07 0:06 (wsgi:ipa) -DFOREGROUND
system_u:system_r:httpd_t:s0 apache 12322 0.0 1.3 324060 26040 ? S 19:07 0:00 /usr/sbin/httpd -DFOREGROUND
system_u:system_r:httpd_t:s0 kdcproxy 12319 0.0 1.2 628032 23604 ? Sl 19:07 0:00 (wsgi:kdcproxy) -DFOREGROUND
system_u:system_r:httpd_t:s0 kdcproxy 12320 0.0 1.2 628032 23604 ? Sl 19:07 0:00 (wsgi:kdcproxy) -DFOREGROUND
system_u:system_r:httpd_t:s0 root 12314 0.0 1.3 298704 24652 ? Ss 19:07 0:00 /usr/sbin/httpd -DFOREGROUND
system_u:system_r:httpd_t:s0 root 12318 0.0 0.5 53880 11096 ? S 19:07 0:00 /usr/libexec/nss_pcache 4423694 off /etc/httpd/alias
unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 root 17615 0.0 0.0 112648 972 pts/0 R+ 21:11 0:00 grep --color=auto http
$ find /var/run/ipa* | xargs getfacl
getfacl: Removing leading '/' from absolute path names
# file: var/run/ipa
# owner: root
# group: root
user::rwx
group::---
other::---
# file: var/run/ipa/services.list
# owner: root
# group: root
user::rw-
group::r--
other::r--
# file: var/run/ipa/renewal.lock
# owner: root
# group: root
user::rw-
group::---
other::---
# file: var/run/ipa_memcached
# owner: apache
# group: apache
user::rwx
group::---
other::---
# file: var/run/ipa_memcached/ipa_memcached.pid
# owner: apache
# group: apache
user::rw-
group::r--
other::r--
# file: var/run/ipa_memcached/ipa_memcached
# owner: apache
# group: apache
user::rwx
group::---
other::---
edit Found the problem
audit2why showed me what was actually causing my problems
grep jsilverman /var/log/audit/audit.log* | grep denied| audit2why
...
/var/log/audit/audit.log.1:type=AVC msg=audit(1471293346.098:440365): avc: denied { create } for pid=13668 comm="httpd" name="krbcc_A_jsilverman" scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:memcached_var_run_t:s0 tclass=file
Was caused by:
The boolean httpd_manage_ipa was set incorrectly.
Description:
Allow httpd to manage ipa
Allow access by executing:
# setsebool -P httpd_manage_ipa 1
... (and more, similar messages) ...
This led to compare selinux settings
Server A
$ getsebool -a| grep httpd_manage_ipa
httpd_manage_ipa --> off
Server B
$ getsebool -a| grep httpd_manage
httpd_manage_ipa --> on
Now, how could that have happened? Also, how can I clone selinux settings from B->A?
