Apache Failed to Start in Pacemaker

Question

I am using Pacemaker with Corosync to set up a basic Apache HA cluster with 3 nodes running CentOS. For some reasons, I cannot get the apache resource started in pcs.

Cluster IP: 192.168.200.40

# pcs resource show ClusterIP
     Resource: ClusterIP (class=ocf provider=heartbeat type=IPaddr2)
      Attributes: cidr_netmask=24 ip=192.168.200.40
      Operations: monitor interval=20s (ClusterIP-monitor-interval-20s)
                  start interval=0s timeout=20s (ClusterIP-start-interval-0s)
                  stop interval=0s timeout=20s (ClusterIP-stop-interval-0s)



# pcs resource show WebServer
 Resource: WebServer (class=ocf provider=heartbeat type=apache)
  Attributes: configfile=/etc/httpd/conf/httpd.conf statusurl=http://localhost/server-status
  Operations: monitor interval=1min (WebServer-monitor-interval-1min)
              start interval=0s timeout=40s (WebServer-start-interval-0s)
              stop interval=0s timeout=60s (WebServer-stop-interval-0s)



# pcs status
Cluster name: 
WARNING: corosync and pacemaker node names do not match (IPs used in setup?)
Stack: corosync
Current DC: server3.example.com (version 1.1.18-11.el7_5.2-2b07d5c5a9) - partition with quorum
Last updated: Thu Jun  7 21:59:09 2018
Last change: Thu Jun  7 21:45:23 2018 by root via cibadmin on server1.example.com

3 nodes configured
2 resources configured

Online: [ server1.example.com server2.example.com server3.example.com ]

Full list of resources:

 ClusterIP  (ocf::heartbeat:IPaddr2):   Started server2.example.com
 WebServer  (ocf::heartbeat:apache):    Stopped

Failed Actions:
* WebServer_start_0 on server3.example.com 'unknown error' (1): call=49, status=Timed Out, exitreason='',
    last-rc-change='Thu Jun  7 21:46:03 2018', queued=0ms, exec=40002ms
* WebServer_start_0 on server1.example.com 'unknown error' (1): call=53, status=Timed Out, exitreason='',
    last-rc-change='Thu Jun  7 21:45:23 2018', queued=0ms, exec=40003ms
* WebServer_start_0 on server2.example.com 'unknown error' (1): call=47, status=Timed Out, exitreason='',
    last-rc-change='Thu Jun  7 21:46:43 2018', queued=1ms, exec=40002ms


Daemon Status:
  corosync: active/enabled
  pacemaker: active/enabled
  pcsd: active/enabled

The httpd instance is enabled and running on all three nodes. The cluster IP and individual node IPs are able to access the web page. The ClusterIP resource also works well for failover. What may go wrong for the apache resource in this case?

Thank you very much!

Update:

Here is more information from the debug output. It seems the Apache is unable to bind to the port, but there is no error from the apache log, and systemctl status httpd gave all green on all nodes. I can open web pages via the cluster IP and node IPs. The ClusterIP resource failover works fine, too. Any idea on why Apache resource doesn't work with pacemaker?

# pcs resource debug-start WebServer --full
Operation start for WebServer (ocf:heartbeat:apache) failed: 'Timed Out' (2)
 >  stderr: ERROR: (98)Address already in use: AH00072: make_sock: could not bind to address [::]:80 (98)Address already in use: AH00072: make_sock: could not bind to address 0.0.0.0:80 no listening sockets available, shutting down AH00015: Unable to open logs
 >  stderr: INFO: apache not running
 >  stderr: INFO: waiting for apache /etc/httpd/conf/httpd.conf to come up
 >  stderr: INFO: apache not running
 >  stderr: INFO: waiting for apache /etc/httpd/conf/httpd.conf to come up
 >  stderr: INFO: apache not running
 >  stderr: INFO: waiting for apache /etc/httpd/conf/httpd.conf to come up
 >  stderr: INFO: apache not running

"The httpd instance is enabled and running on all three nodes." Surprising. What do you mean? Edit the facts into the question. — kubanczyk, Jun 08 '18 at 16:43
@kubanczyk Hi Kubanczyk, thanks for taking a look at my question. I just figured it out. The problem was solved by stopping and disabling the httpd service on all nodes via systemctl. The service should be controlled by pcs resource manager, rather than the systemd service manager. — cody, Jun 08 '18 at 16:53
Ah, solved then. It's surprising that pacemaker didn't recognize `systemctl` started the httpd. Normally it does detect it. — kubanczyk, Jun 08 '18 at 17:16

score 2 · Answer 1 · answered Feb 02 '21 at 17:50

In CentOS8

doing this...

pcs resource create httpd_monitor ocf:heartbeat:apache \
configfile="/etc/httpd/conf/httpd.conf" \
statusurl="http://127.0.0.1/server-status" --group apache

The file /etc/httpd/conf/httpd.conf is checked for the PidFile parameter. This is not defined, but defaults to /var/run/httpd/httpd.pid

[root@hanode1 ~]# pcs resource
  * Resource Group: apache:
    * httpd_fs  (ocf::heartbeat:Filesystem):     Started hanode1.lab.local
    * httpd_vip (ocf::heartbeat:IPaddr2):        Started hanode1.lab.local
    * apache_service    (service:httpd):         Started hanode1.lab.local
    * httpd_monitor     (ocf::heartbeat:apache):         Stopped

You get this error message

Feb 02 17:39:21 INFO: apache not running
Feb 02 17:39:21 INFO: waiting for apache /etc/httpd/conf/httpd.conf to come up

So if you define this in /etc/httpd/conf/httpd.conf

# this is the default but is required by pcs to be defined
PidFile /var/run/httpd/httpd.pid

This will run fine, as below:

[root@hanode1 ~]# pcs resource debug-start httpd_monitor
Operation start for httpd_monitor (ocf:heartbeat:apache) returned: 'ok' (0)
Feb 02 17:39:57 INFO: apache already running (pid 88022)

Then you can clean up with pcs resource cleanup httpd_monitor

# pcs resource
  * Resource Group: apache:
    * httpd_fs  (ocf::heartbeat:Filesystem):     Started hanode1.lab.local
    * httpd_vip (ocf::heartbeat:IPaddr2):        Started hanode1.lab.local
    * apache_service    (service:httpd):         Started hanode1.lab.local
    * httpd_monitor     (ocf::heartbeat:apache):         Started hanode1.lab.local

kudos to @cleverpig

score 0 · Answer 2 · answered Apr 22 '20 at 13:46

The standard path for Apache PID file in CentOs/RHEL7 is /var/run/httpd/httpd.pid. However, pacemaker keeps looking PID file in /var/run/httpd.pid. We can fix pacemaker script using following command.

/bin/sed -i 's/RUNDIR\/${httpd_basename}.pid/RUNDIR\/${httpd_basename}\/${httpd_basename}.pid/g' /usr/lib/ocf/lib/heartbeat/apache-conf.sh

score 0 · Answer 3 · answered Jan 23 '21 at 06:49

0

You can claim pidfile loaction in httpd.conf:/var/run/httpd.pid. refer from: https://httpd.apache.org/docs/2.4/mod/mpm_common.html

answered Jan 23 '21 at 06:49

cleverpig

101
1

Apache Failed to Start in Pacemaker

3 Answers3