1

Trying to configure Tomcat8.5 as backend and Apache2.4 for receiving requests to redirect it to tomcat via ajp port on Linux/CentOS. The tomcats can be reached directly via Port. 8181, 8282, 8383 and apache works as well fine. However, I cannot reach the tomcats by using localhost/app1 ../app2 ../app3 as configured. I am getting "Service unavailable ERROR 503". I don't understand the error messages in mod_jk.log (e.g.connecting to tomcat failed) as I think, all have been configured correctly.

Where is the mistake?

server.xml of Tomcat1 (Tomcat2/3 port offset +100)

<Connector port="8181" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" 
               maxThreads="500" 
               secret="F45A93BF-3AA7-4CB4-E49A-DB34573E4A25" 
               allowedRequestAttributesPattern=".*"/>

<Connector protocol="AJP/1.3"
               address="localhost"
               port="8109"
               redirectPort="8443" />

httpd.conf

LoadModule jk_module modules/mod_jk.so    
JkWorkersFile conf/workers.properties

JkLogFile     logs/mod_jk.log
JkLogLevel    debug
JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "
JkOptions     +ForwardKeySize +ForwardURICompat -ForwardDirectories
JkRequestLogFormat     "%w %V %T"

JkMount /app1* tomcat1
JkMount /app2* tomcat2
JkMount /app3* tomcat3

workers.properties:

worker.list=tomcat1,tomcat2,tomcat3

worker.tomcat1.type=ajp13
worker.tomcat1.host=localhost
worker.tomcat1.port=8109
worker.tomcat1.secret=F45A93BF-3AA7-4CB4-E49A-DB34573E4A25

worker.tomcat2.type=ajp13
worker.tomcat2.host=localhost
worker.tomcat2.port=8209
worker.tomcat2.secret=4F5A93BF-3AA7-4CB4-E49A-DB34573E4A52

worker.tomcat3.type=ajp13
worker.tomcat3.host=localhost
worker.tomcat3.port=8309
worker.tomcat3.secret=45FA93BF-3AA7-4CB4-E49A-DB34573EA452

mod_jk.log

[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] jk_servlet_normalize::jk_util.c (2184): URI on entering jk_servlet_normalize: [/app1]
[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] jk_servlet_normalize::jk_util.c (2278): URI on exiting jk_servlet_normalize: [/app1]
[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] map_uri_to_worker_ext::jk_uri_worker_map.c (1167): Attempting to map URI '/app1' from 3 maps
[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] find_match::jk_uri_worker_map.c (977): Attempting to map context URI '/app1*=tomcat1' source 'JkMount'
[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] find_match::jk_uri_worker_map.c (990): Found a wildchar match '/app1*=tomcat1'
[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] jk_handler::mod_jk.c (2821): Into handler jakarta-servlet worker=tomcat1 r->proxyreq=0
[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] wc_get_worker_for_name::jk_worker.c (119): found a worker tomcat1
[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] wc_maintain::jk_worker.c (352): Maintaining worker tomcat1
[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] wc_maintain::jk_worker.c (352): Maintaining worker tomcat2
[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] wc_maintain::jk_worker.c (352): Maintaining worker tomcat3
[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] wc_get_name_for_type::jk_worker.c (303): Found worker type 'ajp13'
[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] init_ws_service::mod_jk.c (1178): Service protocol=HTTP/1.1 method=GET ssl=false host=(null) addr=127.0.0.1 name=localhost port=80 auth=(null) user=(null) laddr=127.0.0.1 raddr=127.0.0.1 uaddr=127.0.0.1 uri=/app1
[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] ajp_get_endpoint::jk_ajp_common.c (3356): (tomcat1) acquired connection pool slot=0 after 0 retries
[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] ajp_marshal_into_msgb::jk_ajp_common.c (680): (tomcat1) ajp marshaling done
[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] ajp_service::jk_ajp_common.c (2587): processing tomcat1 with 2 retries
[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] ajp_send_request::jk_ajp_common.c (1718): (tomcat1) no usable connection found, will create a new one.
[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] jk_open_socket::jk_connect.c (673): socket TCP_NODELAY set to On
[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] jk_open_socket::jk_connect.c (797): trying to connect socket 18 to ::1:8109
[Tue May 18 08:39:27 2021] [67918:139621179135744] [info] jk_open_socket::jk_connect.c (815): connect to ::1:8109 failed (errno=111)
[Tue May 18 08:39:27 2021] [67918:139621179135744] [info] ajp_connect_to_endpoint::jk_ajp_common.c (1064): (tomcat1) Failed opening socket to (::1:8109) (errno=111)
[Tue May 18 08:39:27 2021] [67918:139621179135744] [error] ajp_send_request::jk_ajp_common.c (1724): (tomcat1) connecting to backend failed. Tomcat is probably not started or is listening on the wrong port (errno=111)
[Tue May 18 08:39:27 2021] [67918:139621179135744] [info] ajp_service::jk_ajp_common.c (2774): (tomcat1) sending request to tomcat failed (recoverable), because of error during request sending (attempt=1)
[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] ajp_service::jk_ajp_common.c (2623): (tomcat1) retry 1, sleeping for 100 ms before retrying
[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] ajp_send_request::jk_ajp_common.c (1718): (tomcat1) no usable connection found, will create a new one.
[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] jk_open_socket::jk_connect.c (673): socket TCP_NODELAY set to On
[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] jk_open_socket::jk_connect.c (797): trying to connect socket 18 to ::1:8109
[Tue May 18 08:39:27 2021] [67918:139621179135744] [info] jk_open_socket::jk_connect.c (815): connect to ::1:8109 failed (errno=111)
[Tue May 18 08:39:27 2021] [67918:139621179135744] [info] ajp_connect_to_endpoint::jk_ajp_common.c (1064): (tomcat1) Failed opening socket to (::1:8109) (errno=111)
[Tue May 18 08:39:27 2021] [67918:139621179135744] [error] ajp_send_request::jk_ajp_common.c (1724): (tomcat1) connecting to backend failed. Tomcat is probably not started or is listening on the wrong port (errno=111)
[Tue May 18 08:39:27 2021] [67918:139621179135744] [info] ajp_service::jk_ajp_common.c (2774): (tomcat1) sending request to tomcat failed (recoverable), because of error during request sending (attempt=2)
[Tue May 18 08:39:27 2021] [67918:139621179135744] [error] ajp_service::jk_ajp_common.c (2795): (tomcat1) connecting to tomcat failed (rc=-3, errors=1, client_errors=0).
[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] ajp_reset_endpoint::jk_ajp_common.c (847): (tomcat1) resetting endpoint with socket -1 (socket shutdown)
[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] ajp_abort_endpoint::jk_ajp_common.c (817): (tomcat1) aborting endpoint with socket -1
[Tue May 18 08:39:27 2021] [67918:139621179135744] [debug] ajp_done::jk_ajp_common.c (3287): recycling connection pool for worker tomcat1 and socket -1
[Tue May 18 08:39:27 2021] [67918:139621179135744] [info] jk_handler::mod_jk.c (2991): Service error=-3 for worker=tomcat1
[Tue May 18 08:39:27 2021] tomcat1 localhost 0.102405
[Tue May 18 08:39:27 2021] [68200:139621321500544] [debug] do_shm_open::jk_shm.c (678): Attached shared memory /usr/local/apache/logs/jk-runtime-status.67915 [5] size=1536 workers=2 free=0 addr=0x7efc1f470000
[Tue May 18 08:39:27 2021] [68200:139621321500544] [debug] do_shm_open_lock::jk_shm.c (472): Duplicated shared memory lock /usr/local/apache/logs/jk-runtime-status.67915.lock
[Tue May 18 08:39:27 2021] [68200:139621321500544] [debug] jk_child_init::mod_jk.c (3474): Initialized mod_jk/1.2.48
[Tue May 18 08:39:29 2021] [68200:139621179135744] [debug] jk_servlet_normalize::jk_util.c (2184): URI on entering jk_servlet_normalize: [/favicon.ico]
[Tue May 18 08:39:29 2021] [68200:139621179135744] [debug] jk_servlet_normalize::jk_util.c (2278): URI on exiting jk_servlet_normalize: [/favicon.ico]
[Tue May 18 08:39:29 2021] [68200:139621179135744] [debug] map_uri_to_worker_ext::jk_uri_worker_map.c (1167): Attempting to map URI '/favicon.ico' from 3 maps
[Tue May 18 08:39:29 2021] [68200:139621179135744] [debug] find_match::jk_uri_worker_map.c (977): Attempting to map context URI '/app1*=tomcat1' source 'JkMount'
[Tue May 18 08:39:29 2021] [68200:139621179135744] [debug] find_match::jk_uri_worker_map.c (977): Attempting to map context URI '/app2*=tomcat2' source 'JkMount'
[Tue May 18 08:39:29 2021] [68200:139621179135744] [debug] find_match::jk_uri_worker_map.c (977): Attempting to map context URI '/app3*=tomcat3' source 'JkMount'
[Tue May 18 08:39:29 2021] [68200:139621179135744] [debug] jk_translate::mod_jk.c (3970): no match for /favicon.ico found
[Tue May 18 08:39:29 2021] [68200:139621179135744] [debug] jk_servlet_normalize::jk_util.c (2184): URI on entering jk_servlet_normalize: [/favicon.ico]
[Tue May 18 08:39:29 2021] [68200:139621179135744] [debug] jk_servlet_normalize::jk_util.c (2278): URI on exiting jk_servlet_normalize: [/favicon.ico]
[Tue May 18 08:39:29 2021] [68200:139621179135744] [debug] map_uri_to_worker_ext::jk_uri_worker_map.c (1167): Attempting to map URI '/favicon.ico' from 3 maps
[Tue May 18 08:39:29 2021] [68200:139621179135744] [debug] find_match::jk_uri_worker_map.c (977): Attempting to map context URI '/app1*=tomcat1' source 'JkMount'
[Tue May 18 08:39:29 2021] [68200:139621179135744] [debug] find_match::jk_uri_worker_map.c (977): Attempting to map context URI '/app2*=tomcat2' source 'JkMount'
[Tue May 18 08:39:29 2021] [68200:139621179135744] [debug] find_match::jk_uri_worker_map.c (977): Attempting to map context URI '/app3*=tomcat3' source 'JkMount'
[Tue May 18 08:39:29 2021] [68200:139621179135744] [debug] jk_map_to_storage::mod_jk.c (4074): no match for /favicon.ico found
ng-User
  • 243
  • 1
  • 6
  • 18
  • Does this answer your question? [(111)Connection refused - Apache Reverse Proxy and Tomcat 8.5.51 - Docker Compose](https://stackoverflow.com/questions/60625395/111connection-refused-apache-reverse-proxy-and-tomcat-8-5-51-docker-compos) – Olaf Kock May 18 '21 at 15:49
  • @OlafKock: I tried the solution in the other post, no success. – ng-User May 18 '21 at 17:13
  • 1
    Well, your connectors are definitely incorrect according to that post. Please edit your question with the changed configuration for the required secret – Olaf Kock May 18 '21 at 17:35
  • @OlafKock: done – ng-User May 18 '21 at 18:04
  • 1
    Replace `address="localhost"` with `address="ip6-localhost"` in `server.xml` and `localhost` with `ip6-localhost` in `workers.properties` (or whatever unique alias you have for `::1` in `/etc/hosts`): Tomcat can not bind two addresses at once, so it listens on `127.0.0.1`, while Apache HTTP Server tries to connect to `::1` and `127.0.0.1` in a round-robin fashion. – Piotr P. Karwasz May 18 '21 at 18:44
  • @PiotrP.Karwasz: Ok, that was the problem. Thank you very much! – ng-User May 18 '21 at 20:58
  • well, now your question states that the HTTP connector has the secret, not the AJP connector... but good to know that the problem is that "localhosts" now technically often need to be written in plural (for IPV4, IPV6). @PiotrP.Karwasz : you might want to convert your comment to an answer. Although I assume that this must be a duplicate, I couldn't find a matching question to link – Olaf Kock May 19 '21 at 06:37
  • I strongly suggest you use AJP instead of `mod_jk`. It is much easier to configure, and doesn't require the interaction of Tomcat creating Apache HTTPD configuration files or *vice versa* or whatever it is. – user207421 May 19 '21 at 07:45
  • @user207421 you meant migrating from `mod_jk` to `mod_proxy_ajp`? Anyway AJP will slowly be deprecated (see [this talk](https://www.youtube.com/watch?v=qUjUEvGFstI)), so `mod_proxy_http` is the way to go. – Piotr P. Karwasz May 19 '21 at 10:20

1 Answers1

1

The problem is that nowadays localhost resolves to both 127.0.0.1 and ::1. A Tomcat connector can only bind to one address. So you can either:

  1. Configure two AJP connectors and use localhost:
<Executor name="localhost-ajp" namePrefix="ajp-nio-localhost-8109-exec-"/>
<Connector protocol="AJP/1.3"
           address="127.0.0.1"
           port="8109"
           executor="localhost-ajp" />
<Connector protocol="AJP/1.3"
           address="::1"
           port="8109"
           executor="localhost-ajp" />
  1. Use a name like ip6-localhost that resolves to a single IP.
Piotr P. Karwasz
  • 12,857
  • 3
  • 20
  • 43