0

Elastic Beanstalk: Create a new environment: .ebextensions with SSL certificate fails to start (tomcat-single-instance)

I am trying to create a new environment with the current production WAR package.

New instance deployment fails and comes up with "Green" status. We originally followed this sample to create the .ebextensions (https://s3.amazonaws.com/elasticbeanstalk-single-instance-ssl-demo/tomcat-single-instance.zip) and extended as described here https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/https-singleinstance-tomcat.html .

New Platform: Managed, Tomcat 8.5 with Java 8 on 64bit Amazon Linux
Platform version: 3.4.1(Recommended)

Error:

httpd24-tools conflicts with httpd-tools-2.2.34-1.16.amzn1.x86_64
mod24_ssl conflicts with 1:mod_ssl-2.2.34-1.16.amzn1.x86_64
httpd24 conflicts with httpd-2.2.34-1.16.amzn1.x86_64

To resolve this error, I replaced

packages:
  yum:
    mod_ssl : []

with

packages:
  yum:
    mod24_ssl.x86_64 : []

But that caused this error:

Httpd configuration detected in the '.ebextensions/httpd' directory. AWS Elastic Beanstalk will no longer manage the httpd configuration for this environment.
Executing: /usr/sbin/apachectl -t -f /var/elasticbeanstalk/staging/httpd/conf/httpd.conf
httpd: Syntax error on line 21 of /var/elasticbeanstalk/staging/httpd/conf/httpd.conf: Include/IncludeOptional: No matches for the wildcard '*.conf' in '/etc/httpd/conf.d/elasticbeanstalk', failing
Failed to execute '/usr/sbin/apachectl -t -f /var/elasticbeanstalk/staging/httpd/conf/httpd.conf'
Failed to execute '/usr/sbin/apachectl -t -f /var/elasticbeanstalk/staging/httpd/conf/httpd.conf' (Executor::NonZeroExitStatus)

AWS is asking to replace the current production server (Amazon Linux/2.3.1) without delay as it is "Retired". I have posted this issue on AWS Forum as well. Please help.

Roy
  • 673
  • 11
  • 21
  • I think you are in the right path when applying the new mod ssl package. @Roy, please, can you indicate how does your `httpd.conf` file looks like? I assume you included some configuration like the examples provided in the section `Extending and overriding the default Apache configuration' in the [AWS documentation](https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/java-tomcat-proxy.html), is it right? – jccampanero Dec 30 '20 at 19:20
  • @jccampanero Thank you for your response. Only difference is that I don't have "myconf.conf" under "conf.d" folder. I followed AWS sample (https://s3.amazonaws.com/elasticbeanstalk-single-instance-ssl-demo/tomcat-single-instance.zip). My production server is running fine over last two years on Amazon Linux 2.3.1. Only issue is that I can't migrate to AWS recommended Amazon Linux 3.4.1. – Roy Dec 31 '20 at 06:29
  • I included an answer @Roy, I hope it helps. – jccampanero Dec 31 '20 at 13:52

2 Answers2

2

Save yourself pain. Do not configure SSL in your Tomcat Server, do it on an AWS Elastic Load Balancer ELB.

Derrops
  • 7,651
  • 5
  • 30
  • 60
  • Thank you for your suggestion. I am using ebextensions with SSL in production (Amazon Linux 2.3.1). Once the script is configured, it works perfectly. The single instance SSL configuration costs only 30% compared to Elastic Load Balancer. Hope someone can answer my question. – Roy Dec 29 '20 at 06:22
  • You can also do SSL config in Cloudfront. Honestly Roy my old friend, you will just have pain, gain nothing, lose the ability to auto-renew certs, have a security risk with managing private certs, it's just no good. Not sure where you get those costs from, but pretty sure after you spend more time and run all the numbers, you are losing $$$$$ – Derrops Dec 29 '20 at 06:30
  • 1
    I couldn't agree more with this. Configuring SSL through ALB/ELB is so much simpler and more secure than manually setting it all up. – Alex Bailey Jan 01 '21 at 14:41
2

As indicated in the AWS documentation:

Starting with Tomcat platform version 3.0.0 configurations, which were released with the Java with Tomcat platform update on May 24, 2018, Apache 2.4 is the default proxy of the Tomcat platform.

After digging into the problem, as can be seeing for the comments and the companion chat, the actual solution for the problem was either create or clone the existing environment, with the idea of being able to use a fresh beanstalk environment for Amazon Linux 3.4.2, without any customization.

Then, in order to avoid the mentioned problem with SSL, the .ebextensions directory should only include a convenient ssl.conf and the environment.config script provided in the sample zip file indicated in the question, without the packages section because mod_ssl is already installed in the beanstalk image.

In this specific case, there were some additional problems related with the creation of the files required for logging. After adjusting the path to one in which the application has the ability to write, the default for Tomcat in Beanstalk, /var/logs/tomcat8, everything seems to work properly.

jccampanero
  • 50,989
  • 3
  • 20
  • 49
  • Thank you for your answer. I have already tried IncludeOptional in the httpd.conf file. That bypassed the wildcard matching but gave me: AH00534: httpd: Configuration error: No MPM loaded. I couldn't make any progress after that. – Roy Dec 31 '20 at 19:19
  • Thank you. I have created an account but it says that the answer is "SUBSCRIBER EXCLUSIVE CONTENT". If you have access to the answer, please post the answer here. – Roy Jan 01 '21 at 05:33
  • Hi @Roy. First, sorry for the quick replies yesterday, I had no time to explain me better. As I told you, I think the problem is related with the configuration of the MPM module required by Apache 2.4. I updated the answer with the information formerly described in my previous comments. I hope it helps. – jccampanero Jan 01 '21 at 11:04
  • Thank you @jccampanero. Now I am facing new error. I tried all three mod_mpm options one by one. But all resulted in same error: Executing: /usr/sbin/apachectl -t -f /var/elasticbeanstalk/staging/httpd/conf/httpd.conf AH00526: Syntax error on line 16 of /etc/httpd/conf.d/autoindex.conf: Invalid command 'IndexOptions', perhaps misspelled or defined by a module not included in the server configuration Failed to execute '/usr/sbin/apachectl -t -f /var/elasticbeanstalk/staging/httpd/conf/httpd.conf' Failed to execute '/usr/sbin/apachectl -t -f /var/elasticbeanstalk/staging/httpd/conf/httpd.conf' – Roy Jan 01 '21 at 16:10
  • Hi Roy. I am sorry to hear you got a new error. I think is unrelated with the previous one. The problem seems to be related with some configuration required by the autoindex module. You can try to load the module in your `httpd.conf` file, `LoadModule autoindex_module modules/mod_autoindex.so` - please, verify the existence of the library- although I think it will be worth value to try to understand where this functionality is required. I suppose that it is loaded when including the different configuration under the `conf.d` directory when `IncludeOptional conf.d/*.conf` is applied. – jccampanero Jan 01 '21 at 16:31
  • If you have the ability to ssh into the VM that supports your installation, you can issue `apachectl configtest` to verify if your configuration is correct prior to actually start Apache. – jccampanero Jan 01 '21 at 16:35
  • @Roy If the configuration of Apache 2.4 gets problematic, my best advice is that, if you have the opportunity, create a new beanstalk environment from scratch for Amazon Linux 3.4.1, without any customization. It will provide you a fresh and working configuration of tomcat and Apache. With this working configuration in place, try then to customize the SSL behavior, it should be straightforward, instead of using and try to adapt the old configuration described in the zip file you provided. – jccampanero Jan 01 '21 at 17:46
  • Appreciate your help @jccampanero. I can launch Amazon Linux 3.4.1 instance without the .ebextensions in my WAR package. That works fine without SSL. However I need to avoid making any change through SSH to the EC2. Followed your suggestions and at this time I have reached to the ssl.conf file. New error is: Executing: /usr/sbin/apachectl -t -f /var/elasticbeanstalk/staging/httpd/conf/httpd.conf AH00526: Syntax error on line 18 of /etc/httpd/conf.d/ssl.conf: Invalid command 'SSLPassPhraseDialog'. – Roy Jan 01 '21 at 20:02
  • You are welcome @Roy ;). The problem is now SSL. First, verify if the module is installed by looking into the `/etc/httpd/modules/` folder and trying to find `mod_ssl.so`. If it is not installed, you need to install it using the `mod24_ssl.x86_64` package like you indicated in the question. Any way, after installing it or because the module was already installed, you need to configure `ssl.conf`. Place only this file into the `.ebextensions/httpd/conf.d`. Please, verify when deployed that the correct file is there. See next comment – jccampanero Jan 01 '21 at 20:55
  • This file needs to be self contained. As a consequence, you need to include in it the `LoadModule ssl_module modules/mod_ssl.so` directive. See, for example, this [blog](https://chrisjean.com/adding-ssl-support-to-apache-on-centos/). Of course, provide in your extensions the necessary configuration for your certificate and private key. There is no need to override `https.conf`. I hope it helps. – jccampanero Jan 01 '21 at 21:00
  • Hi @jccampanero: The LoadModule ssl_module modules/mod_ssl.so directive is already there in the AWS sample zip file (mentioned in the question). I only added my SSL cert in the sample and deployed in production system and that is still working fine. Problem is with Amazon Linux 3.4.1 migration. As you suggested, I have just tried mod24_ssl and mod24_ssl.x86_64 both in environment.config and ssl.config. Sometimes EC2 deployment is failing permanently and I had to terminate it. – Roy Jan 02 '21 at 09:09
  • Hi @Roy, morning.But there is no need to upload a custom `httpd.conf` file.The idea is only provide the necessary SSL configuration, i.e., your crypto material and `ssl.conf`.When you created the new env from scratch, without SSL, did you modify the `httpd.conf` file? Why the OS complained about SSL initially? If you create the environment from scratch, without SSL configuration, it should not provide any error.You can connect through ssh to the ec2 machine, only to check if mod_ssl is already installed, and in order to review how the `httpd.conf` file provided by AWS from scratch looks like – jccampanero Jan 02 '21 at 10:04
  • @Roy, first, try to verify if the installation of mod_ssl is or not necessary. As you can see in the [documentation you indicated](https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/https-singleinstance-tomcat.html), although I am aware that they provide a different configuration file in the zip, they do not mention that the installation of `mod_ssl` is necessary. Just include the environment.config without the yum packages installation, only with you crypto material config, and the `ssl.conf` file. I suggest you to include the LoadModule directive in `ssl.conf`. See next comment – jccampanero Jan 02 '21 at 10:12
  • but perhaps, depending on the Apache configuration, it is even unnecessary. Your `.ebextensions` directory will contain only the two mentioned files, without the yum package section. If you consider necessary, we can chat. – jccampanero Jan 02 '21 at 10:13
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/226731/discussion-between-roy-and-jccampanero). – Roy Jan 02 '21 at 11:29
  • Sorry for the late reply @Roy, I am in the chat, although I need to leave now. I will join the chat again in a couple of hours. If you wish, please, join me there – jccampanero Jan 02 '21 at 12:02