AWS Fargate hostname not doable?

Question

I am trying to set up a simple app. It is dragged from https://budgetapp.docsapp.io/ and meant to be working somewhere in public. My task is to deploy it in most automate way and expose metrics of java machine to the public (remote jConsole). Eventually, it should be stood up in around 15 exact copies.

For this task I chose AWS Fargate. It sets up the app beautifully, it gives me back the Metrics on admin port (it is deployed by Dropwizard). What I struggle with is monitoring. It should be real live and showing at least CPU and mem usage. I am able to do it locally on docker, but Fargate is missing one crucial thing. Setting up hostname for deployed task.

Error that I am getting: Error: Exception thrown by the agent : java.net.MalformedURLException: Local host name unknown: java.net.UnknownHostException: 578463faab0f: 578463faab0f: System error. It is due to missing entry in /etc/hosts file with container's uname -n. It seems like I cannot set it up! Is this possible to be done somehow?

I am running my java service like this: java -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=127.0.0.1 -Djava.rmi.server.useLocalHostname=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=5000 -Dcom.sun.management.jmxremote.rmi.port=5000 -jar budgetapp.jar server config.yml

I am happy to take any advice!

nope, no ssh access. Fargate does not allow it. I don't have any console access to any container or underlying host — agat90, Mar 31 '18 at 22:03

score 2 · Answer 1 · answered Mar 31 '18 at 22:12

2

This is a known issue in the ecs agent. Have you tried this as a workaround:

echo "$(ip a | grep -A2 eth1 | grep inet | awk '{print $2}' | sed 's#/.*##g' ) $(hostname)" >> /etc/hosts

You can grab the ip and modify /etc/host during the entry point in your container.

I believe there is an ecs agent fix on its way, but I cannot find the issue on GitHub.

answered Mar 31 '18 at 22:12

Roy Kachouh

1,855
16
24

should I be able to do it in Fargate deployment? – agat90 Apr 03 '18 at 08:56
You will want to add that workaround as part of your ENTRYPOINT in your Dockerfile – Roy Kachouh Apr 05 '18 at 06:27

aaronsteers · Answer 2 · 2019-06-07T23:38:32.513

I ran into the same issue on ECS Fargate, with code that worked fine in docker and in CI/CD pipeline. I did not yet have a bootstrap script, so it was a little work to integrate Roy's solution.

Here's what worked for me:

Create a ecs_bootloader.sh shell script into a directory which was already being copied into the image (bin in my case):

#!/bin/bash

echo "$(ip a | grep -A2 eth1 | grep inet | awk '{print $2}' | sed 's#/.*##g' ) $(hostname)" >> /etc/hosts

#My command here:
python bin/run_all.py

Modified my ECS task definition, changing ENTRYPOINT and command to:

ENTRYPOINT: bin/ecs_bootstrap.sh
COMMAND:    bin/run_all.py

NOTES:

This could be improved by calling COMMAND from the ecs_bootstrap.sh script. (Currently COMMAND is ignored by the bootstrap script.)
Because I was already copying the bin folder to the container, I did not have to modify my Dockerfile.

score 0 · Accepted Answer · answered Apr 05 '18 at 09:37

Thanks for all the help. The final answer was a combination of what I read in the internet and answers here.

The key was to change java.rmi.server.hostname value to its external IP address, which is the actual address that I will use while filling jConsole form. (It turns out that this value is not the interface's where service is bind, but the value of IP address that external monitoring tools will use to access it.)

As well, in entrypoint I need to add an external IP to container's hostname, just for the sake of the hostname being added to the /etc/hosts

AWS Fargate hostname not doable?

3 Answers3