1

First, I use the server environment

  • sever: django + nginx + uwsgi
  • cloud: docker + AWS ECS
  • logging: AWS CloudWatch log service + watchtower third party app

If I run server locally with python manage.py runserver, the log is stored well in the CloudWatch log. However, if I build my project with docker and docker run --rm -it -p 8080: 80 image_name command, the following error occurs.

Traceback (most recent call last):
  File "/usr/lib/python3.5/logging/config.py", line 558, in configure
    handler = self.configure_handler(handlers[name])
  File "/usr/lib/python3.5/logging/config.py", line 731, in configure_handler
    result = factory(**kwargs)
  File "/usr/local/lib/python3.5/dist-packages/watchtower/__init__.py", line 78, in __init__
    self.cwl_client = (boto3_session or boto3).client("logs")
  File "/usr/local/lib/python3.5/dist-packages/boto3/__init__.py", line 83, in client
    return _get_default_session().client(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/boto3/session.py", line 263, in client
    aws_session_token=aws_session_token, config=config)
  File "/usr/local/lib/python3.5/dist-packages/botocore/session.py", line 836, in create_client
    client_config=config, api_version=api_version)
  File "/usr/local/lib/python3.5/dist-packages/botocore/client.py", line 70, in create_client
    verify, credentials, scoped_config, client_config, endpoint_bridge)
  File "/usr/local/lib/python3.5/dist-packages/botocore/client.py", line 224, in _get_client_args
    verify, credentials, scoped_config, client_config, endpoint_bridge)
  File "/usr/local/lib/python3.5/dist-packages/botocore/args.py", line 45, in get_client_args
    endpoint_url, is_secure, scoped_config)
  File "/usr/local/lib/python3.5/dist-packages/botocore/args.py", line 103, in compute_client_args
    service_name, region_name, endpoint_url, is_secure)
  File "/usr/local/lib/python3.5/dist-packages/botocore/client.py", line 297, in resolve
    service_name, region_name)
  File "/usr/local/lib/python3.5/dist-packages/botocore/regions.py", line 122, in construct_endpoint
    partition, service_name, region_name)
  File "/usr/local/lib/python3.5/dist-packages/botocore/regions.py", line 135, in _endpoint_for_partition
    raise NoRegionError()
botocore.exceptions.NoRegionError: You must specify a region.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "django_app/manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.5/dist-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.5/dist-packages/django/core/management/__init__.py", line 341, in execute
    django.setup()
  File "/usr/local/lib/python3.5/dist-packages/django/__init__.py", line 22, in setup
    configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
  File "/usr/local/lib/python3.5/dist-packages/django/utils/log.py", line 75, in configure_logging
    logging_config_func(logging_settings)
  File "/usr/lib/python3.5/logging/config.py", line 795, in dictConfig
    dictConfigClass(config).configure()
  File "/usr/lib/python3.5/logging/config.py", line 566, in configure
    '%r: %s' % (name, e))
ValueError: Unable to configure handler 'watchtower': You must specify a region.

The error message points out the region problem and I do not know how to fix it. The django logging setting is shown below.

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'verbose': {
            'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
        },
        'simple': {
            'format': '%(levelname)s %(message)s'
        },
    },
    'handlers': {
        'watchtower': {
            'level': 'DEBUG',
            'class': 'watchtower.CloudWatchLogHandler',
            'formatter': 'verbose',
        },
        'console': {
            'level': 'INFO',
            'class': 'logging.StreamHandler',
        },
    },
    'loggers': {
        'django': {
            'handlers': ['watchtower', 'console'],
            'level': 'INFO',
            'propagate': True,
        },
        'django.user': {
            'handlers': ['watchtower'],
            'level': DJANGO_LOG_LEVEL,
            'propagate': False,
        },
        'django.partner': {
            'handlers': ['watchtower'],
            'level': DJANGO_LOG_LEVEL,
            'propagate': False,
        }
    }
}

Where is the problem?

byunghyun park
  • 499
  • 1
  • 8
  • 25

2 Answers2

2

You need to provide aws credentials inside your container. You can mount them dinamically using a volume:

docker run -v $HOME/.aws:/root/.aws --rm -it -p 8080: 80 image_name

Your credentials should be locally in $HOME/.aws, then they are mounted into the home directory of the user that runs your application (change /root to another dir if the user is other)

Robert
  • 33,429
  • 8
  • 90
  • 94
  • Thank you. You could create a docker container locally with your command. – byunghyun park Jun 01 '17 at 02:05
  • hi @Robert Actually, I just want to ask you one more question. I uploaded a container to AWS ECS and can not do a 'docker run' at this time. So I added the 'COPY .aws / /.aws' command to the Dockerfile I have verified that this method passes credentials. The 'docker exec' looks inside the container. The .aws directory is normally copied to the root path. But it does not pass credentials like 'docker run -v $ HOME / .aws: /root/.aws --rm -it -p 8080: 80 image_name'. Is there any other way? – byunghyun park Jun 01 '17 at 04:20
  • ah... My first comment was wrong. :( You could create => I could create – byunghyun park Jun 01 '17 at 04:34
  • 1
    I wrote the above question in the following url. thank you https://stackoverflow.com/questions/44299786/how-do-i-get-aws-credentials-in-the-aws-ecs-docker-container – byunghyun park Jun 01 '17 at 06:13
  • Ok I'll take a look there! – Robert Jun 01 '17 at 10:40
  • In case you're getting your AWS credentials from system environment variables, you might want to set those using `export =` from you terminal. This is most likely for AWS ElasticBeanstalk users. – JovanToroman Aug 15 '22 at 15:28
0

The AWS region is not configured inside your container, and probably neither are the credentials. Have a look at the documentation or here.

The files generated by the CLI for the profile configured in the previous section look like this:

~/.aws/credentials

[default] aws_access_key_id=AKIAIOSFODNN7EXAMPLE aws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

~/.aws/config

[default] region=us-west-2 output=json

The following settings are supported.

aws_access_key_id – AWS access key.

aws_secret_access_key – AWS secret key.

aws_session_token – AWS session token. A session token is only required if you are using temporary security credentials.

region – AWS region.

output – output format (json, text, or table)

Raf
  • 9,681
  • 1
  • 29
  • 41
  • However, if I change django logging to default, docker run is no problem, and pushing to AWS ECS also succeeds, so the server works well with nginx. Is it a credentials issue even though I get an error when using watchtower handler? The local .aws folder is of course set with credentials. – byunghyun park May 31 '17 at 09:20
  • Thank you. The credentials problem was right as you said. Thanks to the local test was successful. @Raf – byunghyun park Jun 01 '17 at 04:23