Boto3 - How to keep session alive

Question

I have a process that is supposed to run forever and needs to updates data on a S3 bucket on AWS. I am initializing the session using boto3:

        session = boto3.session.Session()
        my_s3 = session.resource(
            "s3",
            region_name=my_region_name,
            aws_access_key_id=my_aws_access_key_id,
            aws_secret_access_key=my_aws_secret_access_key,
            aws_session_token=my_aws_session_token,
        )

Since the process is supposed to run for days, I am wondering how I can make sure that the session is kept alive and working. Do I need to re-initialize the session sometimes?

Note: not sure if it is useful, but I have actually multiple threads each using its own session.

Thanks!

score 0 · Answer 1 · answered Mar 26 '21 at 22:40

There is no concept of a 'session'. The Session() is simply an in-memory object that contains information about how to connect to AWS.

It does not actually involve any calls to AWS until an action is performed (eg ListBuckets). Actions are RESTful, and return results immediately. They do not keep open a connection.

A Session is not normally required. If you have stored your AWS credentials in a config file using the AWS CLI aws configure command, you can simply use:

import boto3

s3_resource = boto3.resource('s3')

The Session, however, is useful if you are using temporary credentials returned by an AssumeRole() command, rather than permanent credentials. In such a case, please note that credentials returned by AWS Security Token Service (STS) such as AssumeRole() have time limitations. This, however, is unrelated to the concept of a boto3 Session.

score 0 · Answer 2 · answered May 15 '23 at 07:37

As of botocore#2766, you can use the tcp_keepalive parameter (docs):

import botocore, boto3, environs

env = environs.Env()

S3_CLIENT = boto3.session.Session().client(  # this init style makes the client thread-safe
    "s3",
    config=botocore.client.Config(
        tcp_keepalive=env.bool("AWS_TCP_KEEPALIVE", True),
        max_pool_connections=env.int("AWS_S3_CONNECTIONS", 100),
        retries={
            "max_attempts": env.int("AWS_MAX_ATTEMPTS", 6),
            "mode": env("AWS_RETRY_MODE", "adaptive"),
        },
    ),
)

Boto3 - How to keep session alive

2 Answers2