Example for using Locust with Boto3

Question

I am trying to use Locust together with Boto3 for load testing of S3, in particular for PutObject.

I found a tutorial and another github project, but both have been written for the older locustio package and seem incompatible with the current version locust 2.5.1.

https://medium.com/@allankp/populating-dashboards-with-boto3-and-locust-ff38b113349a https://github.com/twosigma/locust-s3

I have made a short attempt at adapting this code to newer versions but this seems to require a good understanding of locust which I do not possess yet.

Does anyone have a simple beginner's example to share? Or would you rather recommend using the old locustio package or another tool altogether?

Many thanks.

score 2 · Answer 1 · answered Jan 03 '22 at 15:05

Helped by this example in the official docs and Cyberwiz's answer, I created this bare-bones solution for my specific problem.

import time
import boto3
from locust import User, task, constant, events


class BotoClient:
    def __init__(self):
        self.s3_client = boto3.client(
            's3',
            aws_access_key_id="ACCESS_KEY",
            aws_secret_access_key="SECRET_KEY"
        )

    def send(self, f, bucket_name, object_name):

        request_meta = {
            "request_type": "upload file",
            "name": "S3",
            "start_time": time.time(),
            "response_length": 0,
            "response": None,
            "context": {},
            "exception": None,
        }
        start_perf_counter = time.perf_counter()

        try:
            self.s3_client.upload_fileobj(f, bucket_name, object_name)
        except Exception as e:
            request_meta['exception'] = e

        request_meta["response_time"] = (time.perf_counter() - start_perf_counter) * 1000

        events.request.fire(**request_meta)


class BotoUser(User):
    abstract = True

    def __init__(self, env):
        super().__init__(env)
        self.client = BotoClient()

    wait_time = constant(1)


class MyUser(BotoUser):

    wait_time = constant(1)

    @task
    def send_request(self):
        with open("FILE_NAME", 'rb') as f:
            self.client.send(f, "BUCKET_NAME", "OBJECT_NAME")

score 0 · Answer 2 · answered Dec 21 '21 at 21:12

The changes needed to adapt the code to version 1+ of Locust are very small.

Quoting the documentation changelog:

Locust class renamed to User

We’ve renamed the Locust and HttpLocust classes to User and HttpUser. The locust attribute on TaskSet instances has been renamed to user.

Example for using Locust with Boto3

2 Answers2