How to interpret LocustIO's output / simulate short user visits

Question

I like Locust, but I'm having a problem interpreting the results.

e.g. my use case is that I have a petition site. I expect 10,000 people to sign the petition over a 12 hour period.

I've written a locust file that simulates user behaviour:

Some users load but don't sign petition
Some users load and submit invalid data
Some users (hopefully) successfully submit.

In real life the user now goes away (because the petition is an API not a main website).

Locust shows me things like:

with 50 concurrent users the median time is 11s
with 100 concurent users the median time is 20s

But as one "Locust" just repeats the tasks over and over, it's not really like one user. If I set it up with a swarm of 1 user, then that still represents many real world users, over a period of time; e.g. in 1 minute it might do the task 5 times: that would be 5 users.

Is there a way I can interpret the data ("this means we can handle N people/hour"), or some way I can see how many "tasks" get run per second or minute etc. (ie locust gives me requests per second but not tasks)

> "But as one "Locust" just repeats the tasks over and over, it's not really like one user. ..." What is the difference in your case? A new task iteration is pretty much the same as a new user arriving (except it will reuse the http connection) — Cyberwiz, Nov 28 '19 at 08:18
@Cyberwiz sure, but the metric I'm after is *tasks completed*. If one user did one task and exited, then the "users" figure would equal the tasks figure. I'm not saying it *should* work that way, I'm just saying that if it did, I would be able to see the metric I need. — artfulrobot, Nov 28 '19 at 09:13
Ok. Tasks (unfortunately) dont really exist at that level in locust. If you want, you could log your own fake samples, and use that as your task counter. Like this: from locust.events import request_success request_success.fire(request_type="task", name="completed", response_time=None, response_length=0) (but this will pollute your rps count) — Cyberwiz, Nov 28 '19 at 09:37
@Cyberwiz yeah, right, I thought of that but couldn't figure out how to add it in without adding in an actual other http request - if you have time to explain that in an answer, I'd give it a big green tick! I don't care about RPS, it's pretty meaninless in this context. — artfulrobot, Nov 28 '19 at 09:39

score 1 · Accepted Answer · answered Nov 28 '19 at 09:48

Tasks dont really exist on the logging level in locust.

If you want, you could log your own fake samples, and use that as your task counter. This has an unfortunate side effect of inflating your request rate, but it should not impact things like average response times.

Like this:

from locust.events import request_success 

...

      @task(1)
      def mytask(self):
          # do your normal requests
          request_success.fire(request_type="task", name="completed", response_time=None, response_length=0)

Brilliant! It works, gives me a clear read out, and as you say, it does not impact the average response time. This is great as I can now use distributed loci again. Thanks — artfulrobot, Nov 28 '19 at 10:04

score 0 · Answer 2 · answered Nov 21 '19 at 11:22

Here's the hacky way that I've got somewhere. I'm not happy with it and would love to hear some other answers.

Create class variables on my HttpLocust (WebsiteUser) class:

WebsiteUser.successfulTasks = 0

Then on the UserBehaviour taskset:


      @task(1)
      def theTaskThatIsConsideredSuccessful(self):
          WebsiteUser.successfulTasks += 1
          # ...do the work...

      # This runs once regardless how many 'locusts'/users hatch
      def setup(self):
          WebsiteUser.start_time = time.time();
          WebsiteUser.successfulTasks = 0

     # This runs for every user when test is stopped.
     # I could not find another method that did this (tried various combos)
     # It doesn't matter much, you just get N copies of the result!
     def on_stop(self):
          took = time.time() - WebsiteUser.start_time
          total = WebsiteUser.successfulTasks
          avg = took/total
          hr = 60*60/avg
          print("{} successful\nAverage: {}s/success\n{} successful signatures per hour".format(total, avg, hr)

And then set a zero wait_time and run till it settles (or failures emerge) and then stop the test with the stop button in the web UI.

Output is like

188 successful
0.2738157498075607s/success
13147.527132862522 successful signatures per hour

I think this therefore gives me the max conceivable throughput that the server can cope with (determined by changing the No. users hatched until failures emerge, or until the average response time becomes unbearable).

Obviously real users would have pauses, but that makes it harder to test the maximums.

Drawbacks

Can't use distributed Locust instances
Messy; also can't 'reset' - have to quit the process and restart for another test.

How to interpret LocustIO's output / simulate short user visits

2 Answers2