2

I'm able to successfully load test my bot server by getting the proper auth token from Microsofts auth URL (basically through this page)

I was wondering if this was a valid test on the service considering that we're not actually hitting the bot frameworks endpoint (which has rate limiting)

Is there another way to load test a bot service wherein i can replicate the bot frameworks throttling/rate limits?

Fei Han
  • 26,415
  • 1
  • 30
  • 41
Jayaram
  • 6,276
  • 12
  • 42
  • 78
  • `not actually hitting the bot frameworks endpoint` How do you send the message(s) to your bot? – Fei Han Aug 23 '18 at 09:26
  • @FeiHan - i'm basically sending a POST message to my servers endpoint with a bearer token i get from microsoft auth. The POST message has a json body with details like conversationId, activity ID etc – Jayaram Aug 23 '18 at 10:42

2 Answers2

0

I ended up with using load test with Visual Studio and Visual Studio Team Services. The reason why I used this approach is that you can setup full path of load tests. Azure Bot Service can be either Web App or Function App with endpoint prepared for receiving messages - using HTTP POST so in the end is just web service.

You can setup load tests for different endpoints including number of hits to selected endpoint. In case of Bots you can for instance setup test with 100 fake messages sent to the bot to see the performance.

You can read more under these two links below:

Load test your app in the cloud using Visual Studio and VSTS

Quickstart: Create a load test project

enter image description here

Daniel Krzyczkowski
  • 2,732
  • 2
  • 20
  • 30
0

Unfortunately as stated in the documentation you linked, the rates are not publicly available due to how often they are adjusted.

Regarding user-side throttling- this should not actually have an effect either way as long as you simulate reasonable traffic, but even if you go a bit overboard, an individual user hitting rate-limiting would be functionally equivalent to just having a bit more traffic. The single user sending more messages to the bot is the same as three users sending the same amount of messages slightly slower and there's no limit for your bot in terms of how many customers you might have. That said, a user getting a message, reading it, and typing up a response should not put themselves into a situation where they are rate-limited.

However, regarding bot side throttling it is useful to know if your bot is sending messages too fast for the system. If you are only ever replying directly to messages from users, this will not be an issue, as the system is built with replying to each user message in mind. The only area you might run into trouble is if you are sending additional (or unsolicited) messages, however even here as long as you are within reasonable limits you should be OK. (i.e. if you aren't sending several messages back to a user as fast as possible for each message they send you, you will probably not have problems.) You can set a threshold for bot replies within your channel at some reasonable-sounding limit to test this.

If you would like to see how your bot responds in cases where throttling is occurring (and not necessarily forcing it into tripping the throttling threshold), consider setting your custom channel to send 429 errors to your bot every so often so that it has to retry sending the message.

Mark B
  • 581
  • 2
  • 14