2

I have developed a chatbot using azure bot framework SDK, it is using python aiohttp and asyncio to handle requests. It is deployed on the azure app service and is running through gunicron, but its ability to handle multiple users is low it can only handle 3-4 users per second, I like to increase its capacity.

To do so I did some research and found that with multiple gunicron workers we can handle more users. But when I am using 3 workers instead of 1 on one CPU core, the bot's state is not maintained and it again asks the same question which user has already answered. I am using waterfall dialog for my bot.

I tried to have a JSON file that maintains the user state in local, but that is also not working.

Please help me with the issue as there are at least 1000 users per second is forecasted.

NOTE:- Scaling vertically does not resolve the issue and it does not increase capacity to handle even 100 users per sec. I am a beginner at bot framework and azure.

Any help would be appreciated.

Thank you

Krupal Purohit
  • 51
  • 6
  • 1
    Are you using `MemoryStorage`? Use a persistent storage: https://learn.microsoft.com/en-us/azure/bot-service/bot-builder-howto-v4-storage?view=azure-bot-service-4.0&tabs=python#using-cosmos-db – aaron Feb 01 '22 at 07:17
  • yes, I am using MemoryStorage. Can you help me with how to replace MemoryStorage? This is how I have used MemoryStorage MEMORY = MemoryStorage() USER_STATE = UserState(MEMORY) CONVERSATION_STATE = ConversationState(MEMORY) – Krupal Purohit Feb 01 '22 at 07:21
  • Take a look at the guide that I linked in my comment about how to "Set up a Cosmos DB resource", and then replace `MemoryStorage` with `CosmosDbPartitionedStorage`. – aaron Feb 01 '22 at 07:31
  • @aaron it seems cosmos DB increases the cost. Can we use any other persistent storage like MongoDB? – Krupal Purohit Feb 01 '22 at 08:17
  • 1
    You can implement custom storage for your bot: https://learn.microsoft.com/en-us/azure/bot-service/bot-builder-custom-storage?view=azure-bot-service-4.0 – aaron Feb 01 '22 at 08:29

1 Answers1

0

Thank you Aaron. Posting your discussions as answer to help other community members.

Persistent storage can be used on behalf of MemoryStorage. It can be replaced with CosmosDB with CosmosDbPartitionedStorage.

Customer storage can also be implemented for your bot with the below link.

Reference link: https://learn.microsoft.com/en-us/azure/bot-service/bot-builder-custom-storage?view=azure-bot-service-4.0

SureshBabu
  • 418
  • 2
  • 9