How to trigger a Container App from Azure Storage Queue?

Question

I'm a bit new to developing in NodeJS and using containers.

I'm developing a small application in NodeJS which listens to an Azure Storage Queue. Now I want the app to be triggered by a new message in the queue, but I have no idea how to do that. I'm planning to use Azure Container Apps.

I saw the methods which can be used to read messages from the queue in the Microsoft SDK, but then it's in an imperative way. Maybe something with a timer. Is it possible to execute a function in the container app when there's a new message in the queue?

Does anyone know?

Answer 1

There are 2 general approaches you can take for this:

1. Using the event source SDK (As you mentioned). This is as imperative (or declarative) as the particular SDK makes it to be. In this case it's the Azure Storage Queue SDK and you'll need something that looks like

const sleep = t => new Promise(resolve => setTimeout(resolve, t));

while (true) {
    const receivedMessages = await queueClient.receiveMessages();

    if (receivedMessages.receivedMessageItems.length > 0) {
        for (const msg of receivedMessages.receivedMessageItems) {
            // call your logic to handle and delete the msg                
            // await queueClient.deleteMessage(message.messageId, message.popReceipt);
        }
    } else {
        // queue is empty, check in 5 seconds
        await sleep(5000)
    }
}

2. Use some other framework that abstracts that away for you, and calls your method on a message. For example, dapr or Azure Functions.

• For Dapr, create a dapr component that looks like https://docs.dapr.io/reference/components-reference/supported-bindings/storagequeues/ (note that if you're deploying on Azure Container Apps, you'll have to define that component using Azure's ARM (or bicep) or using the az cli. This example creates a blob component, but it should be very similar to queue components. So you'll have a dapr component that looks like:

# dapr-queue-component.yaml
apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: ricardos-queue
spec:
  type: bindings.azure.storagequeues
  version: v1
  metadata:
  - name: accountName
    value: "yourAccountName"
  # you can use managed identity in Azure to not have to set this here
  # the doc linked above shows how to do that.
  # - name: accountKey
  #  value: "***********"
  - name: queueName
    value: "myqueue"

then create it:

➜ az containerapp env dapr-component set \
    --name $CONTAINERAPPS_ENVIRONMENT --resource-group $RESOURCE_GROUP \
    --dapr-component-name ricardos-queue \
    --yaml dapr-queue-component.yaml

and finally your code will look like this (see dapr docs for more details):

//dependencies 
import { DaprServer, CommunicationProtocolEnum } from '@dapr/dapr'; 

//code
const daprHost = "127.0.0.1"; 
const serverHost = "127.0.0.1";
const serverPort = "6002"; 
const daprPort = "3602"; 

start().catch((e) => {
    console.error(e);
    process.exit(1);
});

async function start() {
    const server = new DaprServer(serverHost, serverPort, daprHost, daprPort, CommunicationProtocolEnum.HTTP);
    await server.binding.receive('ricardos-queue', async (msg) => {
        // handle msg
    });
    await server.startServer();
}

• For Azure Functions: You can install the Azure Functions cli then do

➜ func init . --javascript
➜ func new --template "Azure Queue Storage trigger" --name my-function-name
➜ cat my-function-name/function.json
{
  "bindings": [
    {
      "name": "myQueueItem",
      "type": "queueTrigger",
      "direction": "in",
      "queueName": "js-queue-items", # change to your queue name
      "connection": "CONNECTION_STRING" # env var with a connection string in it. or value in local.settings.json
    }
  ]
}

➜ cat my-function-name/index.js

module.exports = async function (context, msg) {
    # handle msg
};

when deploying on Azure Container Apps, you can set your scale rules to someting like this (see docs here for more details about defining scale rules so your application will scale to 0 if no messages are on the queue)

    "scale": {
        "maxReplicas": 10,
        "minReplicas": 0,
        "rules": [
          {
            "name": "my-queue-scale-rule",
            "custom": {
              "type": "azure-queue",
              "metadata": {
                "queueName": "my-queue",
                "queueLength": "50" // each instance can handle up to 50 messages with a max of 10 instances
              },
              "auth": [
                {
                  "secretRef": "my-connection-string",
                  "triggerParameter": "connection"
                }
              ]
            }
          }
        ]
      }

You can always use a scale rule like that regardless of the SDK or framework you're using (Azure Storage SDK, dapr, Azure Functions, etc) to scale your queue processor up and down depending on the load