0

I need to handle a large amount of separated queues, the queues need to be separated but handled in the same way, I don’t want to setup multiple queue-functions so I thought of this solution but I’m not sure it’s a safe way to do: 

var connectors = GetTheConnectors();

        var tasks = new List<Task>();
        foreach (var item in connectors)
        {
            var task = Task.Factory.StartNew(() => {
                var host = new JobHost(new JobHostConfiguration
                {
                    NameResolver = new QueueNameResolver(item.Name)
                });
                host.RunAndBlock();
            });
            tasks.Add(task);
        }

        Task.WaitAll(tasks.ToArray());

If not, Do anyone have a better solution?

Cinaird
  • 785
  • 4
  • 13
  • 33

1 Answers1

1

Ideally you would write separate functions per queue, and use a single JobHost - that's the way JobHost is designed to work of course. I understand though that in your scenario your large number of queues makes that difficult?

Having a separate JobHost per queue method is pretty heavy weight - that'd be my main concern. We've had some customers resort to codegen to cover scenarios like this - they generate their functions based on input metadata. I actually have a project that is doing this: azure-webjobs-sdk-script. This library exports types that allow you to generate SDK functions in memory using IL Gen based on metadata input. For the generated function you can specify the action, to invoke when the function is triggered. You can see an example of how to run a JobHost based on dynamically generated functions in the tests here.

So mapping this to your scenario, you could generate a function per queue that you want to monitor, and have all those functions funnel into the same invoker action.

mathewc
  • 13,312
  • 2
  • 45
  • 53
  • Sorry that i did not respons sooner, In the beginning it will only be a couple of instances but it can rapidly escalate. The script-solution seems interesting the queues but it seems hard to debug/test? – Cinaird Dec 07 '15 at 12:14
  • But to clarify, It seems my approach is safe but heavy, but not more heavy than to scale one job multiple times? So the only danger is it will drain the power on the current machine? – Cinaird Dec 07 '15 at 13:24