How to determine job's queue at runtime

Question

Our web app allows the end-user to set the queue of recurring jobs on the UI. (We create a queue for each server (use server name) and allow users to choose server to run) How the job is registered:

RecurringJob.AddOrUpdate<IMyTestJob>(input.Id, x => x.Run(), input.Cron, TimeZoneInfo.Local, input.QueueName);

It worked properly, but sometimes we check the log on Production and found that it runs on the wrong queue (server). We don't have more access to Production so that we try to reproduce at Development but it's not happened.

To temporarily fix this issue, we need to get the queue name when the job running, then compare it with the current server name and stop it when they are diferent.

Is it possible and how to get it from PerformContext?

Noted: We use HangFire version: 1.7.9 and ASP.NET Core 3.1

score 1 · Accepted Answer · edited Aug 19 '21 at 06:17

You may have a look at https://github.com/HangfireIO/Hangfire/pull/502

A dedicated filter intercepts the queue changes and restores the original queue.

I guess you can just stop the execution in a very similar filter, or set a parameter to cleanly stop execution during the IElectStateFilter.OnStateElection phase by changing the CandidateState to FailedState

Maybe your problem comes from an already existing filter which messes up with the queues.

Here is the code from the link above :

    public class PreserveOriginalQueueAttribute : JobFilterAttribute, IApplyStateFilter
    {
        public void OnStateApplied(ApplyStateContext context, IWriteOnlyTransaction transaction)
        {
            var enqueuedState = context.NewState as EnqueuedState;
    
            // Activating only when enqueueing a background job
            if (enqueuedState != null)
            {
                // Checking if an original queue is already set
                var originalQueue = JobHelper.FromJson<string>(context.Connection.GetJobParameter(
                    context.BackgroundJob.Id,
                    "OriginalQueue"));
    
                if (originalQueue != null)
                {
                    // Override any other queue value that is currently set (by other filters, for example)
                    enqueuedState.Queue = originalQueue;
                }
                else
                {
                    // Queueing for the first time, we should set the original queue
                    context.Connection.SetJobParameter(
                        context.BackgroundJob.Id,
                        "OriginalQueue",
                        JobHelper.ToJson(enqueuedState.Queue));    
                }
            }
        }
    
        public void OnStateUnapplied(ApplyStateContext context, IWriteOnlyTransaction transaction)
        {
        }
    }

score 0 · Answer 2 · answered Aug 19 '21 at 04:55

I have found the simple solution: since we have known the Recurring Job Id, we can get its information from JobStorage and compare it with the current queue (current server name):

public bool IsCorrectQueue()
{
    List<RecurringJobDto> recurringJobs = Hangfire.JobStorage.Current.GetConnection().GetRecurringJobs();
    var myJob = recurringJobs.FirstOrDefault(x => x.Id.Equals("My job Id"));
    var definedQueue = myJob.Queue;
    var currentServerQueue = string.Concat(Environment.MachineName.ToLowerInvariant().Where(char.IsLetterOrDigit));

    return definedQueue == "default" || definedQueue == currentServerQueue;
}

Then check it inside the job:

public async Task Run()
{
    //Check correct queue
    if (!IsCorrectQueue())
    {
        Logger.Error("Wrong queue detected");
        return;
    }

    //Job logic
}

How to determine job's queue at runtime

2 Answers2