0

I have hangfire implemented on my API and I'm using SQL Server with it. It has worked in each environment until I released it to production. All my jobs are just sitting as "Enqueued" and aren't starting at all. I've looked and there are servers available but nothing is happening.

I then included hangfire debug logs and I see the following error. Any ideas on what is wrong:

Server PrdEnv heartbeat successfully sent

Execution loop Worker:cea1c95b caught an exception and will be retried in 00:05:00
System.Data.SqlClient.SqlException (0x80131904): The target table 'JQ' of the DML statement cannot have any enabled triggers if the statement contains an OUTPUT clause without INTO clause.

at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action1 wrapCloseInAction) at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose) at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady) at System.Data.SqlClient.SqlDataReader.TryConsumeMetaData() at System.Data.SqlClient.SqlDataReader.get_MetaData() at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString) at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, SqlDataReader ds) at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior) at Dapper.SqlMapper.ExecuteReaderWithFlagsFallback(IDbCommand cmd, Boolean wasClosed, CommandBehavior behavior) at Dapper.SqlMapper.QueryMultipleImpl(IDbConnection cnn, CommandDefinition& command) at Dapper.SqlMapper.QueryMultiple(IDbConnection cnn, String sql, Object param, IDbTransaction transaction, Nullable1 commandTimeout, Nullable1 commandType) at Hangfire.SqlServer.SqlServerJobQueue.<>c__DisplayClass10_0.<DequeueUsingSlidingInvisibilityTimeout>b__1(DbConnection connection) at Hangfire.SqlServer.SqlServerStorage.UseConnection[T](DbConnection dedicatedConnection, Func2 func)
at Hangfire.SqlServer.SqlServerJobQueue.DequeueUsingSlidingInvisibilityTimeout(String[] queues, CancellationToken cancellationToken)
at Hangfire.SqlServer.SqlServerJobQueue.Dequeue(String[] queues, CancellationToken cancellationToken)
at Hangfire.SqlServer.SqlServerConnection.FetchNextJob(String[] queues, CancellationToken cancellationToken)
at Hangfire.Server.Worker.Execute(BackgroundProcessContext context)
at Hangfire.Server.BackgroundProcessDispatcherBuilder.ExecuteProcess(Guid executionId, Object state)
at Hangfire.Processing.BackgroundExecution.Run(Action`2 callback, Object state) ClientConnectionId:dcb0bd28-c8c4-42bf-95dd-84a1af466993
Error Number:334,State:1,Class:16

Jobs

marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
user2661305
  • 311
  • 1
  • 4
  • 16
  • In all your "other" environments, does the table JQ have any triggers? – SMor Apr 05 '22 at 18:06
  • I've no idea what the table JQ is, its not in my database which is whats so confusing. I'm also seeing the following 2 errors also: Execution loop CountersAggregator:f7464ea7 caught an exception and will be retried in 00:05:00 System.Data.SqlClient.SqlException (0x80131904): Incorrect syntax near the keyword 'Key'. Execution loop ExpirationManager:6e87d5c2 caught an exception and will be retried in 00:05:00 System.Data.SqlClient.SqlException (0x80131904): Incorrect syntax near the keyword 'Key'. – user2661305 Apr 05 '22 at 18:15
  • Seems like that is a table (alias of a table perhaps) used internally by hangfire - and guessing it might be located in a different database than you expect. Seems like you will need to debug it yourself to understand why this occurs. – SMor Apr 05 '22 at 19:48

1 Answers1

0

It turns out our dbas added audit triggers to our database and some of these were added to the hangfire schema. This meant hangfire wasnt able to pull from the jobqueue table because it did an update statement with an output, which isn't allowed with a trigger on the table

user2661305
  • 311
  • 1
  • 4
  • 16