I'm using Hangfire and for some reason, two server instances are created during startup.
During startup I use
services.AddHangfire((provider, hangfireConfig) => hangfireConfig
.UseDatabase(storageSettings.StorageProvider, storageSettings.ConnectionString, config)
.UseConsole());
services.AddHangfireServer(options => config.GetSection("HangfireSettings:Server").Bind(options));
Later I build
var app = builder.Build();
And finally add the dashboard
var dashboardOptions = config.GetSection("HangfireSettings:Dashboard").Get<DashboardOptions>();
dashboardOptions.Authorization = new[]
{
new HangfireCustomBasicAuthenticationFilter
{
User = config.GetSection("HangfireSettings:Credentials:User").Value,
Pass = config.GetSection("HangfireSettings:Credentials:Password").Value
}
};
app.UseHangfireDashboard(config["HangfireSettings:Route"], dashboardOptions);
Once I run the app (app.Run()) two server instances are created.
From the debug log I cannot see server with guid C7518906-D440-4113-85AD-0C1A97E01ACC, but the heartbeat is updated a few seconds before the instance shown in the log.
[09:01:20 INF] Starting Hangfire Server using job storage: 'SQL Server: xxxx'
[09:01:20 INF] Using the following options for SQL Server job storage: Queue poll interval: 00:00:01.
[09:01:20 INF] Using the following options for Hangfire Server:
Worker count: 5
Listening queues: 'default', 'default', 'notdefault', 'background', 'export', 'provisioning'
Shutdown timeout: 00:00:15
Schedule polling interval: 00:00:15
[09:01:20 DBG] Execution loop BackgroundServerProcess:6446fe1e has started in 14,3531 ms
[09:01:20 INF] Application started. Press Ctrl+C to shut down.
[09:01:20 INF] Hosting environment: NotDevelopment
[09:01:20 INF] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c successfully announced in 189,132 ms
[09:01:20 INF] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c is starting the registered dispatchers: ServerWatchdog, ServerJobCancellationWatcher, ExpirationManager, CountersAggregator, SqlServerHeartbeatProcess, Worker, DelayedJobScheduler, RecurringJobScheduler...
[09:01:20 DBG] Execution loop ServerHeartbeatProcess:bab9cda3 has started in 7,4454 ms
[09:01:20 DBG] Execution loop ServerWatchdog:d2946b36 has started in 7,503 ms
[09:01:20 DBG] Execution loop ServerJobCancellationWatcher:8325617f has started in 11,7685 ms
[09:01:20 DBG] Execution loop ExpirationManager:e95adc5c has started in 10,9894 ms
[09:01:20 DBG] Execution loop CountersAggregator:cedbdca3 has started in 12,2819 ms
[09:01:20 DBG] Removing outdated records from the 'AggregatedCounter' table...
[09:01:20 DBG] Execution loop SqlServerHeartbeatProcess:f57fa439 has started in 11,9128 ms
[09:01:20 DBG] Aggregating records in 'Counter' table...
[09:01:20 DBG] Execution loop Worker:e40656de has started in 12,6065 ms
[09:01:20 DBG] Execution loop Worker:aa196c8a has started in 18,7138 ms
[09:01:20 DBG] Execution loop Worker:f73ee2ec has started in 24,7742 ms
[09:01:20 DBG] Execution loop Worker:6c8a5c72 has started in 30,6857 ms
[09:01:20 INF] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c all the dispatchers started
[09:01:20 DBG] Execution loop Worker:21626abb has started in 37,1683 ms
[09:01:20 DBG] Execution loop DelayedJobScheduler:e419d23f has started in 12,6418 ms
[09:01:20 DBG] Execution loop RecurringJobScheduler:677d3276 has started in 12,5518 ms
[09:01:20 DBG] Removing outdated records from the 'Job' table...
[09:01:20 DBG] Removing outdated records from the 'List' table...
[09:01:20 DBG] Removing outdated records from the 'Set' table...
[09:01:20 DBG] Removing outdated records from the 'Hash' table...
[09:01:50 DBG] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c heartbeat successfully sent
[09:02:20 DBG] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c heartbeat successfully sent
[09:02:50 DBG] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c heartbeat successfully sent
[09:03:20 DBG] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c heartbeat successfully sent
[09:03:50 DBG] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c heartbeat successfully sent
[09:04:20 DBG] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c heartbeat successfully sent
[09:04:50 DBG] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c heartbeat successfully sent
[09:05:20 DBG] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c heartbeat successfully sent
[09:05:50 DBG] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c heartbeat successfully sent
[09:06:20 DBG] Aggregating records in 'Counter' table...
[09:06:20 DBG] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c heartbeat successfully sent
[09:06:50 DBG] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c heartbeat successfully sent
[09:07:20 DBG] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c heartbeat successfully sent
Here are the instances from the Hangfire.Server table:
Id Data LastHeartbeat
55b4c516-d5a9-4516-be31-29b9a4e83f1c
{"$type":"Hangfire.SqlServer.Entities.ServerData, Hangfire.SqlServer","WorkerCount":5,"Queues":{"$type":"System.String[], System.Private.CoreLib","$values":["default","default","notdefault","background","export","provisioning"]},"StartedAt":"2023-03-08T08:01:20.073434Z"}
2023-03-08 08:24:51.687
c7518906-d440-4113-85ad-0c1a97e01acc
{"$type":"Hangfire.SqlServer.Entities.ServerData, Hangfire.SqlServer","WorkerCount":5,"Queues":{"$type":"System.String[], System.Private.CoreLib","$values":["default","default","notdefault","background","export","provisioning"]},"StartedAt":"2023-03-08T08:01:32.2476057Z"}
2023-03-08 08:24:32.387
I've tested truncating all hangfire-related tables but also during a fresh run two instances are created.
I also tested writing an initializer where I deleted all, if any, servers before startup.
