Implement HealthChecks for long running process .Net Core

Question

I have a long-running process: IHostedService. It runs all day long calling different external apis (services) to pull in data. I would like to get notified if during this process any of these external services failed/exceptions got thrown, the process is taking longer than 30 min etc. How would I set it up?

After some research, I ended up with this: https://learn.microsoft.com/en-us/aspnet/core/host-and-deploy/health-checks?view=aspnetcore-5.0#separate-readiness-and-liveness-probes (StartupHostedServiceHealthCheck section)

How do I implement .NET Core Health Checks on a Hosted Service?

This is what I have so far:

// Registered as Singleton
public interface IHostedServiceStatus
{
    bool IsHostedServiceRunning { get; set; }
    DateTime RunTime { get; }
    string Message { get; }

    void Setup(string name, DateTime runTime, string message = null);
}

public class HostedServiceStatus : IHostedServiceStatus
{
    private string _message;
    private string _name;

    public string Message => $"HostedService {_name} started on: {RunTime} failed to complete. {_message}";
    public bool IsHostedServiceRunning { get; set; }
    public DateTime RunTime { get; private set; }

    public void Setup(string name, DateTime runTime, string message = null)
    {
        _name = name;
        RunTime = runTime;

        IsHostedServiceRunning = true;

        if (!string.IsNullOrEmpty(message))
            _message = message;
    }
}

// HealthCheck
public class HostedServiceHealthCheck : IHealthCheck
{
    private readonly IHostedServiceStatus _hostedServiceStatus;
    private readonly TimeSpan _duration = TimeSpan.FromMinutes(30);

    public HostedServiceHealthCheck(IHostedServiceStatus hostedServiceStatus)
    {
        _hostedServiceStatus = hostedServiceStatus;
    }

    public Task<HealthCheckResult> CheckHealthAsync(
        HealthCheckContext context,
        CancellationToken cancellationToken = default)
    {
        if (_hostedServiceStatus.IsHostedServiceRunning)
        {
            if (_hostedServiceStatus.RunTime.Subtract(DateTime.Now).Duration() >= _duration)
                return Task.FromResult(
                    HealthCheckResult.Unhealthy(_hostedServiceStatus.Message));
        }

        return Task.FromResult(
            HealthCheckResult.Healthy("Task is finished."));
    }
}


// Long running process
public async void Process(object state)
{
    // each service runs about 10 min
    foreach (var externalService in externalServices)
    {
        try
        {
            _hostedServiceStatus.Setup(externalService.Name, DateTime.Now); // setup healthcheckStatus - injected
            ...
            // calls externalService gets data and saves to db
            
            
            _dataMinerStatus.IsHostedServiceRunning = false; // udpate Healthcheck - finished successfully
        }
        catch (Exception ex)
        {
            // set MInDateTime so that it becamse UnHealthy
            _hostedServiceStatus.Setup(externalService.Name, DateTime.MinValue); 
            // HealthCheck injected
            await _healthCheck.CheckHealthAsync(new HealthCheckContext()); // send notification? (webhook setup) - will this work?
        }
}

The section you link to essentially says `create a static bag of status properties and return a status based on those properties`. That's not *the* way of checking the health status of a service, it's just *a* way, perhaps the simplest one. To track external services it's probably bette to use other healthchecks like `AddUrlGroup`. There's no reason to use only a single property either. Use a `duration` property only to track the duration of calls. Use *separate* properties to handle errors for the various services — Panagiotis Kanavos, Apr 14 '21 at 14:07
You'll have to reset the status of the service status fields. I suspect each field will need different logic. Some could be reset the next time they succeed. Some may be reset after some time — Panagiotis Kanavos, Apr 14 '21 at 14:10
how would I add multiple healthchecks dynamically? so for example I could have a list of healthchecks. instead of doing registration one by one — ShaneKm, Apr 14 '21 at 14:50

Implement HealthChecks for long running process .Net Core

0 Answers0