Django threading behind IIS and wfastcgi

Question

I have a Django application that executes a lengthy task, that is controlled by Ajax on one of the app's pages. The first Ajax call creates an entry in a task table in the db, and then creates a thread that executes that task, by calling a Java app via Py4J. This sends a command to a legacy server and then regularly queries it for status and updates the task table until the task is complete.

This tasks takes about 10/15 minutes and works perfectly via manage.py runserver.

However in deployment, I have installed it behind IIS with wfastcgi, and the task is launched correctly but after a little while, minutes, it seems to stop. The task in the db is not updated and there are no exceptions logged or anything. As if IIS or wfastcgi killed the thread without warning. Note that it is an internal tool that will be used at most by 4/5 people so load is not really an issue.

Is there anything I have missed or that could be done?

The code is as follows:

# Get a PyFJ command
def getMCLCmd():
    gateway = JavaGateway()
    try:
        cmd = gateway.entry_point.getMCLCmd()
        return cmd
    except Py4JNetworkError as e:
        # maybe server not started
        logger.info('Py4J not started, starting it')
        pyserver = py4JVM()
        pyserver.start()

    # try again
    try:
        cmd = gateway.entry_point.getMCLCmd()
        return cmd
    except Py4JNetworkError as e:
        logger.debug('Py4J error' + e)
        return None


class py4JVM(threading.Thread):
   def __init__(self):
        threading.Thread.__init__(self, daemon=True)
        self.error = None

   def run(self):
        try:
            process = subprocess.run(MCL_CMD, shell=True, check=True, stdout=subprocess.PIPE, universal_newlines=True)
            output = process.stdout
            logger.info(f'Py4J output: {output}')
        except subprocess.CalledProcessError as e:
            self.error = f"command return with error (code {e.returncode}): {e.output}"
            logger.error(self.error)


# the thread class
class FirstInitThread(threading.Thread):
    def __init__(self, task: ClientTask):
        self.task = task
        threading.Thread.__init__(self)

    def run(self):
        cmd = getMCLCmd()
        if cmd is None:
            self.task.status = 0
            self.task.status_msg = 'Cannot start Py4J'
            self.task.save()
            logger.error('Cannot start Py4J')
            return

        self.task.status = 1
        try:
            if cmd.sendCommand(self.task.config):
                while cmd.isRunning():
                    time.sleep(1)
                    msg = cmd.getNextMessage()
                    if msg:
                        logger.info(msg)
                        self.task.status_msg = msg
                        self.task.save()
                    else:
                        self.task.status_msg = self.task.status_msg + '.'
                        self.task.save()
                self.task.status = 2
                self.task.save()

        except Py4JJavaError as e:
            self.task.status = 2
            self.task.status_msg = 'Client error'
            logger.error('Client error')
            self.task.save()

        except Exception as e:
            self.task.status = 2
            self.task.status_msg = 'Thread stopped: ' + str(e)
            logger.error(self.task.status_msg )
            self.task.save()

The view to launch the process is

@csrf_exempt
def remoteConfig(request: HttpRequest):
    ... creation of config object
    task = ClientTask()
    task.config = config
    task.status_msg = 'Initialisation'
    task.status = 0
    task.save()

    thread = FirstInitThread(task)
    thread.start()
    return HttpResponse(str(task.pk))

And then to update the HTML page with Ajax we call:

@csrf_exempt
def initProgress(request: HttpRequest):
    if request.method != 'POST':
        return HttpResponse("Invalid")

    # construct the client config
    task_id = request.POST.get('task_id')
    task: Task = Task.objects.get(pk = task_id)
    if task is None:
        return HttpResponse('Invalid Task')
    
    if task.status == 2:
        return HttpResponse('Completed: ' + msg + '')
    
    return HttpResponse(task.status_msg)