5

I have a conceptual question regarding background tasks in Django. I am running a Python/Django site through PythonAnywhere.

I have a webapp called "databooks" which is comprised of a form that has the following entries:

  1. Databook Name
  2. Folder Location/Link (which is on my PythonAnywhere file system)

This databooks app then looks at the folder location and compiles all the miscellaneous files within to create one large PDF. However, due to the nature of merging pages and adding footers, etc, it is considered a "heavy process" by PythonAnywhere, which they define as a web worker that exceeds 5 minutes and which they respectively kill.

My solution in mind would be to execute a background-python script after the form is submitted, which inputs the entires from the databooks views.py file into a backgrounddatabookgenerator.py file which can run independent of what a user does in the browser window.

After looking at PythonAnywhere's diagnosis of this, I have been researching some options, but have been unsuccessful so far in implementing a background task (i.e., django-background-tasks).

Is anyone familiar with a Django workflow I could implement to call another python file as a background task after a submit button is clicked? In doing so, I would like to make this background task independent of what a user does after clicking submit and to allow the heavy-process of databook generation to complete itself, regardless of time.

views.py

def Databook_req(request):
    submitted = False
    if request.method == 'POST':
        form = DatabookForm(request.POST, request.FILES)
        What_would_you_like_to_save_the_completed_data_book_as = request.POST['What_would_you_like_to_save_the_completed_data_book_as']
        Major_Equipment_Folder = request.POST['Major_Equipment_Folder']
        if form.is_valid():
            data_dict = {
                'What_would_you_like_to_save_the_completed_data_book_as_': str(What_would_you_like_to_save_the_completed_data_book_as),
                'Major_Equipment_Folder': str(Major_Equipment_Folder)
            }
            form.save()
            DataBookName = str(What_would_you_like_to_save_the_completed_data_book_as) + '.pdf'
            original_path = str(Major_Equipment_Folder)

***Then the databook code follows this by referring to the above variables. I am unsure if I need a separate python file to run everything below as a background task, but as is it is currently timing out.

JcoOnline
  • 243
  • 2
  • 13

2 Answers2

2

Thank you for the feedback. django-background-tasks , the recommendation provided by PythonAnywhere, was able to provide a solution. By queuing the heavy-processing merger function of the request object (which is the last callable of the object), the request was completed and the merger initiated and completed regardless of any action taken on the front-end after submission.

# Merge the PDFs and add Outline/Links
@background(queue='my-queue')
def merger(DataBookName, folders, x, pieceFiles, original_path, allFolders, other_files, others_path, count):
JcoOnline
  • 243
  • 2
  • 13
1

Notice: background decorator place limits on the parameters that can be passed when calling the function - they must all be serializable as JSON, while your request object is not JSON-serializable. so, don't pass request object to your task function. Define your background task function outside your view function and pass only the parameters that json-serializable, invoke the function in your views.One major disadvantage of using django-background-tasks is that it may block the main program.

Another way of solving this problem is spawning a new task thread for generating the file, while using ajax to query the status.

minglyu
  • 2,958
  • 2
  • 13
  • 32