Laravel job runs multiple times when executing long task

Question

I'm currently trying to convert a video into multiple .ts segments (for HTTP video streaming). This is a long task so I'm doing this in a Laravel job.

The problem is that the job run multiple times after a few minutes and because of that, the video is processed multiple times. Here you can see my Job class:

<?php

namespace App\Jobs;

use App\Models\Media\MediaFile;
use FFMpeg\Filters\Video\ResizeFilter;
use FFMpeg\Format\Video\X264;
use Illuminate\Bus\Queueable;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Support\Facades\Log;
use Illuminate\Support\Str;
use FFMpeg;

class ConvertVideoForStreaming implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;


    public $mediaFile;
    public $deleteWhenMissingModels = true;
    public $timeout = 3600;
    public $retryAfter = 4000;
    public $tries = 3;

    /**
     * Create a new job instance.
     *
     * @return void
     */
    public function __construct(MediaFile $mediaFile)
    {
        $this->mediaFile = $mediaFile;
    }

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        $this->mediaFile->update([
            'status' => 'Processing',
            'status_message' => null,
        ]);


        $convertingId = Str::random(6);


        $lowBitrateFormat  = (new X264('aac', 'libx264'))
            ->setKiloBitrate(500)
            ->setAudioKiloBitrate(128);
        $midBitrateFormat  = (new X264('aac', 'libx264'))
            ->setKiloBitrate(1500)
            ->setAudioKiloBitrate(192);
        $highBitrateFormat = (new X264('aac', 'libx264'))
            ->setKiloBitrate(3000)
            ->setAudioKiloBitrate(256);

        \Log::info('Started a new converting process with convertingID: ' . $convertingId);

        FFMpeg::fromDisk('public')
            ->open($this->mediaFile->file)
            ->exportForHLS()
            ->setSegmentLength(4) // optional
            ->addFormat($lowBitrateFormat, function($media) {
                $media->addFilter(function ($filters) {
                    $filters->resize(new \FFMpeg\Coordinate\Dimension(640, 480), ResizeFilter::RESIZEMODE_INSET);
                });
            })
            ->addFormat($midBitrateFormat, function($media) {
                $media->addFilter(function ($filters) {
                    $filters->resize(new \FFMpeg\Coordinate\Dimension(1280, 720), ResizeFilter::RESIZEMODE_INSET);
                });
            })
            ->addFormat($highBitrateFormat, function($media) {
                $media->addFilter(function ($filters) {
                    $filters->resize(new \FFMpeg\Coordinate\Dimension(1920, 1080), ResizeFilter::RESIZEMODE_INSET);
                });
            })
            ->onProgress(function ($percentage) use($convertingId)  {
                \Log::info($this->mediaFile->name . " (ConvertingID: $convertingId) - $percentage % transcoded");
                \Redis::set('video-transcoded-' . $this->mediaFile->id, $percentage);
            })
            ->toDisk('public')
            ->save('livestream/' . Str::slug($this->mediaFile->name) . '/playlist.m3u8');

        $this->mediaFile->update([
            'status' => 'Active',
            'status_message' => null,
        ]);

    }


    public function failed(\Exception $exception)
    {
        $this->mediaFile->update([
            'status' => 'Failed',
            'status_message' => $exception->getMessage(),
        ]);
    }

}

How can I solve my problem?

Could you let us know how you solved the problem? – Grant Aug 23 '21 at 12:51 — Grant, Aug 23 '21 at 12:51

score 0 · Answer 1 · answered Aug 06 '19 at 16:48

Laravel offers a 'lock' feature for scheduled commands. See the 'Prevent Task Overlaps' in this documentation.

In case you'd like to have non-scheduled jobs being processed only once I'd advise you to look into the Symfony Lock Component. This component offers you to lock a certain task for a period of time or until it's unlocked. This way you can do something along the lines of:

At the start of your handle() method
a. check if a lock already exists, otherwise skip this job
b. If the lock does not exist, create it
Execute your long running task
At the end of your handle() logic release the lock

Laravel job runs multiple times when executing long task

1 Answers1