Multiple apps running on Docker process files in shared directory

Question

I have two or more PHP apps running on Docker that process files in a shared directory. So I need a lock system to prevent those apps working on the same file. Ex: there are 3 files in the directory, if app1 works on file1, app2 have to skip this file1 and pass to file2. I have a script like this in the apps:

// $files is a list of all files in the directory
foreach($files as $file) {
    $this->performFile($file);
    $this->moveFileToHistory($file);
}

I tried to use file or directory as lock so for example if a file1.lock(file or directory) exists it is locked.

$dir = md5($file) . 'lock';

if (@mkdir($dir, 0700)) {
    $this->performFile($file);
    $this->moveFileToHistory($file);
    rmdir($dir);
}

and it works but in case of crash or interruption I think that it is better to use something that can be cleaned or released even if PHP itself crashes.So I was thinking about:

semaphore, but it doesn't work as each app runs on Docker the process is isolated, right?
open a socket on a port and if this port is already open, then it is locked. But we have several files to process and the number of the file is not fixed.How can we handle that?

Is there a better way to go than using file or directory in this case?

Redis is often used for locking across multiple containers. – Hans Kilian Aug 09 '21 at 11:55 — Hans Kilian, Aug 09 '21 at 11:55
Thanks @HansKilian, I'll dig into it – Yoyo Aug 09 '21 at 13:17 — Yoyo, Aug 09 '21 at 13:17

score 0 · Accepted Answer · answered Aug 11 '21 at 06:34

I finally came up with a solution. There is this component symfony/lock from symfony that you can use for managing locks. You can define the expiration time that fits exactly my needs. Here an example:

foreach($files as $file) {
  // the lock expires after 60s if $lock->release() is not called due to a randomly reason.
  $lock = $this->lockFactory->createLock(md5($file), 60);
  if (!$lock->acquire()) {
    continue;
  }

  try {
    $this->performFile($file);
    $this->moveFileToHistory($file);
  } finally {
    $lock->release();
  }
}

You can use those available stores to manage locks.

Multiple apps running on Docker process files in shared directory

1 Answers1