0

I'm programming a tool that gathers images uploaded by a user into a zip-archive. For this I came across ZipArchiveAdapter from Flysystem that seems to do a good job.

I'm encountering an issue with the memory limit when the amount of files in the zip archive goes into the thousands.

When the amount of images for a user starts to go beyond a 1000 it usually fails due to the available memory being exhausted. To get to the point where it seems to handle most users with less than 1000 images I've increased memory limit to 4GB, but increasing it beyond this is not really an option.

Simplified code at this point: 

<?php
use League\Flysystem\Filesystem;
use League\Flysystem\ZipArchive\ZipArchiveAdapter;
use League\Flysystem\Memory\MemoryAdapter;

class User {
    // ... Other user code

    public function createZipFile()
    {
        $tmpFile = tempnam('/tmp', "zippedimages_");
        $download = new FileSystem(new ZipArchiveAdapter($tmpFile));

        if ($this->getImageCount()) {
            foreach ($this->getImages() as $image) {
                $path_in_zip = "My Images/{$image->category->title}/{$image->id}_{$image->image->filename}";
                $download->write($path_in_zip, $image->image->getData());
            }
        }
        $download->getAdapter()->getArchive()->close();
        return $tmpFile;

        // Upload zip to s3-storage
    }
}

So my questions: a) Is there a way to have Flysystem write to the zip-file "on the go" to disk? Currently it stores the entire zip in memory before writing to disk when the object is destroyed.

b) Should I utilize another library that would be better for this?

c) Should I take another approach here? For example having the user download multiple smaller zips instead of one large zip. (Ideally I want them to download just one file regardless)

user1015149
  • 187
  • 1
  • 10
  • 1
    if it uses the base ZipArchive ( built in PHP class) then you can pull files out by using [getStream](https://secure.php.net/manual/en/ziparchive.getstream.php) which may help with the unzipping – ArtisticPhoenix Apr 25 '18 at 17:27
  • Hi, it's the process of packing the zip file that I have a problem with. Unpacking is not an issue here, as that is something the user will do. Or do I misunderstand you? – user1015149 Apr 25 '18 at 17:30
  • As far as writing all I ever use for zips is the built in PHP class, but how [ziparchive::addfile](https://secure.php.net/manual/en/ziparchive.addfile.php) works under the "hood" I have no idea. Which means it may or may not perform better then this adapter your using that is probably a wrapper of it. But you never actually said what framework your using. – ArtisticPhoenix Apr 25 '18 at 17:33
  • So you say you're having trouble with memory while using the MemoryAdapter... how strange. – Sammitch Apr 25 '18 at 17:56
  • MemoryAdapter is not actually something I use myself in the code. This is not any particular framework, I just use what I need. – user1015149 Apr 25 '18 at 18:35
  • there are 2 things a good zip class utilizing ziparchive should do, use `addFile` not `addFromString` and after a few hundred files the zip archive should be closed (all the files stored in memory at written to disk) and reopened (thus freeing memory). i haven't looked at flysystem's zip system recently, but last time I was non-to-impressed. I'd suggest writing your own adapter. I realize this may be to late to be helpful. – Alex Aug 23 '19 at 07:27

0 Answers0