Laravel filesystem sftp cached adapter

Question

I am struggling with this issue for some time. I am using the sftp adapter to connect to another server where i read/write files a lot.

For thumbnail creation i use background jobs with laravel horizon to retrieve pdf contents from the remote sftp server and then generate a jpg and place in local filesystem.

For first setup i need to make around 150k of thumbnails. When i use a lot of processes in horizon the remote server can't handle this number of connections.

I must limit to max 2 processes at the moment (10 secs~ * 150k~) not optimal.

I want to cache the connection because i know it is possible and probably solves my problem, but can't get it to work:(

The only reference/tutorial/example/docs i could find is

https://medium.com/@poweredlocal/caching-s3-metadata-requests-in-laravel-bb2b651f18f3 https://flysystem.thephpleague.com/docs/advanced/caching/

When i use the code from the example like this:

Storage::extend('sftp-cached', function ($app, $config) {
    $adapter = $app['filesystem']->createSftpAdapter($config);
    $store = new Memory();

    return new Filesystem(new CachedAdapter($adapter->getDriver()->getAdapter(), $store));
});

I get the error: Driver [] is not supported.

Is there anyone here who can help me a bit further on this?

score 3 · Accepted Answer · answered Jul 21 '18 at 16:32

It appears necessary to adjust your configuration:

In your config/filesystems.php file, add a 'caching' key to your storage:

'default' => [
    'driver' => 'sftp-cached',

    // ...

    'cache' => [
        'store' => 'apc',
        'expire' => 600,
        'prefix' => 'laravel',
    ],
],

This example is based on official documentation (https://laravel.com/docs/5.6/filesystem#caching), but it is not described well how the 'store' key is used here (where memcached is the example), and you would need to change the implementation of your driver to new Memcached($memcached); (with an instance to inject) instead.

In your case, since the sftp-cached driver implements $store = new Memory();, the cache config must reflect this with 'store' => 'apc' (which is RAM based cache). Available 'store' drivers are found in config/cache.php.

(If you use APC and get an error message Call to undefined function Illuminate\Cache\apc_fetch(), this PHP extension must be installed, see e.g. http://php.net/manual/en/apcu.installation.php)

Finally, I believe the 'prefix' key in config/filesystems.php must be set to the same as the cache key prefix in config/cache.php (which is 'prefix' => 'cache' by default).

Laravel 9's [`Flysystem`](https://laravel.com/docs/9.x/upgrade) no longer supports "cached adapters", what now? — Top-Master, Mar 11 '23 at 10:07
The "cached adapters" feature would just cache responses, meaning only repeated requests to the same file would get faster, hence OP's 150K requests would remain as slow as before, right? — Top-Master, Mar 11 '23 at 10:11

Laravel filesystem sftp cached adapter

1 Answers1