0

We have configured Sulu to store uploaded media files in S3, however when various image sizes and formats are generated (according to the image formats specified in /config/image-formats.xml) the resulting files are stored on the local server in /public/uploads/media

How should we configure it to store the generated variants on S3 also?

This is config/packages/sulu-media.yaml:

services:
    # Overwrite adapter creation due to error: "mime_content_type(): stream does not support seeking"
    # https://github.com/sulu/sulu/issues/5468
    sulu_media.storage.s3.adapter:
        class: League\Flysystem\AwsS3v3\AwsS3Adapter
        arguments:
            - '@sulu_media.storage.s3.client'
            - '%sulu_media.media.storage.s3.bucket_name%'
            - '%sulu_media.media.storage.s3.path_prefix%'
            - []
            - false

sulu_media:
    storage: local
    storages:
        s3:
            key: '%env(AWS_S3_KEY)%'
            secret: '%env(AWS_S3_SECRET)%'
            bucket_name: '%env(AWS_S3_BUCKET)%'
            path_prefix: 'sulu-media/%kernel.environment%'
            region: '%env(AWS_REGION)%'
            arguments:
                use_path_style_endpoint: true
                bucket_endpoint: false

This is config/packages/prod/sulu_media.yaml

sulu_media:
    storage: s3

jamiemax
  • 179
  • 13

1 Answers1

0

Short Answer

Image formats are not saved in S3 because they are generated on request and S3 and the other media storages do not work as a reverse proxy. So you will need to configure an own reverse proxy to cache the generated images and disable the local saving of the image format.

Long Answer

Image Format and the Media Storage are different things in Sulu. Where a fast response when Downloading a PDF file or a Original Big Image file is not necessary. A image which is embedded in the website needs a fast response time.

The most important note about image formats in Sulu is they are generated when requested. So if you upload a image there are no image formats generated. A image format is only generated when really used. This means also a local url where Sulu is running need to be called to generate the image format.

So if we embed the following in our twig:

<img src="{{ media.thumbnails['50x50'] }}" alt="{{ media.title }}">

As in the example the /uploads/media/50x50/1-image.jpg url is called on your webserver. Your webserver will then look for that file in public/uploads/media/50x50/1-image.jpg if it does not exist public/index.php will be called where sulu is running. This requested the sulu media stream controller and will generate the request image format. This process will also generate then the image format into public/uploads/media/50x50/1-image.jpg. So the next time the image format url is called the webserver will directly return the image from the filesystem. This is some kind of a really low level reverse proxy mechanism.

Here a image representation:

enter image description here

So we are now at the important keyword "reverse proxy". As S3 does not work as a reverse proxy or as reverse proxy cache, S3 config does not have any effect on the image formats. So if you just have a multi server setup, this is mostly no problem as the image formats will on every server be generated and served fast. But if you are in a cloud setup where you would lose this local generated files you need to find another way how to cache this images, else a restart of the server could kill the server when generate again every image formats.

Some cloud provider are here very simple to be configurable. For example in google cloud you just set that you want for /uploads/media CDN then the images are cached in the google CDN cache and so the sulu local cache can be disable the following way:

sulu_media:
    format_cache:
         save: false

!!Important never disable this cache if you are not sure if you have a other reverse proxy configured, as mention above triggering always the image generate could kill your server.

As you mention Amazon S3, I think you maybe can configured some reverse cache for uploads/media using cloudflare or the CDN service which amazon provides. There are also other third party services like enter link description here which are build for this kind of caching mechanism.

You maybe know ask yourself how can I self cache the images, without a external services. Mostly the easiest way here is just use a nginx configured with a reverse proxy mechanism and cache.

proxy_cache_path /var/nginx/cache levels=1:2 keys_zone=my_cache:16m max_size=16g inactive=180d use_temp_path=off;

server {
    root /usr/share/nginx/html;

    location / {
        index index.html;
    }

    location /uploads/media {
        proxy_cache my_cache;
        proxy_pass {{ backend.uri }};
        proxy_ignore_headers "Set-Cookie";
        proxy_ignore_headers Cache-Control;
        proxy_cache_valid 200 365d;

        add_header X-Cache-Status $upstream_cache_status;

        add_header Cache-Control "public, max-age=31536000, immutable";
   }
}
Alexander Schranz
  • 2,154
  • 2
  • 25
  • 42
  • Thanks, that's helpful. We're going with the option of pointing cloudfront to our application and allowing it to cache the generated media files – jamiemax Feb 03 '21 at 12:56