Up scaling image size downloaded by scrapy image pipeline

Question

I'm trying to overwrite the convert_image method in my pipeline.py from the herited ImagesPipeline class but it does not work as expected.

Actully i'm just trying to upscale the image downloaded to my requirements: 700px but the downloaded images still the original size also i tested the resize fonction outside scrapy and it works well

For info i'm not using in my setting IMAGES_THUMBS so size should be None and IMAGES_EXPIRES = 0

If someone have a good solution to get downloaded image directly to 700x700 minimun conversion when it does not meet this requirements.

Here is my code:

class MyImagesPipeline(ImagesPipeline):

    def get_media_requests(self, item, info):
        for image_url in item['image_urls']:
            yield scrapy.Request(image_url)

    def convert_image(self, image, size=None):
        if image.format == 'PNG' and image.mode == 'RGBA':
            background = Image.new('RGBA', image.size, (255, 255, 255))
            background.paste(image, image)
            image = background.convert('RGB')
        elif image.mode != 'RGB':
            image = image.convert('RGB')
        if size is None:
            image = image.copy()
            basewidth = 700
            wpercent = (basewidth/float(image.size[0]))
            hsize = int((float(image.size[1])*float(wpercent)))
            image.resize((basewidth,hsize), Image.ANTIALIAS)
            #image = image.copy()
            #image.thumbnail(size, Image.ANTIALIAS)

       buf = BytesIO()
       try:
            image.save(buf, 'JPEG')
       except Exception, ex:
            raise ImageException("Cannot process image. Error: %s" % ex)

       return image, buf

    def item_completed(self, results, item, info):
        image_paths = [x['path'] for ok, x in results if ok]
        if not image_paths:
            raise DropItem("Item contains no images")
        item['image_paths'] = image_paths
        return item

here is the original pipeline image class i'm trying to overide: github

score 0 · Accepted Answer · answered Mar 07 '16 at 16:22

0

It looks like you're using PIL or Pillow. resize doesn't alter the image in-place, it returns a new image. So you need this:

image = image.resize((basewidth,hsize), Image.ANTIALIAS)
^^^^^^^^

answered Mar 07 '16 at 16:22

Mark Ransom

299,747
42
398
622

So many thanks. I understand now how pillow make the workflow. I did not noticed that. when i port my basic batch script to resize... For info i made the change inside the def get_images method because i didn't want to change the logic of converted image. – Andronaute Mar 07 '16 at 16:50
"So many thanks": In addition to nice words, you could also upvote Mark's answer. – Steven Almeroth Mar 08 '16 at 18:29

Up scaling image size downloaded by scrapy image pipeline

1 Answers1