0

ImageMagick's 'compare' seems to provide irrelevant numbers when using SSIM as the comparison metric. I'm using ImageMagick 7.0.8-58 Q16.

I tried it with various levels of WebP compression (including lossless)

magick compare -metric SSIM original.png lossless.webp difference.png

provides 0.734809. While I would expect something around 0.999 for a visually indistinguishable image (apparently no artifacts, because it's a WebP file compressed in lossless mode)

Receiving such results, I assumed that there is a problem with comparing PNG to WebP specifically, so I tried to compare original PNG to compressed JPEG with a target SSIM. SSIM package from Fred's ImageMagick Scripts gave results of ssim=0.949 and dssim=0.051. ImageMagick on contrary gave 0.711049 result for the very same file.

magick compare -metric SSIM original.png compressed.jpg difference.png

Sadly, Fred's SSIM doesn't seem to work with WebP and GraphicsMagick's compare doesn't support SSIM. So how do I get an accurate SSIM from ImageMagick or what are the other ways to get SSIM for WebP files?

fmw42
  • 46,825
  • 10
  • 62
  • 80
vptest
  • 246
  • 1
  • 17

2 Answers2

1

Do you have webp installed as a delegate to ImageMagick? If so, it should show in the delegates list from magick -version. What is your platform/OS?

My ssim script seems to work for me on IM 7.0.8.59 Q16 Mac OSX

magick rose: rose.webp
ssim rose.webp rose.webp

ssim=1 dssim=0


Also ImageMagick compare works fine for me also.

magick compare -metric ssim rose.webp rose.webp null:

1


I believe that lower values for webp to png may have to do with the webp lossy compression. I get similar values from ImageMagick and from my script:

magick rose: rose.webp
magick rose: rose.png

magick compare -metric ssim rose.webp rose.png null:
0.895189

ssim rose.webp rose.png
ssim=0.895 dssim=0.105


However, if I use lossless compression, then I get a perfect comparision:

magick rose: -define webp:lossless=true rose.webp
magick rose: rose.png

ssim rose.webp rose.png
ssim=1 dssim=0

magick compare -metric ssim rose.webp rose.png null:
1

ADDITION:

For me, it does not matter which order the webp image is in the command line.

Here, I use lossless compression.

magick rose: rose.png
magick rose: -define webp:lossless=true rose.webp

magick compare -metric ssim rose.webp rose.png null:
1
1magick compare -metric ssim rose.png rose.webp null:
1

Here, I do not use lossless compression.

magick rose: rose.png
magick rose: rose.webp

magick compare -metric ssim rose.webp rose.png null:
0.895189
magick compare -metric ssim rose.png rose.webp null:
0.895189
fmw42
  • 46,825
  • 10
  • 62
  • 80
1

Actually the answer is pretty simple. I used a bugged version of ImageMagick. Just changing the order of comparison did the trick:

magick compare -metric SSIM compressed.jpg original.png difference.png
vptest
  • 246
  • 1
  • 17
  • The order of the images is not important and should not make a difference. If it does, then please report this to the Bugs forum at https://imagemagick.org/discourse-server/. Your test here does not include any webp image, just jpg and png? – fmw42 Aug 05 '19 at 16:26
  • I tested with several images, including webp. And yes, it does make a difference for me (apparently it shouldn't) and it works for all supported images: webp, jpeg, png etc. If compressed image is provided first in command line, for some reason it shows the right result. If second - it shows random result. Looks like a bug in ImageMagick. I will update it and report it as a bug unless it gets fixed. – vptest Aug 06 '19 at 17:08
  • See my addition to my answer. For me it does not matter which order for the webp image. But do post a bug report as it could be version or OS dependent. – fmw42 Aug 06 '19 at 18:20