Create a montage of 15000 images

Question

I am trying to create a collage/montage of 15000 images using imagemagick montage command.The tool works perfectly for a small subset of the images but when trying to create a montage using 15K images the program crashes because it can't just load 15K images into main memory. I think opening the files in streams and then stacking in batches of 100 would work but if someone has a nice solution,it would be helpful.

Answer 1

libvips can make huge image montages without needing a lot of memory. For example:

$ vips arrayjoin "a.png b.png c.png d.png" output.png --across 2

This will read the four PNG images, lay them out in rows in a 2 x 2 grid, and write to out.png. You can use $() to find lots of images and sort by filename, perhaps:

$ vips arrayjoin "$(ls *.jpeg | sort -t_ -k2g -k1g)" x.tif --across 20

That will load all JPEG images with filenames of the form x_y.jpeg, sort them numerically by row and then by column, and assemble them into a 20 x N grid.

15,000 images is too much for the command-line on linux (you'll get an Argument list too long error from bash), but you can do it from python with pyvips. Perhaps:

#!/usr/bin/python3

import sys
import glob
import pyvips

images = [pyvips.Image.new_from_file(filename, access="sequential")
          for filename in glob.glob(sys.argv[1] + "/*.jpg")]

joined = pyvips.Image.arrayjoin(images, across=100)

joined.write_to_file(sys.argv[2])

Then:

$ vips crop ~/pics/k2.jpg x.jpg 0 0 1000 1000
$ for i in {1..15000}; do cp x.jpg $i.jpg; done
$ VIPS_CONCURRENCY=1 /usr/bin/time -f %M:%e \
    ./arrayjoin.py . x.tif[compression=jpeg,tile]
4689264:78.38
$ vipsheader x.tif
x.tif: 100000x150000 uchar, 3 bands, srgb, tiffload

So that's assembling 15,000 1,000 x 1,000 pixel JPG images to make a 100,000 x 150,000 pixel JPEG-compressed TIFF. It takes 80s and needs a peak of 4.7gb of memory.

Answer 2

It's a bit late here to write and test anything, but I'd probably do something like this:

• generate a text file with all the filenames to montage
• use split command to break that into groups of 100
• iterate over all the groups making a row of images from each
• stack the rows

---

First part is like:

find . -name "*.jpg" -print > filelist.txt

Next part is like:

split -l 100 filelist.txt ROWS

Next part is like:

n=0
for f in ROWS* ; do
    magick @"$f" +append row${n}.jpg
    ((n=n+1))
done

Last part is like:

magick row*.jpg -append result.jpg