Extract Vips image and save it to numpy array same as with PIL?

Question

I have looked through some of the documentation available Here for Vips but have not found an answer yet.

I want to put the image into a numpy 3D array similar to how PIL images automatically get processed this way:

In[1]:  import numpy
In[2]:  from PIL import Image
In[3]:  image = Image.open('43.jpg')
In[4]:  image
Out[4]: <PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=3216x2136 at 0x7F0C8D8B9950>
In[5]:  imgArray2 = numpy.asarray(image, dtype=numpy.float32)
In[6]:  imgArray2.shape
Out[6]: (2136, 3216, 3)

.

This is what I have for Vips so far...

In[1]:  import numpy
In[2]:  from gi.repository import Vips
In[3]:  image = Vips.Image.new_from_file('43.jpg')
In[4]:  image
Out[4]: <Image object at 0x7f0c9a66c5f0 (VipsImage at 0x338a190)>
In[5]:  imgArray2 = numpy.asarray(image, dtype=numpy.float32)
Out[5]: ValueError: setting an array element with a sequence.

So I get that error at the end because I'm not pulling the data in the right format from the Vips Image object.

Answer 1

You need .write_to_memory() and .new_from_memory(). The C docs are here:

http://www.vips.ecs.soton.ac.uk/supported/8.0/doc/html/libvips/VipsImage.html#vips-image-write-to-memory

http://www.vips.ecs.soton.ac.uk/supported/8.0/doc/html/libvips/VipsImage.html#vips-image-new-from-memory

You use them in Python like this:

$ python
Python 2.7.9 (default, Apr  2 2015, 15:33:21)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from gi.repository import Vips
>>> x = Vips.Image.new_from_file("/home/john/pics/k2.jpg")
>>> y = x.write_to_memory()
>>> type(y)
<type 'str'>
>>> z = Vips.Image.new_from_memory(y, x.width, x.height, x.bands, Vips.BandFormat.UCHAR)
>>> x.avg()
102.79218031609196
>>> z.avg()
102.79218031609196

So that's sending the JPEG file off to a string, then creating a new image back from the string again. The string is just the sequence of bytes which make up the image values, so for an 8-bit RGB image like this, it will start with:

R1 G1 B1

where R1 is the value of the red channel for the left-left pixel in the image. Pixels are stored as a series of scanlines, top to bottom.

Does numpy need float pixels? You can ask vips to generate float pixels with x.cast(Vips.BandFormat.FLOAT).write_to_memory().

Answer 2

Expanding on the answer from user894763

from gi.repository import Vips
from PIL import Image
import numpy as np

with open(path_or_url) as image_file:

  start_pillow = time.time()
  pillow_img = np.asarray(Image.open(image_file))
  print('Pillow Time:', time.time()-start_pillow)
  print('original shape', pillow_img.shape)

  start_vips = time.time()
  img = Vips.Image.new_from_file(path_or_url)
  print('Image bit depth', img.Bbits)
  mem_img = img.write_to_memory()

  # Note that my specific image was 8 bit
  np_3d = np.fromstring(mem_img, dtype=np.uint8).reshape(img.width, img.height, 3)

  print('Vips Time:', time.time()-start_vips)
  print('final shape', np_3d.shape)

  # Just to verify we have the same result
  print('Sum of the Differences:', np.sum(np_3d-pillow_img))

output:

('Pillow Time:', 0.3100590705871582)
('original shape', (2500, 2500, 3))
('Image bit depth', 8)
('Vips Time:', 0.1401970386505127)
('final shape', (2500, 2500, 3))
('Sum of the Differences:', 0)

This was on a 2500x2500 8bit color .jpg image.